Standard Libraries and Test Automation

Problem
You need to develop a suite of automated test cases for an application you own. You dig in and start writing all the functionality required to build up a robust test suite. Are you proceeding wisely?

Solution
As it turns out, no you’re not. There’s a good chance someone has already solved your problem or problems. It’s best to see if you can find the capabilities you desire in a third-party library.

Comments
Favor reusable libraries to make your solution simpler and clearer. An added benefit to this approach is well-tested code.

Coordinate-Based Clicks and Non-Standard Controls

Problem
You’re automating a test, and you need to work with a custom control. The tool you’re using doesn’t want to cooperate. Should you implement click actions using X and Y coordinates?

Solution
It’s wise to avoid implementing hard-coded clicks. Instead, use selectors to obtain a parent element and work your way down the DOM tree.

Comments
Why? Using coordinates will make your tests more brittle; especially when it comes to responsive applications.

Add Randomness to Automated Tests

Question
How should you automate cases that exercise functionality in a variety of ways?

Answer
Write several workflow-based functions or methods and randomly inject them in your tests.

Comments
It’s important to add an element of randomness in your automated tests. This holds true for both data and operations. For example, use data-driven techniques to switch up the data used by tests.

Example
The example below demonstrates a data-driven approach to testing a login feature.

import { t } from 'testcafe';

fixture `login`
    .page `www.example.com`;

const testCases = [
    {
        name:     'Valid User',
        email:    'valid@example.fake',
        password: 'pass'
    },
    {
        name:     'Invalid User',
        email:    'invalid@example.fake',
        password: 'fail'
    }
];

async function performLogin (email, password) {
    await t
        .typeText('#email', email)
        .typeText('#password', password)
        .click('#submit');
}

for (const user of testCases) {
    test('test ' + user.name, async t => {
        await performLogin(user.email, user.password);

        // Confirm valid user can login.
        await t.expect('#username').eql('Valid User');

        // Confirm invalid user can't login.
        ...
    });
}

Using Static Code Analyzers

Question
What can be done to check for common programming errors and standards violations in automation code?

Answer
Use static analysis tools to analyze the code. For example, jslint for the JavaScript language.

Comments
Static analysis tools help identify common syntax errors and standards violations. Some even raise warnings about poor programming style. For example, functions that are too long or take too many arguments.

Use Coding Standards for Test Code

Question
Should you adhere to coding standards when writing test automation code?

Answer
In short, yes. Follow coding standards for the language and technology used. Do this even if the quality of test automation code isn’t expected to be at the level of application code.

Example
The example below uses the fantastic tool TestCafe.

import { Selector } from 'testcafe';

fixture `example`
    .page `https://google.com`;

// Incorrect. An obscure way to name a selector.
const txtSearch = Selector('#q');

// Correct. Selectors have meaningful names.
const searchField = Selector('#q');
const pageTitle = Selector('title');

test('User can search Google', async t => {
    await t
        .typeText(searchField, 'Jason Riley')
        .expect(pageTitle.innerText).contains('Jason Riley');
});

Confirm All Options in Logical Conditions

Question
What should you do when you have automated scripts that contain assertions with many logical conditions?

Answer
Automated tests should not contain logic in them or many assertions, for that matter. But, if a test requires logic, be sure to check all conditions.

Comments
Consider using parentheses with logical expressions. It helps with code comprehension.

Bulky Code

Problem
What should you do when the need to check many conditions results in deeply nested code?

Solution
Avoid nesting more than three levels when using conditional and looping constructs. A great resource on refactoring techniques for minimizing nesting is here.

Comments
Avoid deeply nested code because it’s difficult to understand and debug. Use static analysis tools to determine the cyclomatic complexity of your code.

Production-Quality Test Automation

Problem
You want to know the level of code quality required for automated test solutions and utilities.

Solution
Do not treat test automation as a full-fledged development project. Favor simplicity over production-grade code.

Comments
Strive to write maintainable test automation, but don’t sweat writing code that’s on the same level of the engineers building the product. Follow good programming practices, and prefer adding value to writing perfect code. In general, test automation is written for internal purposes only.

Exiting Loops Using Timeouts

Problem
On certain occasions, a test’s execution should only continue after a certain event. You’re using a looping construct to check for a condition, and you want to know what to do when it has been met.

Solution
Use timeouts to avoid infinite loops.

Comments
You may want to implement global timeouts in your project to reduce maintenance.

Example

// Method that checks the existence of a file.
bool FileExists() {
    ...
}

// Decrease timeout by 1 and wait for a second with each iteration.
// After 10 seconds, break out of the loop.
var maxWaitTime = 10;
while (!FileExists()) {
    if (maxWaitTime == 0) {
        break;
    }
    maxWaitTime--;
    Thread.Sleep(1000);
}

Sleeps and Pauses

Problem
Your test automation isn’t synchronizing well with the application under test (AUT). You need a way for your script to wait until the application is ready for the next action. Should you implement sleeps or pauses in your tests to overcome the problem? Thread.Sleep(), anyone?

Solution
As a general rule, you should not insert pauses or sleeps into your automation code. The best solution is to write a custom routine or leverage an existing library (e.g. Selenium WebDriver) to wait on a property of a UI element.

Comments
Sprinkling hard-coded sleeps in automation code isn’t desirable, because scripts will wait for a specified amount of time regardless of the application’s speed. And, as a result, they will take longer to execute.

Example

// Wrong. Note the sleep that waits 5 seconds for the page to refresh.
browser.FindElement(By.Id("submitButton")).Click();
System.Threading.Thread.Sleep(5000);
Assert.That(browser.FindElement(By.TagName("body")).Text.Contains("Order No:"));

// Correct. Explicitly wait for an element before proceeding.
browser.FindElement(By.Id("submitButton")).Click();
WebDriverWait wait = new WebDriverWait(browser, TimeSpan.FromSeconds(5));
wait.Until(brw => brw.FindElement(By.Id("orderNo")));
Assert.That(browser.FindElement(By.TagName("body")).Text.Contains("Order No:"));