How to cut your Selenium test run time by 70%

Deep inside the implementation of every automated UI test lies the potential to turn something simple into something slow and unreliable—simply by adding extra Selenium commands. The reason: Longer tests are less likely to pass.

Here's an example of how that can happen in real life. In the situation below I'll analyze a test in order to reveal and resolve a 70% inefficiency in test run time. You can apply the lessons learned to your own tests.

The problem

One reason that many Selenium commands in your test can make tests dramatically slower and less reliable is that every Selenium command carries the potential for an invalid element locator or a synchronization issue.

In this example, a legacy version test takes 300 seconds to run and executes 364 Selenium requests.

But after the test optimizations described below, the same test executes in 92 seconds—nearly four times faster.

Identifying the inefficiencies

In this case, the root cause of failure is the inability to locate the element below (Scheduler). The failure occurs 72 seconds into the test.

The rest of the test is spent performing irrelevant and repetitive actions that don't result in progress. For example, the test continues to search for Scheduler for another 60 seconds, until 2:12.

At this point, the test performs some random operations, likely to refresh a state, and then continues searching for Scheduler, which, if it hadn't shown up in the previous 60 seconds, is never going to do so (see below).

Only 72 out of 300 seconds were necessary to discover that the Scheduler element doesn't exist. Hence, 76% of the test time is a waste; it provides no new or meaningful information. That translates into a 76% slower feedback loop for your teams.

The solution

The vast majority of modern web applications don't take longer than 20 seconds to load, so waiting for an element to appear for longer than that is wasteful. This claim can be backed up with more data from the app under test.

In the image below, you can see that the time from loading the URL (at 11:00) to finding a username field and sending keys is only 5.75 seconds. That's 5.75 seconds from page open to render to first interaction with an element.

This test would take only 92 seconds, versus 300 seconds for the original test, with the same outcome.

The takeaway: Focus on outcomes

Creating efficient Selenium tests is extremely easy. First, figure out the maximum amount of time that is acceptable to wait for an element to appear on a web page. No, that's not 60 seconds; 20 seconds is recommended for most modern web apps.

Then use an explicit wait to wait for an element to be in a specific state for up to 20 seconds. At that point the test should fail with a timeout exception that lets you know that the element isn't visible on the page in the expected time. This strategy is applicable to most modern web applications.

For more on mastering automated testing, register for Advolodkin's Complete Selenium Java Bootcamp. Enter your email and you'll receive a coupon for 45% off the regular price on Black Friday.