You are here

You are here

How to get more from your load-testing tools

Israel Rogoza Inbound Product Manager, Micro Focus

Your company just bought the best load-testing tool on the market. Now your boss expects you to improve application or website performance. There's just one problem: Even though you have set up and are using your shiny new tool, your application is still slow, and users are unhappy.

Sound familiar? If so, you're not alone. Many organizations aren't gaining the full measure of productivity from the tools they have because they're not taking full advantage of key features the tools offer — either because they don't know about those features, or they haven't invested the time to learn how to use them.

Evaluating your load-testing tools

At a recent Google Test Automation Conference, Google technology leaders talked about how they evaluate and remove duplicate tools from their testing arsenal. The process yields better results across the testing organization and helps them focus on getting the most out of their testing. Among their suggestions: If the tool you're looking at doesn't have a significant advantage over other tools you've evaluated or the tools you already own, end the evaluation and uninstall the tool.

Here are a few other tips:

  • Decide if you want a SaaS-based solution or an on-premises tool.
  • Open source is great, but you will need a tool that will allow you to build a test suite; perhaps using the open-source scripts on a proprietary tool will be the best approach.
  • Know exactly what you want to test; I've seen many customers buy the most expensive tool, then use it to run only one simple test very occasionally.

Once you've identified a clear winner from the evaluation process, move forward with acquisition, training, and implementation. Be sure to sunset an existing tool if it can be replaced by the new one, freeing up licensing and maintenance costs.

Get more from what you have

To get the biggest productivity boost from the tools you have purchased, you need to take the time to study up on all of the features available to you. Yes, you're busy, and no one likes to read documentation, but knowing your tool's abilities is critical. And the good news is that tools today are easier to learn.

To get up to speed before you start testing:

  • Ask for a product demo. A good demo can save you a lot of time once you start using the tool.
  • Review the tutorials. In addition to what the vendor offers, YouTube is full of great tutorials for virtually every product on the market.
  • Register in your product's support forum. It's a great place to get support and answers to frequently asked questions.
  • Determine a team tool strategy and a policy for using your chosen tools consistently.
  • Know your own development environment and architecture. That makes it easier to select a script type for recording a user scenario.
  • Identify the key features of the tool that will have the biggest impact on your productivity—and learn how to fully utilize them.

I worked with one large communications company that complained that the tool it was using was no good. Although the company got excellent results when testing, the results in production were poor. When I asked if the customer had simulated real user conditions and all of the network issues that can occur, he replied that he had not. Users running the app over mobile networks were the ones experiencing the problem, but he hadn't done the testing needed to realize that. 

Use your load-testing tools daily 

After deciding which tool to buy, you need to find the best ways to combine it with your daily work. Performance testing is critical in the development world and in my experience it is better to add load testing to daily jobs. Don't wait for the regression cycle, where it might be too late (expensive, time-consuming, etc.).

There are many ways to do that. For one tester I know, performance testing is his final "gate" before deploying a new build. 

Here are some basic tips for getting started:

  • If you are using DevOps as part of your QA process, integrate the tool with your pipeline. This will ensure a high-level QA performance.
  • Define SLAs to target the amount of time for each step of your application process—for example, log in to your website.
  • Create scenarios for special occasions, such as acceptance testing, or after a big code change so you can compare those changes to the previous state.
  • If you are working with agile, add to your definition of "done" a process of verifying and comparing your application's performance.

Don’t just run a test. Analyze the results

Buying a tool and then using it every day are the first two steps toward keeping your application performance stable and making it better. But the third and most important step is to analyze the results and understand them so that you can change any defects.

A few years ago, QA engineers from a retailer with a crisis in the middle of Black Friday contacted us. They asked for tutorials about basic flows and how to run the tests, but refused to pay for analyzing the results. They felt that because they were engineers, they should know how to figure it out without any help. This was a mistake; running a test is great, but analyzing it and sending the report to the right person is key.

Here are some tips for how to make analysis much more efficient:

  • Know your product analysis abilities and use those features: benchmark, compare, merge, etc.
  • Ensure that a human being goes over the detailed report and understands it. I’ve seen many cases where the report was directed to the QA trash.
  • Understand the language of the report; learn how it was calculated, the measurement computation for each metric, and so on.
  • Use the report to raise red flags, highlight issues, and send it to the right focal point.

Get the full measure of value

You want to purchase the right testing tools without overspending on unnecessary or redundant features. But once you have them, make sure you get the full measure of value from them.

A performance-measurement tool is not only pertinent to testing teams. Anyone with a stake in performance can benefit from such a tool. Investing in training—and making sure you have a well-defined process—will go a long way toward that.

Keep learning