You are here

49 great resources for performance testing teams

public://pictures/Joe-Colantonio .jpg
Joe Colantonio, Founder, TestTalks

One major roadblock that prevents teams from successfully executing performance testing is a lack of training. Sometimes, just knowing where to start can be a challenge.

Not many software engineering organizations are familiar with the needs of performance testing. Even worse, hardly anyone is attempting to add this activity early in the software development lifecycle (SDLC), as advised in the shift-left movement

Here's my list of mostly open-source resources you can use or share with your team to start incorporating performance testing into your team’s SDLC. 

[ Get Report: Buyer’s Guide to Software Test Automation Tools ]

The impact of web performance

This resource was recommended to me by Jacek Okrojek, a lead test automation engineer at TestArmy and a speaker at this year’s PerfGuild online performance testing conference.

  • WPO stats has case studies and experiments demonstrating the impact of web performance optimization (WPO) on user experience and business metrics. If you are trying to convince your organization that performance testing is important to your software development efforts, this is a great place to start.

[ Related story: 50+ resources for test automation engineers

Performance testing example sites

If you're just getting started or don't have an app to run performance tests against, it's useful to have an example app to test against to help you as you learn. Here are a few web pages designed solely for this purpose.

  • If you're an old-time Mercury automation engineer, you'll probably remember the sample site called Mercury Tours that shipped with WinRunner, QTP, and UFT. You can also use it as an example performance testing site.
  • Another old-time favorite website for performance testing is a full web application called JpetStore; it's built on top of MyBatis 3, Spring 5, and Stripes.
  • Blazemeter also has a popular test site; it's called BlazeDemo.

[ Webinar: How to Fit Security Into Your Software Lifecycle With Automation and Integration ]

Resources for performance testing newbies

[ Related story: 6 test automation tools developers love ]

Performance testing tools

Once you have a website to test against and a basic understanding of performance testing, you can take a look at some open-source performance testing tool options.

  • JMeter is the most popular open-source tool in the load-testing space. Apache’s JMeter describes itself as a Java application that was designed specifically for load testing with the ability to measure performance.
  • Taurus has the user-friendly option of allowing you to write your tests in YAML. You can describe a full-blown script in about 10 lines of text, which gives your teams the ability to describe their tests in a YAML or JSON file.
  • Gatling is a stress tool built on Scala, Akka, and Netty. It allows you to test and measure your application’s end-to-end performance.
  • Locust is a simple-to-use, distributed, Python-based load-testing tool.
  • k6 is a developer-centric, open-source load-testing tool for testing the performance of your back-end infrastructure.
  • Grinder is a load-testing Java framework that makes it easy to run a distributed test that leverages a host of load injector machines.

Performance monitoring tools

Monitoring performance is a key element to ensure that your application will perform perfectly in the wild. However, if you look at the Gartner Magic Quadrant for Application Performance Monitoring report, all the tools listed are paid tools.

What about the folks who don’t have a budget for paid tools? Here are some free APM solutions you should check out:

  • Pinpoint is currently the most popular application performance management tool on Github.  
  • JavaMelody monitors JavaEE apps.
  • Skywalking monitors microservices and container-based architectures.
  • Stagemonitor monitors Java server applications.
  • Dynatrace's Appmon offers a personal license you can use free for life in your local environment.
  • The Prefix web request profiler can help you validate the behavior of your code and find hidden exceptions in your code. It can also find slow SQL query issues.

Client-side performance resources

If your application is very heavy on JavaScript client-side functionality, you're going to want to measure that as well. Here are some tools to analyze web pages, monitor performance, or find front-end code quality issues:

  • YSlow analyzes web pages and why they're slow based on Yahoo's rules for high-performance websites.
  • YSlow for PhantomJS also introduces new output formats for automated test frameworks.
  • YellowLab Tools contain an online test to help speed up heavy web pages.
  • WebPageTest provides a website speed test that allows you to run from multiple locations using real browsers at real consumer connection speeds.
  • Varvy tests your site to determine whether it follows the Google guidelines for speed.
  • Benchmark.js is a benchmarking library that supports high-resolution timers and returns statistically significant results.
  • PerfMap allows you to to create a front-end performance heat map of resources loaded in the browser using the Resource Timing API.
  • Phantomas is a PhantomJS-based Web performance metrics collector and monitoring tool.
  • Sitespeed.io is an open-source tool that helps you analyze and optimize your website speed and performance, based on best practices.
  • Webperf Heat Maps generate performance heat maps of Web pages.
  • Zombie.js is a lightweight framework for testing client-side JavaScript code in a simulated environment. No browser required.

Google performance resources

Google has a bunch of great resources to help you understand how to improve the performance of your web and other applications. Here are a few:

  • Google Developer Tools help you get started analyzing runtime performance issues.
  • Google Lighthouse is an open-source, automated tool for improving the quality of web pages. It can be run against any web page, whether it's public or requires authentication. It has audits for performance, accessibility, progressive Web apps, and more.
  • Web Fundamentals contains explanations, examples, and recommendations that focus on low-effort, high-return performance wins.
  • Google speed test, a tool that reports on the performance of a page on both mobile and desktop devices, provides suggestions on how that page may be improved.
  • Chrome User Experience Report is a treasure trove of metrics, powered by real user measurement data. Knowing this data can help you create more realistic tests for how your application will be used in production.

Test infrastructure performance resources

Paul Bruce, DevOps advocate and automation and performance engineer at Neotys, says that in his experience, the traditional folks managing performance are missing the huge dangers and opportunities in the Kubernetes world.

Here are some Kubernetes performance testing resources:

  • Kubernetes perf-tests is a repository dedicated to storing various Kubernetes-related performance test-related tools. If you want to add your own load test, benchmark, framework, or other tool, contact one of the owners.
  • JMeter Cluster supports Kubernetes and OpenShift.
  • Measuring of API performance describes a test plan for quantifying the API performance of container cluster systems.
  • Load Testing Kubernetes is a resource on how to optimize your cluster resource allocation in production.

Let's not forget Docker. Here are some Docker images that have performance testing tools baked into them to start with:

  • JMeter Docker is a lightweight Docker image for Apache JMeter.
  • Jenkins Performance Plugin allows you to capture reports from popular testing tools. Jenkins will generate graphics with a trend report of performance and robustness. It allows you to set the final build status as good, unstable, or failed, based on the reported error percentage.

Site reliability

Because I'm old school, I still call many software development activities "performance testing," but I’ve begun to see a trend that has this term falling by the wayside.

Mark Tomlinson, the "performance sherpa," has an interesting theory. He says that if you compare response counts on "performance testing" vs. "webperf," it's very clear that developer/engineers are doing performance work. Web designers are doing performance work. Infrastructure teams are doing performance work. And none of them self-identify as "testers" in their organizations.

If we compare response counts on "security testing" vs. "infosec," we see the same trends.

So, if you’re like me and still call it performance testing, don’t ignore resources that also are for site reliability engineers or infrastructure teams. Here are some examples:

  • Site Reliability Engineering BookMembers of Google's SRE team explain how their engagement with the entire software lifecycle has enabled Google to build, deploy, monitor, and maintain some of the largest software systems in the world.
  • Chaos EngineeringIn this e-book, members of the Netflix team that developed chaos engineering explain how to apply these principles to your own system. By introducing controlled experiments, you'll learn how behavior from component interactions can cause your system to drift into an unsafe, chaotic state.
  • Chaos Monkey randomly terminates virtual machine instances and containers that run inside your production environment. Exposing engineers to failures more frequently gives them incentives to build resilient services.

If you’ve tried using Chaos Monkey in the past, you may have found it difficult to use and understand. There’s another product built on top of it called Gremlin that makes it easier to get started.

  • Gremlin Free provides the ability to run both Shutdown and CPU attacks on your hosts or containers, controlled by a simple user interface at no cost.

That's my list. What did I miss? Share your suggestions in a comment below.

Hear about more performance testing resources, trends, and best practices, drop in on the PerfGuild online performance testing conference, which runs April 8-9. Registration is $197. Groups of five or more receive a 20% discount.

[ Get Report: How to Get the Most From Your Application Security Testing Budget ]