9 great open-source API testing tools: How to choose

I wrote the original version of this article a year ago. Since then, there have been a few developments in API testing that should be noted.

For one, there are a few more tools that you should look at. There's also a new technique that is all the buzz around AI automation.

While most vendors are talking up the benefits of AI- and UI-based testing tools, there are also some AI/machine-learning applications that can help with API testing as well.

Here are nine great open-source API testing tool options.

World Quality Report 2018-19: The State of QA and Testing

Why care about API testing?

As more companies make the shift left toward DevOps, continuous integration (CI), and continuous deployment (CD), test feedback needs to be quicker than ever. Focusing solely on UI automation—which is notoriously slow—can kill your test automation efforts.

As you scramble to ensure that your applications are ready to ship, API testing should be part of your overall automation strategy.

APIs are the basis of modern software development, especially as more and more teams move away from monolithic applications and begin adopting a microservices approach to software development.

With microservices making up the backbone of most newer development efforts, API testing becomes even more critical than before.

Roadblocks to API testing adoption

I wrote a book on API testing in 2014, believing it would really take off. It hasn't. There are reasons for this. 

One reason that API testing isn’t used by more organizations is that even though they may claim to be agile, they still have distinct roles defined for developers and testers.

Testers believe developers should be doing the API tests, while developers believe the opposite. Testers may also be technically unaware of how to even get started testing an API, so they may simply be focusing on what they know—which is UI automation.

AI, machine learning can help

To reduce the complexity of API testing, some vendors have created utilities that leverage AI to convert manual UI tests into API tests.

The tools basically act like sniffers that use machine learning to automatically find patterns and connect relationships among different services as users interact with the applications.

As these approaches become more refined, you can imagine how the complexity of API test generation gets significantly reduced.

Unfortunately, I’m not aware of this functionality being available yet in any open-source API tools, but that shouldn't prevent you from getting started. Knowing the basics of API testing will help you now and in an AI-driven API future.

The following are the top nine API testing tools I believe can help you, with descriptions that might guide you in choosing one over the others—though they're all great options.

REST-Assured

When using Java, REST-Assured is my first choice for API automation. In fact, it's the main tool I use for API testing.

REST-Assured is a fluent Java library you can use to test HTTP-based REST services. It's designed with testing in mind, and it integrates with any existing Java-based automation framework. The REST-Assured API was created so that you don't necessarily need to be an HTTP expert.

It provides a behavior-driven development (BDD)-like, domain-specific language that makes creating API testing in Java simple. It also has a bunch of baked-in functionalities, which means you don't have to code things from scratch.

Bonus: If you're like me and use the Serenity automation framework, REST-Assured integrates seamlessly with it, which means you can combine your UI and REST tests all in one framework that generates awesome reports.

Testing and validating REST services is harder in Java than it is in dynamic languages such as Ruby and Groovy. This is one more reason to use REST-Assured, since it brings the simplicity of using those languages into the Java domain.

If your team is made up mainly of Java coders, I highly recommend REST-Assured for API testing.

Postman

Some folks don't want to code in an integrated development environment using the same language as the developers. After all, if your developers don’t contribute to your automation efforts, why force yourself to use their tech stack if its not the best option for you? Sometimes you just want a quick and dirty API test without all the overhead or a tool to help with exploratory testing of your API. Postman is perfect in this scenario.

Postman is also a nice option for exploratory-type API testing. But it's also powerful enough that you can create more integrated solutions if you wish.

Postman is an easy-to-use REST client, and you can get started with it quickly by leveraging its Chrome plug-in. There's a native version for both Mac and Windows.

It has a very rich interface that many REST clients don’t have, making it easy to use. It also lets you easily share your knowledge with your co-workers, because you can package up all your requests and expected responses and send them off to someone else so that he or she can take a look also.

Casual gaming company Big Fish Games uses Postman as a collaborative tool. According to Amber Race, a software development engineer in test (SDET) at the company, one person does the research and figures out all the test cases for the API.

That person then publishes that information to a wiki so that others can run the API tests and ensure that what they're doing doesn't break existing API functionality. Postman even allows you to place a button on your internal website saying, "Run in Postman," and it automatically kicks off your Postman tests.

If your team wants to not only test APIs but also have a tool to help automate some of your exploratory API testing efforts, Postman is a great choice.

SoapUI

SoapUI has been around for a while now. If your teams are doing API testing only and are composed mostly of QA engineers, as opposed to developers, SoapUI might be your best choice.

SoapUI is a fully functional test tool dedicated to API testing. Rather than having to create a solution from scratch, SoapUI allows you to leverage a tool full of functionality aimed strictly at API testing. And if for some reason you need to create a custom workflow or functionality, you can code up your solution in SoapUI using Groovy.

If your team has complicated API testing scenarios and is weighted toward QA/test engineers, SoapUI is a good tool to try first.

JMeter

Although JMeter was created for load testing, many folks use it for functional API testing as well.

JMeter includes all the functionality you need to test an API, plus some extra features that can enhance your API testing efforts. For example, JMeter can automatically work with CSV files, which lets your teams quickly create unique parameter values for your API tests. It also integrates with Jenkins, which means you can include your API tests in your CI pipelines.

If you plan on creating API functional tests that you would also like to leverage in your performance tests, you can kill two birds with one stone by using JMeter as your main API testing solution.

Karate

I first learned about Karate in early 2017, when the tool was still in its early stages of development, and immediately saw its potential. Since then Karate as matured in a stable tool with unique functionality, such as having API Testing, API testing doubles, and API performance testing all available in one framework.

API tests are written using BDD Gherkin syntax. But unlike most BDD frameworks (Cucumber, JBehave, SpecFlow), you don't need to write step definitions. Karate has already created all the step definitions you need to get started testing APIs.

If you're new to programming or automation testing Karate is easy to use since no Java knowledge is required. If this is you, Karate might be the perfect choice for your team.

Fiddler

Fiddler is a tool that allows you to monitor, manipulate, and reuse HTTP requests. Fiddler does many things that allow you to debug website issues, and, with one of its many extensions, you can accomplish even more.

One of those—the APITest extension—greatly enhances Fiddler to validate the behavior of web APIs. (Validators offer a lightweight way to judge the success or failure of a test.) For more hard-core API testing development, you can use the FiddlerCore.NET Class Library to build your API testing infrastructure.

This is a great choice for teams that use .NET languages, since you can develop your tests using any .NET language you wish.

Citrus Framework

I’m not quite sure why more folks aren't aware of the Citrus Framework. Citrus is an open-source tool that can help you automate integration tests for virtually any messaging protocol or data format.

Among its benefits:

  • Works with REST, SOAP, HTTP, JMS, TCP/IP, and more
  • Creates tests using Java or XML
  • Is mature—version 1 launched in September 2009, and it's been steadily updated ever since, with release 2.7.6 introduced in June
  • Has great documentation

If you plan testing other headless technologies beyond REST services, Citrus is the tool for you. It’s made to handle any headless protocol, giving you an excellent solution for all your non-UI testing needs. This flexibility is beneficial if you work in an enterprise environment and need to test many different kinds of applications. 

PowerShell

You might be saying, "PowerShell? Really?"

But actually, with the adoption of DevOps in Windows-based organizations, PowerShell is awesome at automating lots of things from the command line.

Some benefits of using PowerShell for API testing are:

  • You need only one line of code to import your Web Services Description Language. 
  • You need only one line of code to send in your request and get your output.
  • It's factory-installed on all Windows machines, so it's quickly usable by all your company's employees.
  • It's easy to learn.
  • It's very fast, since it runs from the command line without any UI overhead.

PowerShell is an open-source Microsoft product. So pretty much anytime you want to use the command line or automate anything such as API tests inside of the Windows ecosystem (or pretty much any Microsoft product), you want to go with PowerShell.

Insomnia

After looking at Insomnia, I’m not sure how I ever missed it. It's free and easy to use, and has a nice-looking interface. Some other benefits of Insomnia are that it allows you to:

  • Create HTTP requests
  • View response details
  • Organize your tests
  • Reuse values
  • Generate code snippets 

All that, and it sports a beautiful interface also. There are even more features listed under the vendor's "big ol'" list of features on its website.

(I actually got the idea of adding PowerShell and Insomnia from comments on last year's story, for tools that I forgot. Thanks to all who commented.)

Taurus

Taurus is an automation-friendly framework for continuous testing. Because you can use it with JMeter, it can handle API testing.

The power of Taurus is that it allows you to write your tests in YAML. You can actually describe a full-blown script in about 10 lines of text, which gives teams the ability to describe their tests in a YAML or JSON file.

YAML is a human-readable and editable approach that lets you describe a test in a simple text file. This is a big leap forward from the old days of having to bring up a big, heavy, vendor-specific record and scripting tool.

Other features include:

  • Taurus allows more members of your team to contribute to AI testing. Also, since your test cases are written in YAML, the tests are much more readable, making them easier to perform code reviews on.
  • Taurus basically fits performance testing into your CI/CD pipeline much more efficiently.
  • Taurus provides a sort of abstraction layer on top of JMeter, as well as some other tools such as Locust, Gatling, The Grinder, and Selenium.

Taurus is great to use when teams want to take a more BDD-testing approach to their API testing efforts. Using YAML files gives you clear, easy-to-read tests that anyone on your team can understand.

Which API tool?

There are no perfect tools. Each organization has different requirements. Virtually all API test tools work pretty well and are great options, depending on your team's needs.

There are many other API test tools available, but these are my top choices. What are yours?

Topics: Dev & Test