How many layers does your testing cake have?

When development teams think of building applications, they usually think about what features they want to build. But those applications also needs to satisfy a set of cross-functional requirements.

Here are key factors to consider as you build tests that account for all of your requirements.

World Quality Report 2018-19: The State of QA and Testing

Account for different factors

 Depending on what you're building, usually there's some combination of:

  • Cross-browser support (I'm looking at you, Internet Explorer)
  • Device support: Mobile devices account for a huge portion of web traffic
  • Accessibility: Making sure people can use your product, no matter their abilities
  • Security
  • Performance
  • Localization and internationalization
  • Aesthetics: Making the UI conform to your organization's global design look and feel

Imagine each feature you deliver as a slice of cake. As developers have gotten better at agile, we've learned to make each slice of cake as thin as possible. Cross-functional development is an aspect that's often overlooked, though, in planning as well as in delivery and testing.

Imagine that each of the cross-functional requirements above is another layer to your cake. Turns out that, no matter how thin your slice is, it can still have lots of layers to worry about.

With the pace at which engineering teams are expected to deliver new features to market, how do you make sure you're producing software that covers all these bases—and does so effectively?

You can't have your cake and eat it too

The short answer is that there's no fail-proof method. If you care 100% about all of the aspects I listed, you're going to be shipping features very slowly. So you need to prioritize for and select the ones that are most important to your business and your users.

For example, security will be high on your list if you deal with sensitive user data. Is cross-browser support important to you? Gather some statistics about the users who visit your site to see which browsers are used most frequently.

While it's impossible to predict the future, it's useful to identify important concerns early on. Making an existing system mobile-friendly or able to support multiple languages, for example, will invariably be a massive undertaking. These may or may not be as important as other factors.

Automate what you can

Every time you automate, you free up some mental space to focus on everything else. Here are a few examples:

  • Use a linter to let you know when you've written code that introduces a basic security vulnerability, or when you've forgotten to add an HTML attribute that helps customers who use screen readers.

  • Use services such as npm audit to check if your code depends on libraries with known security vulnerabilities.

  • Use static analysis tools to highlight when your bundle or executable size gets big enough to impact performance.

  • Use tools such as Airbrake to collect data on the kinds of errors users run into in their browsers.

Decide when to test

If you test every cross-functional aspect every time, you're not going to ship new features very fast. But if you don't test everything, you're bound to run into issues. Remember that you have options, and that you should use the priorities you've decided on to choose the right approach. 

Other things to keep in mind:

  • Test only the most important cross-functional requirements as part of every user story.

  • Some things will be fine if you spot-check them once a week or once a month. The trick is to test often enough that you catch issues early, but not so often that you waste or duplicate effort.

  • Some tests are expensive, and you may be able to run these only once a quarter. This includes getting an external company to do penetration testing for you.

  • You can skip testing altogether for some aspects and fix issues only when someone complains. Depending on your system and your relationship with your users, this can be a very powerful and often underestimated strategy.

How thick is each layer?

One trick you can use to reduce the burden on your team is to reduce the cost of testing each layer. For example, for device testing, you could test on physical devices, but you might get almost as good a result from testing on device emulators. If cross-device compatibility is not high on your list of priorities, you could even get away with testing with your browser's built-in developer tools.

For accessibility, consider relying on linting instead of testing with an actual screen reader. Or form a group of trusted beta-testers that use screen readers, and release software to them early for feedback.

Learn together

When issues crop up, share them with one another and talk about how you fixed them. For big issues, it can often be helpful to have a post-mortem or a retrospective and document some learnings, so you can prevent similar mistakes from occurring in future.

It's helpful to learn about common pitfalls. For example, you can avoid a lot of security issues by being aware of the OWASP Top 10. It's also worth knowing the strengths and weaknesses of your specific technology stack. For instance, what performance issues are common with Node's event loop? Which security features come standard in React?

Make mistakes harder to make

As a team, you can build what you learn into the way you work every day, and so make it harder to make mistakes. For example, you can write a common wrapper library for connecting to a database. The library can provide sensible defaults, suited to your environment, and can prevent some performance and security issues without developers even needing to think about them.

Another example is making it easier to support mobile devices by maintaining a shared CSS file that provides the styles required for a standard responsive layout.

It's no cake-walk

Teams can be overwhelmed by the number of cross-functional requirements they need to satisfy. Either delivery grinds to a halt as they try to satisfy all requirements, or software ends up being buggy as they try to keep up the pace. This is not easy, so try some of the techniques above to ease your load.

And remember: You can't realistically test all cross-functional aspects all the time, so prioritize what's important to your users and build those into your workflow.

Topics: Dev & Test