The QA team plays a strategic role in DevOps, but consensus is still only emerging on how best to leverage its skills for application security testing.
Some view application security as just another QA issue and argue that testing a software release for security vulnerabilities is similar to testing it for functional and performance issues. Others view the mission of the QA team as being fundamentally different from that of the app security team, making QA incompatible for anything other than basic security testing.
QA teams verify existing functions, features, and behavior. App sec teams, on the other hand, try to break existing functionality, features, and behavior. The results from app sec testing are often non-deterministic, and hours of analysis may be required to evaluate a single security bug.
Spending hours analyzing a single issue is highly inefficient for QA teams, which often might need to run through thousands of uses cases per release, said Himanshu Dwivedi, founder and CEO of Data Theorem.
"The problem is perspective. While the difference seems subtle, it is a world of difference."
—Himanshu Dwivedi
Here's why application security teams need to let get QA testers on their roster.
Morphing roles and responsibilities
DevOps practices have blurred the boundaries between development, testing, and operations. Increasingly, QA has become everybody's responsibility across the development pipeline. Developers these days routinely check their code for errors, while QA professionals help to fix errors in code.
The trend has prompted questions about the role that QA should—and can—play in app sec testing. In a blog entry last May, Stan Duffy, a senior software development engineer in test at Puppet, argued that security teams that are not working closely with the QA team are making a mistake.
"The QA team, with guidance from the security engineer, should be responsible for the application testing phase of the application vetting processes," Duffy said. It needs to be using static analysis and other tools to test the app against detailed security requirements and providing recommendations to the security engineer on how to make it ready for production from a security standpoint.
Dan Cornell, CTO at application security services consultancy Denim Group, said combining the strengths of the QA team with the app sec team creates critical mass.
"QA teams are large and well-established when compared to app sec teams. Incrementally expanding their mandate to include aspects of the app sec program is a great way for app sec teams to gain leverage."
—Dan Cornell
In the early stages of the development process, Cornell said, QA can help craft testing scenarios that include "abuse cases"—in other words, how a particular software release is likely to be abused. Such testing scenarios can inform developers and help them find ways to avoid introducing code that could result in abuse.
QA's role in static testing
Given proper training and support, QA can certainly take over aspects of security static analysis testing (SAST) and running SAST tools. But hardcore security code reviews such as those performed during penetration tests are best left to security teams that have the training and specialization for the task, Cornell said. "QA teams are best-suited to incorporating application security testing tools—both static and dynamic—into their use of other automated testing tools."
Some of the debate around how to leverage QA in app sec testing is also tied to the evolving roles of developers, testers, and operations teams in the DevOps environment. As boundaries among different functions have blurred, so too have some of the distinctions among job responsibilities, said Chris Romeo, CEO and co-founder of Security Journey.
"Realize that in the new world of DevOps, everyone is a developer, and the title of QA professional gets fuzzy."
—Chris Romeo
The app sec testing function is all about writing functional tests to verify security requirements and using vulnerability scanning and DAST-style tools at some increments to identify vulnerabilities in code that you have written, Romeo said. This is well-suited to the QA function, he said.
In fact, with proper training, QA can be effective even in specialist security tasks such as running penetration tests, Romeo said. The job of QA is to try to break things functionally.
"Pen testing is just redirecting that desire to break things toward security. The bug-bounty industry was created because of the void in skilled, internal application security testers."
—Chris Romeo
Leveraging QA in app sec testing
So, what should organizations being doing to better leverage QA skill sets in app sec testing?
The key is to ensure that the QA team is folded into DevOps, said Romeo and others. The ideal should be to try to integrate resources from across the development, testing, and operations teams, and then focus on collaboratively improving security.
Kurt Bittner, vice president of enterprise solutions at Scrum.org and a noted expert on agile and iterative software development practices, said having QA and app sec teams in separate silos was antithetical to DevOps.
"Having a separate QA function or team creates undesirable hand-offs and delays, and isn't consistent with agile or DevOps practices."
—Kurt Bittner
Both developers and QA pros have much to learn
For their part, QA professionals should be thinking about acquiring a broader set of skills, including coding. One place to begin is by developing test automation of all kinds—including unit, functional, nonfunctional, load, and security testing.
Developing test automation is a great way for QA professionals to learn a codebase, and it accelerates the organization's goal of having most testing automated and executed every time teams do a build, Bittner said.
Crucially, QA professionals need to change their mindset when it comes to app sec testing. They need to think of the negative in order to purposely bypass any "truths" in a product, said Data Theorem's Dwivedi.
"For example, think about how to board a flight with a valid ticket, but to a different location—after you go through security," he said. Is a valid ID required at the gate? If the barcode is wrong, what happens? What if the barcode is scanned twice? Does it cry foul or simply fail to open? "All of these are high-level examples of trying to bypass an existing control, rather than trying to ensure the product works as designed."
Developers have a role as well. They need to learn how to automate QA and build it into their delivery pipeline to improve cycle time and enable faster feedback on the code they are developing. In agile, the development team needs to learn QA skills and take on QA responsibilities, leveraging test automation where they can.
"The ideal for both agile and DevOps is to have a cross-functional team, consisting of members who are able to do all the work needed to produce a release. This means that the people on the team need to have QA skills."
—Kurt Bittner
Organizations with a traditional silo model and a standalone QA team will probably need to form teams consisting of people with dev, ops, and QA skills who are committed to growing their app sec skills in breadth and depth over time, Bittner said.
Keep learning
The future is security as code. Find out how DevSecOps gets you there with TechBeacon's Guide. Plus: See the SANS DevSecOps survey report for key insights for practitioners.
Get up to speed fast on the state of app sec testing with TechBeacon's Guide. Plus: Get Gartner's 2021 Magic Quadrant for AST.
Get a handle on the app sec tools landscape with TechBeacon's Guide to Application Security Tools 2021.
Download the free The Forrester Wave for Static Application Security Testing. Plus: Learn how a SAST-DAST combo can boost your security in this Webinar.
Understand the five reasons why API security needs access management.
Learn how to build an app sec strategy for the next decade, and spend a day in the life of an application security developer.
Build a modern app sec foundation with TechBeacon's Guide.
