The move toward agile programming has many security teams looking for fast testing tools that may discover fewer vulnerabilities but are less likely to issue false alerts that slow down developers.
Yet the focus on potential false alerts—or false positives—has resulted in the increasing acceptance of missed vulnerabilities, said Alex Hoole, head of software security for Micro Focus Fortify, a maker of software analysis tools.
While agile development has resulted in faster, lighter-weight tools, those same tools can miss significant vulnerabilities. Static analysis will return a certain amount of false positives, but that is not necessarily a bad thing, he said.
As long as the false positives are at a level that can be maintained, the real cost in security is false negatives. "[If] you have a critical vulnerability and you are not finding it … that is a much larger cost than dealing with the false positives," Hoole said.
Have software development processes tilted too far toward a focus on fast, iterative measures at the expense of security? Purpose-built, faster tools are being increasingly integrated into DevOps processes, but they may be missing significant categories of vulnerabilities.
The problem is that people write code in ways that you don't expect, said Chris Eng, vice president of research for software-security firm Veracode. "I don't know what I don't know. There is no way to say, 'I missed 20 things.' That would be great, but I would just end up fixing those issues and finding them the next time around."
To help your team find the right balance and make it less likely that their efforts to reduce false positives are not exposing projects to more potential vulnerabilities, software-security experts share five key recommendations.
1. Don't consider any single tool a silver bullet
Security professionals should not pin their hopes on any single tool. Static analysis, like any technology or process, is not a silver bullet. While critics will focus on the fact that static tools produce false positives and false negatives, even if you throw the best penetration testers in the world at your software, you are going to have false negatives, said Veracode's Eng.
The key is to not depend on a single technology to secure your code, he said. "Make sure that any one process is not your be-all and end-all. You have to apply complementary testing approaches."
"Static analysis is just one opportunity to find those issues. You have to layer as many techniques as possible to get to zero, even though you will never get there."
—Chris Eng
2. Focus each tool on what it does best
Static analysis tools are good at finding some classes of vulnerabilities, such as SQL injection, and not good at finding others, such as cross-site request forgery. Companies should focus their tools on their strongest areas and turn off other coverage to reduce false positives, said John Steven, senior director of security technology and applied research for security-software firm Synopsys.
"Coverage is the dirty little secret of any security tool or any analysis."
—John Steven
Generally speaking, organizations need to purchase two or three kinds of tools. "And with their static tools, they will have to focus on 7 to 10 classes of vulnerabilities and use other tools or techniques for others areas," Steven said.
3. Fix what you find
The unknown vulnerability will always be a bogeyman of sorts for companies. However, for most companies, the real problem remains being unable to fix the vulnerabilities that they have already found, said Veracode's Eng.
Even though there are false negatives, that is not the challenge, he said.
"More organizations struggle with fixing what they have found to begin with. … Through automation you are finding a lot of valid results, and you are not fixing all of them."
—Chris Eng
4. More powerful, slower tools have their place
The focus on fast code checkers should not come at the expense of more in-depth software analyses. Companies that run security analyses on a real-time or daily basis should take the time to check their code with more in-depth analyses every time there is a major check-in, said Micro Focus's Hoole.
"If you are starting a new project, why not use the more powerful tool so that you are catching issues on every meaningful build?"
—Alex Hoole
He added: "You don't have to run it every build, but every other day or every week. Because on a three-month project, you don't want to do it at the end of the third month."
5. Focus on positive feedback
Many companies, especially those with more mature software development and testing processes, have refocused their efforts from finding the negatives—such as vulnerabilities—to suggesting secure code replacements for potentially insecure software patterns.
Rather than try to find all the vulnerabilities and create a large list of issues that needs to be triaged, the most advanced users tend to focus the tools on delivering secure code alternatives to developers and turning off features that produce false positives, Synopsys' Steven said.
When you shift the focus that way—from finding and supposing the existence of vulnerabilities to avoiding dangerous ways to do things and opportunistically suggesting vetted ways of doing things—you get a much different and more satisfying experience, he added.
"So what I see in the mature organizations is a total sidestep of the problem. Rather than finding the bugs, you are pointing out to the developer where they can take a more positive step."
—John Steven
Share your team's best practices for managing the security data deluge below.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
