Having been involved in application security testing for the past 17 years, I’m certain about one thing: I could not possibly do the work I do without good tools. As with home inspectors, auto mechanics, surgeons, and the like, I’ve found that you’re only as good as the tools you have at your disposal.
Sure, experience counts, and it’s often the differentiator between a good application security assessment and a great one. However, given the complex nature of today’s applications, there’s no reasonable way to be certain—or even assume—that you can find all of the application security-related things you need to find without using the right tools for the job. There’s not enough time in the day. There’s not enough money available. No one is smart enough.
It’s a simple law of the industry in which we work: To test all the right applications at all the right times on a periodic and consistent basis—and have that testing contribute in a positive way to your overall information security program—you need tools. There's no other way. I test dozens of applications in any given year and have identified some key characteristics of effective application security testing tools. I have also learned how to find the right tools and help vendors make them better over the long term.
This knowledge can be distilled down to four tips:
1. There is no 'one best' application security testing tool
It’s as simple as that. Having used most of the commercial tools and numerous free options, I’ve learned that each has its own unique characteristics and strong points. Some are better at finding security flaws than others. Some have amazing reporting capabilities while others do not. Some are super simple, while some are feature-rich and amazingly powerful. If there’s anything I can recommend, it’s to do your research and test out prospective application security tools in your own environment. You’ll know in short order which one feels right and meets your needs.
2. Minimizing false positives and saving time are just as important as finding vulnerabilities
Quite often, certain Web vulnerability scanners will blindly scan for vulnerabilities and document their findings without checking their own work, so to speak. If a finding appears to be out of line or a near-certain false positive, it's great for the user if the scanner will flag the finding as "Potential" or "Needs confirmation." For example, a scanner would see that a potential Oracle flaw and a confirmed SQL Server flaw exist on the same system and would alert to you to the fact, possibly including supporting evidence so that you don't have to figure it out yourself. Or, the fact that IIS vulnerabilities were found on a system that also appears to be running Apache. It seems simple enough to work around but it’s not always, especially in larger environments and for consultants like myself who may not initially know the full configurations of the systems being tested.
3. The user experience is just as important as the quality of the tool’s findings
This is especially true if you do a lot of application testing on a regular basis. Some web vulnerability scanners look, feel, and work as if no one on the development team has ever actually used the product. Some tools make you manually click every time you want to run some basic scans or reports. You can't deal with that behavior if you want to automate the process.
For my security testing, I need to have an automatic function that can pause a scan in progress, exit the program, reboot, and go back in and resume the scan. Some might need their tool to detect when a testing computer is not connected to the network and send an alert so that you're not sitting and waiting on a failed scan. Small conveniences can also be a big deal, like allowing the user to right click on a discovered vulnerability and launch that finding in a web browser. Even the ability to configure the scanner to email you when a scan is complete or when the scan is having trouble can be important.
It might seem trite, but it’s not. This is a tool you may be using for hours on end each day, so you want it to be convenient to use. Furthermore, you want to have a line of support to the vendor or open source community so you can tell them about bugs and feature requests. Many vendors will do this—sometimes immediately—and it’s great.
4. Enterprise-level testing capabilities are helpful, and sometimes required
Reporting is critical today given all the compliance regulations. Does the tool have scan policies and reports that support the specific regulation(s) you must follow? Auditors who read these reports love this. Perhaps more advanced vulnerability management and trend reports are necessary? Maybe you need to install a sensor on the web server(s) being tested, enabling in-depth testing. Minimizing false positives is important once again, because in a large enterprise, you can't afford to have hundreds or thousands of distractions for your team.
Get smart, and get cracking
Working smart means taking a smart approach with application security tools. What I have learned is that it is important to look beyond traditional network vulnerability scanners. Some tools include certain checks for web servers and popular web flaws, but they’re not designed to find all the nuanced application vulnerabilities you’ll have in your environment. I still see organizations relying on data from the wrong tools. This is the quickest path toward making bad decisions around application security.
It's also a fact that some application vulnerability scanners are simply better at finding flaws than others. The reality is that they almost all find different application vulnerabilities, especially as it relates to cross-site scripting and SQL injection. Given that, it certainly doesn’t hurt to use multiple tools whenever you can.
So what are your best practices for picking application security tools? Share your thoughts in the comments section below.
Image credit: Flickr
Keep learning
The future is security as code. Find out how DevSecOps gets you there with TechBeacon's Guide. Plus: See the SANS DevSecOps survey report for key insights for practitioners.
Get up to speed fast on the state of app sec testing with TechBeacon's Guide. Plus: Get Gartner's 2021 Magic Quadrant for AST.
Get a handle on the app sec tools landscape with TechBeacon's Guide to Application Security Tools 2021.
Download the free The Forrester Wave for Static Application Security Testing. Plus: Learn how a SAST-DAST combo can boost your security in this Webinar.
Understand the five reasons why API security needs access management.
Learn how to build an app sec strategy for the next decade, and spend a day in the life of an application security developer.
Build a modern app sec foundation with TechBeacon's Guide.
