In April and May of last year, more than 1,400 vulnerability researchers and hackers targeted specific systems at the US Department of Defense, finding 138 vulnerabilities. The onslaught was not a new nation-state attack, but a pilot program dubbed "Hack the Pentagon," aimed at discovering whether paying bounties for software security bugs could significantly help US cyber defenses. The cost to taxpayers? $150,000.
"It's not a small sum, but if we had gone through the normal process of hiring an outside firm to do a security audit and vulnerability assessment, which is what we usually do, it would have cost us more than $1 million," Ash Carter, the US secretary of Defense at the time, said in a statement.
The success of corporate bug-bounty programs and the Hack the Pentagon program, which is now being more generally adopted by other groups in the Department of Defense, has spurred other organizations to consider the strategy.
Three types of models exist for bug bounties, or vulnerability disclosure and reward programs. Companies such as HackerOne and BugCrowd help software makers develop and manage public and private vulnerability-research programs. Larger technology companies such as Microsoft, Facebook, and Google run their own programs. And programs such as the Zero-Day Initiative offer vulnerability researchers payment for finding flaws in enterprise software and popular consumer applications.
Today, any application security team can benefit from bug bounties. Here are three steps to make the most of your initiative.
1. Start with a small, private affair
Jumping into bug bounties with a public program that rewards researchers is not recommended. Such programs can be overwhelmed with reports of varying quality, and sorting through all those results will take time.
Instead, a company should start with a private program, inviting a handful of researchers to find a limited set of vulnerabilities in a single application, said Adam Bacchus, the chief bounty officer for HackerOne, a vulnerability-program management firm.
"If you start with a public bug-bounty program, it is like jumping on a 1,000cc motorcycle when you have never ridden a motorcycle before," he said. "With a private program, you work out the kinks in your process with a handful of hackers and a much smaller scope."
2. Promote good reporting, especially impact
A big issue for companies is that bug-bounty programs can produce a lot of noise—reports that do not pan out or, because many researchers are not native English speakers, that are not easily understood.
For that reason, companies should focus initial efforts on creating reporting standards and making sure that researchers with proven track records are part of the first programs.
A good report should follow similar guidelines to those for internal quality-assurance reports. Researchers need to explain the feature that they are testing, what type of behavior was expected, what actually happened, steps to reproduce the issue, and, most importantly, the impact.
"That proof of concept is very important," BugCrowd's Baker said. "If you have the PoC, it is easy to show developers that the impact is plausible."
Getting as much information as possible in an organized way is important. However, sometimes a creative report can have an impact: In one case, HackerOne encountered a researcher who sent in a video that showed him exploiting the issue.
"It's not as easy to remediate as if you got a good [static analysis] control flow trace, showing each line of source code and what is going on, but it worked," HackerOne's Bacchus said. "The same things that you would want from a good QA bug report are the good things that you want from a good bug bounty report."
3. Look for lessons in your data
Bug-bounty and vulnerability-disclosure programs are really great at sussing out individual vulnerabilities that have slipped through existing security processes. Companies should fix every bug submitted in a whack-a-mole approach.
However, at the end of the day, the data needs to be analyzed for lessons, Bacchus said. "The real value is taking that data over time and finding systemic issues," he said. "If 80% of the bugs coming in are cross-site scripting, then that is probably a problem that needs to be fixed. If you sit down the developer and tell them that we have spent $100,000 in bounties on your app, that should make them listen."
Bug bounties grow up
Running a bug-bounty program these days marks a departure from a decade ago, when researchers had to worry that, if they reported a bug, they could open themselves up to civil lawsuits or criminal prosecution. In fact, the trajectory of profiting from vulnerability research has followed the path of digital music, said Dan Cornell, CTO for the Denim Group, a software security consultancy.
"They took a weird marketplace with messed-up economic incentives and said that this might not be ideal, but here is a structured way of doing this that makes sense for everyone," Cornell said. "And that was all that the market needed to stabilize and make a lot of sense."
Vulnerability disclosure is similarly messed up. There is full "here's the bug, screw you" disclosure, coordinated disclosure, and private disclosure. And all it took was some bug-bounty programs to make the market make more sense.
Now, companies are able to profit from others' research. Rather than just pay for penetration tests, which do not always produce results, software makers can host a bug-bounty program and catch any outlying vulnerabilities.
David Baker, vice president of operations for BugCrowd, said the fact that a bug was discovered by someone from outside the bounty-hosting company under prevailing network conditions can demonstrate the danger to software developers. "The engineering team now realizes that people—who the company doesn't know—are able to find these things," he said.
Bug bounties are not the be-all, end-all
Bug-bounty programs are only the starting point for turning discovered bugs into better security. Companies need to turn research into reports that software developers can quickly act on, said Bacchus. "Security only improves when bugs are fixed, not when they are found," he said, quoting a favorite saying of a previous manager at Google.
In addition, bug-bounty programs are not a replacement for good software security practices and a mature secure development lifecycle. The cost of remediating vulnerabilities that make it out into the wild is incredibly high. Solving the problem when apps are in production is much more cost-effective, said Cornell of the Denim Group.
"The danger that we see is that a bug bounty does not replace an application security program for sensible organizations," Cornell said. "There are a lot of bugs that can be found with automation. And, really, do you want a kid in Egypt to just find the problems with your site?"
Keep learning
The future is security as code. Find out how DevSecOps gets you there with TechBeacon's Guide. Plus: See the SANS DevSecOps survey report for key insights for practitioners.
Get up to speed fast on the state of app sec testing with TechBeacon's Guide. Plus: Get Gartner's 2021 Magic Quadrant for AST.
Get a handle on the app sec tools landscape with TechBeacon's Guide to Application Security Tools 2021.
Download the free The Forrester Wave for Static Application Security Testing. Plus: Learn how a SAST-DAST combo can boost your security in this Webinar.
Understand the five reasons why API security needs access management.
Learn how to build an app sec strategy for the next decade, and spend a day in the life of an application security developer.
Build a modern app sec foundation with TechBeacon's Guide.
