Earlier this year, a crew of 80 hackers targeted systems at the US Department of Defense and, within hours, started to find critical flaws in the system. But this was no security breach.
The attacks were part of a test that targeted a copy of the file-transfer network inside the Pentagon—a cyber-range designed to allow attackers to find vulnerabilities without putting the actual systems at risk.
The latest effort in the "Hack the Pentagon" program, the exercise aimed to find vulnerabilities so that digital defenses could be shored up before malicious attackers found those same weaknesses.
While such exercises are generally referred to as penetration tests, this effort had a significant difference: The exercise harnessed the power of crowdsourcing, marshaling independent—though vetted—researchers to attack the systems.
Is this combination of crowdsourcing vulnerability-finding efforts and offering bug bounties the key to better security? Experts weigh in on the matter.
Crowdsourced security earns its stripes
Crowdsourced security firm Synack believe the combination is potent. "We always leverage a bug bounty as part of the solution," said Jay Kaplan, co-founder and CEO of Synack. "The best possible results come from the combination of the creativity we get from humans, the incentives that are driven through bounties, and the scale and efficiency we achieve with our purpose-built technology."
In the past two years, bug bounties and crowdsourced security initiatives have taken off. Currently, only about 7% of all vulnerabilities are found through bug-bounty programs, because many major software vendors do not offer rewards to researchers who find vulnerabilities in their software or web services. But the list of vendors that offer rewards to researchers who find weaknesses in their software is growing. The bug bounty-program manager Bugcrowd says there are now more than 180.
While bug bounties and programs have become popular, the evolution of more in-depth crowdsourced security efforts could result in services that more closely resemble penetration tests of the past. Because the services add depth and the ability to test internal systems by using a copy of the infrastructure in a cyber-range, crowdsourcing will offer a much-needed alternative to traditional penetration tests, said Kaplan.
"To bolster quality, it’s not uncommon for security divisions to hire multiple pen testing firms to make sure they have enough diversity to actually be effective. This is solid evidence that today’s pen testing simply isn’t getting the job done."
—Jay Kaplan, Synack
Will bug bounties replace penetration testing?
Complaints about penetration testing usually arise because in-depth penetration testing is rare. Because of a shortage of knowledgeable security professionals capable of doing research-based penetration testing, most companies have signed on with vendors that use automated scans to test software for flaws, said Adriel Desautels, managing partner and CEO of penetration-testing firm Netragard.
"Anyone who tells me that these bug bounty programs can replace genuine penetration testing, I'll tell that they are full of it."
—Adriel Desautels, Netragard
"The people who say it can replace conventional penetration testing—well, yeah, conventional penetration test is ineffective, because it's automated. It is like saying that I can replace a Nessus scan. Well, of course you can," Desautels said.
While bug bounties have been successful at finding software flaws for a fraction of the cost of funding an in-house team or hiring a group of security consultants, penetration testing is still needed to test defenses and suggest better architecture. A penetration test can include the same coverage as bug bounties, but bug bounties rarely include the same coverage as a penetration test, Desautels said.
"A bug-bounty program focuses—in normal cases anyway—strictly on the technical vector. It neglects the social and physical vectors that are, can be, [or] should be covered by genuine penetration testing," he said.
A test for every situation
The result is a continuum of testing that can help companies better secure their systems. Need to make sure that vulnerabilities did not creep into the latest version of your web application? Automated testing can find those issues.
Beyond that, bug bounties can augment automated testing. Security-focused companies can use ongoing bounty programs or a crowdsourced effort to find the low-hanging fruit and close vulnerabilities that they have not otherwise found in their systems. In addition, bug bounties have a much higher true-positive rate than automated tests—every issue found is likely exploitable.
"Most of the work that is actually being done by testing services—I think that crowdsourcing can actually replace and make more effective a large part of that," said Casey Ellis, CEO and founder of bug-bounty service Bugcrowd. "If it is human-powered vulnerability discovery, single answer, commoditized in its nature, it makes sense to crowdsource that."
Yet penetration tests will continue to have two major roles. When companies need a great deal of trust in the people testing their network—either because they are testing critical systems or because they are testing internal systems—a penetration test from known consultants makes sense. "It mostly applies when there are privileges that are involved in the testing that is being done," Ellis said. "But those are the exception, not the norm, for the engagements that Bugcrowd is part of."
In addition, when companies need a more holistic test of their security—along with recommendations for their security processes—penetration testing is a better fit, said Desautels.
"If I want to get into an application, I have a technical vector, I have a social vector, and I have physical vector, and I will go after all three, and I will get in," Desautels said. "So no bug-bounty program can cover those three bases effectively. Not unless someone completely opens up their entire business to the world and somehow protects their data at the same time."
Security testing: All together now
There is room in the industry for all three models: the automated tests, the crowdsourced efforts, and the broad-scoped penetration tests. And more than likely, the three models will be complementary, not cannibalize each others' business, Ellis said.
"I think a lot of what we see today is going to be replaced by the crowd, but I don't think it is going to shrink, or even maintain the size of, the pen testing and consulting marketplace," he said. "It will create the opportunity for the people who are tied up with vulnerability discovery right now to be in a position where they will be able to be freed up to work on incorporating the security into their architecture."
Keep learning
The future is security as code. Find out how DevSecOps gets you there with TechBeacon's Guide. Plus: See the SANS DevSecOps survey report for key insights for practitioners.
Get up to speed fast on the state of app sec testing with TechBeacon's Guide. Plus: Get Gartner's 2021 Magic Quadrant for AST.
Get a handle on the app sec tools landscape with TechBeacon's Guide to Application Security Tools 2021.
Download the free The Forrester Wave for Static Application Security Testing. Plus: Learn how a SAST-DAST combo can boost your security in this Webinar.
Understand the five reasons why API security needs access management.
Learn how to build an app sec strategy for the next decade, and spend a day in the life of an application security developer.
Build a modern app sec foundation with TechBeacon's Guide.
