For more than a decade, the number of vulnerabilities reported in the National Vulnerability Database (NVD) was fairly stable, varying annually between 4,000 and 8,000 reported issues and for the most part decreasing or increasing by only 10% to 20%—and in only one case, by half—in any given year.
Yet, in 2017, something wild happened. In a single year, the number of reported vulnerabilities jumped by 127%, more than doubling, to 14,600. Rather than an anomalous spike, the higher vulnerability count became the new normal: The number of software flaws logged in the NVD rose to 16,500 in 2018 and is on track to tally a similar number this year.
As part of its annual Application Security Risk Report, Micro Focus's Software Security Research team analyzed the vulnerability data provided by the NVD and noted the continuation of the trend in the 2019 report.
The reasons for the surge in 2017 are unclear, but additional investigation revealed two interesting relationships in the data. First, the increase in vulnerabilities came from a larger collection of software products. On average, each new product included in the NVD accounted for an increase of about 2.3 vulnerabilities. Second, the increase in vulnerabilities came from a larger number of classes of programming weaknesses.
For every year prior to 2017, at least 80% of all vulnerabilities could be placed into 10 categories of software weaknesses, as defined by the Common Weakness Enumeration (CWE) classification. This jumped to 15 categories in 2017 and to 19 categories in 2018, demonstrating a broadening of the research scope required to find more vulnerabilities.
Here's what's behind the surge, and what you need to know about the state of vulnerability reports as you review your own application security stats.
Organic growth in vulnerabilities a small part
Changes in a dataset can be for two reasons: The underlying phenomenon has changed in some way, or the way the data is collected has changed.
Micro Focus looked for possible explanations for the trend in the underlying phenomenon. Did an increase focus on application security by companies result in more reported vulnerabilities? Did bug bounties spur more researchers to spend their time on finding vulnerabilities in software?
There certainly is a greater collection of software. Undoubtedly, the increase in people writing code, and commensurately more software, accounts for some of the surge, said Dustin Childs, communications manager for Trend Micro's Zero Day Initiative (ZDI), which crowdsources vulnerability research.
"There are more bugs to be found because there's more software out there," he said. "And we all know that—until something magical happens in the future—if you're shipping software, you're shipping bugs with it."
In addition, bug-bounty programs, such as ZDI, have accounted for a greater number—and share—of reported software vulnerabilities. Nearly 8% of public vulnerabilities were reported through bug bounty programs in 2018, a new high, according to vulnerability-information firm Risk Based Security.
In the regularly scheduled update in May 2019, researchers submitting vulnerabilities to the ZDI discovered 44% of the 84 vulnerabilities fixed by Adobe and 19% of the 79 flaws fixed by Microsoft.
Newer technologies also targets
"We are getting more bug reports in industrial control systems; the Internet of Things is being targeted as well," Childs said. Attackers are looking at the newer technologies, he said.
These organic increases would not by themselves create the dramatic increases evident in the data. If bug bounties were the reason for the increase—or if a subset of companies had initiated secure-coding projects—then the top companies or products would account for most of the change in the data. This was not the case, according to the Micro Focus analysis.
Instead, the number of products covered by the latest vulnerability data dramatically expanded, as indicated by the strong correlation between the number of products covered by the NVD and the number of vulnerabilities found in each year.
Instead, the main impetus for the surge in vulnerabilities is likely improvements to the process of assigning the Common Vulnerability Enumeration (CVE) identifiers. (MITRE Corp. creates both the CVE and CWE databases, which report and measure different things.)
MITRE as a single point of failure
In 2016, MITRE had significant problems keeping up with the accelerating pace of vulnerabilities. The government contractor, which manages the CVE process with tight control, was very slow to issue vulnerability reports and assign CVEs, to the point where some security researchers were abandoning the program, said Kent Landfield, chief standards and technology policy strategist at security firm McAfee and founding CVE board member.
"MITRE was a single point of failure. They had some antiquated processes that worked well for a couple hundred CVEs a month, and when you started to have a few thousand a month, you are talking about something that was beyond their ability to deliver."
—Kent Landfield
The result: underreporting of vulnerabilities. In 2016, the NVD contained about 6,400 reported vulnerabilities, only 40% of the number of issues documented by the private database maintained by Risk Based Security. (RBS's own database credits 9,300 vulnerabilities with CVE identifiers, resulting in a less stark 58% coverage.)
Brian Martin, vice president of vulnerability intelligence for RBS, noted:
"[CVEs were sitting] in reserve status for days, weeks, months, or even years."
At the urging of its board, MITRE began revamping the CVE assignment process. The number of CVE Numbering Authorities (CNAs) jumped from 22 in 2016 to nearly a hundred today. Initially limited to large software companies, the CNAs are a more diverse group today and could eventually include open-source software communities, such as GitHub.
"The most important thing we have done is to recognize—given this broader landscape of organizations who are dependent on CVEs and a greater number of software-enabled devices—that it was really important to federate CVEs," said Peter Sheingold, cybersecurity portfolio manager for DHS at MITRE. "If it is just MITRE issuing CVEs, there is a certain limit on that. We don't have to run the math on that."
The improvements gained impetus when a 2016 article highlighted the gap in vulnerabilities and Congress began to take note, opening an investigation in March 2017. The House Committee on Energy and Commerce found that the program's budget had been cut by more than two-thirds, falling to $1.7 million in 2015 from $6.7 million in 2012. In 2016, the program's funding jumped to $4 million, an increase of 139%.
In August 2018, the committee issued a letter to MITRE recommending that the CVE program be reviewed every two years and be funded more consistently.
Prepare for a second surge
As part of its ongoing efforts to revamp its CVE operation, MITRE is looking at how to better cover the open-source community and ensure that those vulnerabilities are assigned CVEs.
Every open-source project is different, said Chris Levendis, the project leader in charge of CVE at MITRE.
"Some open-source projects are run by one person out of a garage somewhere. Other open-source projects have big teams of contributors that are well-structured and well-run. So as we expand into open source, we have to figure out how to work with the open-source world."
—Chris Levendis
In addition, the group plans to welcome international contributors to broaden its coverage of software in other countries. The result could be a second surge in the number of vulnerabilities, Levendis said.
"Every time a new product ends up being covered by the program, you are going to see multiple vulnerabilities in that product."
In 2016, MITRE and the CVE board focused on growing the community responsible for reporting and logging vulnerabilities. Now, with outreach to the open-source and international communities, the group will undoubtedly be expanding again.
The result will be continued growth in the number of vulnerabilities assigned a CVE and, more than likely, some chaos, said McAfee's Landfield.
"We are changing the landscape in how we deal with software today. And whenever you do that, there are bumps in the road."
—Kent Landfield
Keep learning
The future is security as code. Find out how DevSecOps gets you there with TechBeacon's Guide. Plus: See the SANS DevSecOps survey report for key insights for practitioners.
Get up to speed fast on the state of app sec testing with TechBeacon's Guide. Plus: Get Gartner's 2021 Magic Quadrant for AST.
Get a handle on the app sec tools landscape with TechBeacon's Guide to Application Security Tools 2021.
Download the free The Forrester Wave for Static Application Security Testing. Plus: Learn how a SAST-DAST combo can boost your security in this Webinar.
Understand the five reasons why API security needs access management.
Learn how to build an app sec strategy for the next decade, and spend a day in the life of an application security developer.
Build a modern app sec foundation with TechBeacon's Guide.
