Software security measuring stick takes off, but is it all that?

For some companies, software security is going critical, as evidenced by the latest data release of the Building Security in Maturity Model (BSIMM) study.

Introduced in 2008 as a measuring stick for software security, BSIMM includes 113 activities organizations can implement to improve software security. By tallying the number of activities it has implemented, a company participating in BSIMM can compare and contrast its security position with peers that also use this model.

The growth in participation is a sign of increased awareness by organizations of the critical importance of software security to their operations, said Gary McGraw, vice president of security technology for Synopsys. McGraw, who pioneered BSIMM, analyzed nine years' worth of data for this year's BSIMM8 report, along with Sammy Migues, principal scientist at Synopsys, and Jacob West, an architect at Oracle.

"The field has grown markedly. Many more firms are starting to do software security properly and measure themselves with BSIMM."
Gary McGraw

Here are the key findings in this year's BSIMM8, as well as an analysis of the results.

Gartner Magic Quadrant for Application Security Testing 2018

Organizations are jump-starting security initiatives

This takeaway is shown by a decrease from last year's average maturity score among the participating companies. BSIMM began with nine companies. This year, 109 organizations participated in the process, including high-visibility outfits ranging from Adobe and Aetna to Home Depot and Verizon.

As new companies join BSIMM, their initiatives aren't as developed as those of firms that have been participating in the process longer. That pulled the maturity score down in BSIMM8 to 33.1, from 33.9 in BSIMM7. The same is true for the average age for software security groups: 3.88 years in BSIMM8, compared to 3.94 in BSIMM7.

Security maturity improving over time

Of the 36 firms that participated in BSIMM at least twice, raw scores went up in 29 of the organizations, and overall scores increased an average of 10.3, or 33.4%. "Benchmarking is an effective exercise in guiding organizations along the optimal path toward building secure software consistently," BSIMM8 researchers noted.

Maturity varies by industry

Each industry prioritizes certain activities over others, and every industry and individual organization has a different path toward building in security. On average, independent software vendors and cloud and financial services firms are more security-mature than are companies in healthcare, IoT, and insurance. Financial services and cloud firms have notably higher scores in compliance and policy practices, while IoT firms have the most mature software environment practices.

Cloud companies are far ahead of other sectors in security maturity because of their dependence on distributed systems, according to McGraw. "When you have communication between clients and servers and software as a service, you really have to pay attention to security," he said.

"Security isn't a nice-to-have. It's a necessity."
—McGraw

BSIMM offers a constructive approach

BSIMM8's authors are careful to explain that the framework isn't a how-to guide or a one-size-fits-all prescription for software security. "Instead, the BSIMM is a reflection of the current state of software security," the report explained.

"By quantifying the practices of many different organizations, we can describe the common ground shared by many as well as the variations that make each unique," the report said.

The maturity model used by BSIMM has some advantages over other security assessment approaches. "It encourages growth and improvement toward better and better security, rather than decreeing a one-size-fits-all security gold standard against which organizations perform gap analysis and invariably find themselves wanting," said Stephen Cobb, a senior security researcher at ESET, an antivirus software maker.

"It is a constructive approach that produces incremental improvements, which add up over time."
Stephen Cobb

Oracle's West pointed out another plus of BSIMM for companies serious about software security. "One of the greatest benefits firms receive from participation in the BSIMM is access to an active community of like-minded professionals," he said.

BSIMM has two annual conferences—one in the United States and one in Europe—and a private online community. McGraw added that BSIMM can be a useful tool in getting support for application security from the upper echelons of an organization.

"Not only do you get a measurement about how you look, but you get a measurement of how you look compared to firms that do the same thing you do."
—McGraw

"If you take a BSIMM set of measurements with the associated graph to the board and you say, 'Here are our competitors and here [we are],'" McGraw continued, "you can garner resources to improve your security position."

"I've personally presented BSIMM results to CEOs and boards in over 20 companies, and it is a very powerful tool indeed."
—McGraw

Deeper problems remain

Nevertheless, BSIMM has its limitations—and its critics.

West acknowledged that BSIMM does not directly measure the importance a firm places on software security. However, it does show growth in investment and interest in security among a wide variety of companies.

"The overall increase in maturity of firms shows continued and growing investment," he explained, "and the growth of BSIMM membership in new verticals—cloud software, insurance, healthcare, and most recently Internet of Things (IoT)—shows that software security is becoming an important focus for firms outside the traditional areas of financial services and software."

Organizations have performed so poorly in securing software applications that anything to raise awareness of the problem will help, but more than a framework is needed to address the core problem, said John Matthew Holt, CTO and founder of Waratek, an application monitoring and security company.

"You're not going to get there from box-ticking in some framework. It's not going to make security a core competency. It's going to give you some value, but something else has to set in to take things to the next level."
John Matthew Holt

In a landscape where organizations are rushing toward digital transformation, secure code becomes more important than ever, added Jeff Williams, CTO and co-founder of Contrast Security, an application protection company. "The BSIMM itself doesn't measure or demonstrate anything about the criticality of software security," he said.

"Everyone understands that companies should be performing asset inventory and risk management activities to understand their exposure." Those practices predate BSIMM by many years, Williams added. "But companies, both before and after BSIMM, are simply not very good at actually doing it," he said.

"I see no evidence that organizations following BSIMM are markedly better at this than other organizations."
Jeff Williams

Gartner Magic Quadrant for Application Security Testing 2018
Topics: Security