You are here

You are here

How effective is your DevOps security integration?

Dan Cornell CTO, Principal, Denim Group
How effective is your DevOps security integration?

Another Black Hat has come and gone. And, for me, Black Hat has come to be synonymous with Black Hat Arsenal, an exhibition space and demonstration area for researchers and developers to highlight free and open-source tools they’ve recently released.

This year marks the fifth year in a row that I’ve had the opportunity to demonstrate the ThreadFix Community Edition application vulnerability resolution platform. I always count on Arsenal to create a space in which I can introduce it to new users and meet with existing users to learn more about their needs and upcoming projects.

Two major themes and questions this year

While at Arsenal, it is encouraging to hear the chatter from the folks who stop by our booth. Our users tend to be responsible for fairly sophisticated application security programs. In most cases, they are running multiple tools and trying to take a much more quantitative view of their programs. These users act as live case studies for us. Once we are able to understand what they are struggling with and the goals that they have set, we are able to gain a solid understanding of where the overall market is headed.

This year, two questions came up repeatedly that resonated with those attending our demos:

  • What are the effective testing tools?
  • How do I infuse security into my developers’ continuous integration/continuous delivery (CI/CD) pipelines?

Testing tool effectiveness

Visitors tend to come though my Arsenal presentations in groups. People gather, a few move on, and others come to replace them. One question that rippled continuously through the crowds was, “What is the best tool for security testing?” Given ThreadFix’s ability to ingest the results from most of the major SAST, DAST, and IAST tools, we’re in a position where organizations expect us to have strong opinions. Though I have personal opinions on the topic, since it acts as a hub for this type of information, I prefer to stay neutral. With that said, our recommendation is consistently to perform testing and benchmarking to figure out which tools are right for the applications in your specific environment. ThreadFix provides reports that make this possible, so the process simply involves testing the same application with several tools, triaging the results to mark false positives, and then running reports to see how the various testing technologies behaved.

That said, I don’t think anyone (who isn’t a tool vendor) believes that there is one “best” tool. Instead, crafting a comprehensive assurance program means evaluating a variety of techniques and technologies and then selecting those that are going to provide sufficient value. Most organizations have large portfolios of applications: web applications, mobile applications, web services, third-party applications, and so on. Expecting one technology or one vendor to sufficiently address all of these is folly—and the questions that arose at Black Hat indicate that more buyers are getting hip to that reality.

I did refer a lot of the attendees to the exceptional work of Shay Chen on his Security Tools Benchmarking blog. Shay and his collaborators do a tremendous amount of work and have produced what I consider to be a fantastic vendor-independent resource for those looking to compare the behavior of various security testing tools. Currently, his work is very DAST-specific, but I find it to be one of the best publicly available data sources out there for those looking for a starting point to evaluate the effectiveness of various tools. Based on the information in Shay’s work, you can then take a look at how you can benchmark tool performance for the specific applications in your organization.

CI/CD integration

The other theme that came up repeatedly as attendees filtered through my Black Hat Arsenal presentation was, “How do I integrate security into my developers’ CI/CD pipelines?” This question echoes a lot of the sentiments I’ve heard in talking with others in the industry. If you look at publicly available industry data, you see that in the most mature organizations, there is a 1:100 ratio between application security staff and software developers. 1:100! This creates a horrible imbalance where application security teams are outnumbered, and the only way they are going to have a chance to keep pace is to elicit the support of development teams. Offloading certain security testing activities to development teams is a great way to help ease the burden on security staff. It also provides the opportunity to take advantage of the transformation of development teams to embrace agile and DevOps approaches, which marks a shining opportunity for application security engineers to insert themselves into the process.

We’ve seen some really cool work from the folks at Pearson Education, Samsung ARTIK, and Sage Payment Solutions about how they’ve accelerated their application security programs by integrating security into the CI/CD pipelines. I believe this will be a success pattern we’ll see replicated in forward-looking organizations in the future. The scale of the application security problem is simply too great to expect security teams to solve it in a vacuum; they’ll need tremendous support from the development teams they’re trying to help secure.

Finger on the pulse of app sec

I have come to love Black Hat Arsenal. Not only is it a great place to see cutting-edge research and new and exciting tools coming from across the security community, but it has also become an opportunity for me to learn what those at the edge of application security are worried about and where the discipline of application security is headed.

This year, I had the opportunity to talk with a lot of attendees about the effectiveness of security tools and the need to integrate security into DevOps CI/CD pipelines, and that lets me know that these topics will be dominating conversations in the near future. I’m looking forward to Black Hat and Black Hat Arsenal 2017 to gain more insights into what the future holds.

That's my take. I look forward to hearing from the community. Share your thoughts on Black Hat Arsenal in the comments section.

Keep learning

Read more articles about: App Dev & TestingDevOps