RSA conference stage

From RSA, secrets of secure DevOps: Enable, educate, automate

Many security managers have experienced the stomach-dropping moment when they realize their systems have been breached.  Zane Lackey, the founder and chief security officer of the agile development security firm Signal Sciences, had such a moment when the security team at his previous company detected an attacker who suddenly succeeded in finding a workable exploit.

When he worked at Etsy, an independent security researcher began testing the platform and found a bug. When the researcher went to report the issue, the security hole had been closed. The researcher contacted the company to explain what he had been doing, worried that they might sue him, Lackey said. (They didn't.) 

The attacker was obviously testing the system for vulnerabilities, and following the success, the team expected the next step would be a systems breach. Instead, the attacker disappeared. "It was clear that the exploit had worked, but then he went away. They never exploited it, and that was terrifying," Lackey said.

The incident underscores that solving problems posed by vulnerabilities requires not just visibility but speed, so that incidents can be quickly blocked. Companies need to work to automate security checks, make their security team agile and focus on enabling developers, not blocking them, Lackey said.

"[Embracing] DevOps, embracing agile, is a net positive for security. This is the level that we need to get our organizations to." —Zane Lackey

Security experts met at RSA Conference last week to share DevOps security best practices. Here is a round-up of some essential steps to secure your organization's application delivery.

What Is the True State of Security in DevOps?

Don't be a blocker

Companies that are implementing DevOps need to recognize that the model does not just apply to developers, but to the entire organization, said Henrik Johansson, security solutions architect with Amazon Web Services, who was also speaking as an expert at the conference. In particular, security teams have to be on board with the move to faster development and automated testing.

If you are not a service team today, you will have a really hard time in the coming years, he said. 

"You cannot be a blocker—your job is not to say no to innovation. Security needs to be everyone's job." —Henrik Johansson

Nir Valtman, head of application security at NCR Corporation, told attendees about his experiences deploying DevOps methodologies across a large portfolio of applications. For a few security team members to help direct hundreds of developers, it requires that they work together, he said. Rather than get in the way, the security team should measure specific metrics and provide feedback to the application developers.

"You don't want to be a speed bump, you want to be a speed camera." —Nir Valtman

Organize for speed and accountability

Valtman organized his application-security team into four specialities, breaking down the security responsibilities to work best with the agile process. 

Application security architects help determine the security design requirements and help developers create code to incorporate strong security into their apps. The application security engineer is like a penetration tester, but with added sophistication and responsibility. The engineer must create specific tests for vulnerabilities, once those issues are found, then push those tests to the quality-control team to automate testing.

When he finds a vulnerability, he has to actually produce the test and push it into Q&A, Valtman said. The reason? "[First] of all, it may break the build, and second of all, it makes sure that no one will repeat the problem. That makes it different from a penetration test," he said.

Because incorporating security into the DevOps process involves applications, Valtman's teams also adopted the application security program manager role to take responsibility for those programs and specific security efforts.

Finally, the security risk and compliance manager is responsible for ensuring that applications conform to security requirements and regulations. They also need to track escalations with a view of reducing the number of incidents, and manage all the training programs—a key function for compliance.

Valtman estimates that the portfolio requires 20 information security people. But filling all of these positions is difficult, he said. "In my experience, it is difficult to hire application security people," he lamented. "In this market, show me how to do that." 

While you can have a small team, assigning team members is critical, especially for Internet-facing applications. Internal applications that are key for large revenue streams are also critical for staffing, Valtman said.

Security needs to scale

Adopting DevOps means that developers will quickly build features into programs and that bugs will creep into deployed code. Automation aims to reduce the number of defects and security issues that are deployed, and it aims to catch issues in deployed code. From code validation to network scanners, automated testing is critical to DevOps and to securing development in a DevOps environment. 

Amazon's Johansson said automation allowed for a programmatic way of application security testing in fast-paced development environments. "All of this amounts to security at scale, being able to take automatic actions without the need for human interaction," he said.

Many traditional security roles have to be modified for DevOps. Signal Sciences' Lackey said threat modeling, for example, can be a hurdle for productivity, slowing down development and adding a process that can stretch out and consume time.

Companies need to shift from the mindset that security is a gatekeeper that needs to find all bugs before software is deployed, and instead move toward the notion that security is a constant observer and code checker through automation. While he admitted that was a bit of hyperbole, the intent is to have the creative people—the developers—be thinking like the attackers as they develop and deploy their code, Lackey  said.

Keeping up with the Joneses

Finally, companies need to remember that automation can help bring visibility. Just because the security managers cannot see the changes made by developers does not mean they are not happening. By adding proper logging, they gain visibility.

"If you have automation, most likely the attacker will use automation, [which means that] failure is always an option, but now it is at script speed." —Henrik Johansson

What Is the True State of Security in DevOps?

Image credit: Flickr

Topics: AgileApp DevSecurity