How playing capture the flag boosts application security

When Mark Hoopes ran security training courses for developers, he often found it hard to keep them engaged. Most of the coders did not understand the importance of paying attention to the security of their code, and many resisted calls to fix vulnerabilities as early in the development cycle as possible.

So Hoopes, a senior application security engineer with Aspect Security, decided to put the developers in the shoes of attackers at the end of training. He deployed an open-source application on a closed network and had the developers look for exploitable vulnerabilities.

Hearing about a website that was hit with a cross-site scripting attack is not nearly as engaging as using an exploit to make a web application do something it was not intended, he said.

"When the developers go back to their desks and start looking at this code ... they have a very narrow view as to what they can do to stop the attack," Hoopes said. "But, allowing them to see it from the attacker's point of view opens their eyes. It really gives developers that broader perspective and makes them more attuned in their development career to security issues." 

DevOps security experts are increasingly looking at gamifying security—turning mandatory training into engaging exercises where developers are tasked with finding ways to attack one another's applications, or sometimes even their own.

Training developers to think about application security requires that the training be more focused, experts say. Here are three recommendations.

1. Make it relevant

Today, several capture-the-flag (CTF) platforms are available for security engineering teams to use as a training tool for developers. In 2016, Facebook open-sourced its platform, posting it on GitHub. Another company, CTF365, has created a capture-the-flag arena in the cloud, allowing people to log in and complete challenges or compete against one another.

Yet developer-focused capture-the-flag platforms need to involve code that the developer knows—not just any application in the cloud, said Kyle Rankin, vice president of engineering operations at financial firm Final Inc. "If you are trying to train a developer, they will not be able to take the experience back to the workplace unless it focuses on their work," he said. "If, for example, it is very red-team-focused—like most capture-the-flag games—that is less useful."

Rankin pitted each of four engineering teams against the other teams' applications. The focus on the company's code made the capture-the-flag more relevant and resulted in useful bug reports.

However, that does not work for smaller teams. And having a development team attempt to attack its own code does not work well, Aspect Security's Hoopes said.

"I don't think you give them a lot of benefit to pointing them back to their own code. They are too close to that code and may be blind to its flaws."
—Hoopes

2. Make sure you are prepared

Instead, Hoopes uses an open-source application from the Internet in which he has already found vulnerabilities. That preparation is key, he said. "As the event organizer, you have to know how the applications are vulnerable. You have to have thought through the process, and you need to have a good idea of what they are going to find and what the consequence of that could be."

Another part of preparing for capture-the-flag is to consider the infrastructure. For example, do you have one copy of the application for each participant—which prevents the participants from interfering with each other—or do you have one copy of the application for the entire class or for certain groups? When Hoopes runs a class, he tends to do the latter and provide one instance of the application per 10 students.

3. Create deliverables from the exercise

Many managers, even those who support security training, may not be on board with the idea of having developers take an entire day off work to hack. As a way to offer up additional deliverables, Final's Rankin requires that every bug be formally put into the company's bug-tracking system—one benefit of having developers attack an internal codebase.

To receive points for your hack, you have to file the bug in the bug-tracking system, Rankin said.

"If you are saying that you are going to take everyone offline for day, that's a big expense. But at the end of this, we had bug reports. At the end, we had all of these great bugs."
—Kyle Rankin

Why gamification works

Rankin took a similar approach to Hoopes, but with a different spin. After an introductory lesson on the fundamentals of hacking, he told the company's four engineering teams that they would be hacking one another's applications in a few months. Then he gave them time to lock down their applications.

The developers started looking at their code differently, with a defensive mindset, as soon as they knew that it would be attacked, he said. He created an in-house deployment, placing a file—the "flag"—on each server. And, when the tournament started, the teams enthusiastically attacked, he said.

"It was a super successful tournament A couple of the teams really got into it. One of them basically rooted everyone."
—Rankin

While capture-the-flag competitions require effort to set up and organize, the payoff is that developers are engaged, and following the contest, are more likely to think about software security, he said.

Capture the flag is time-tested

Capture-the-flag has been used as a computer security training exercise for over two decades. The first capture-the-flag event at Def Con took place in 1996, pitting groups of hackers against one another. A decade ago, Jerome Radcliffe, a security researcher at the SANS Institute, wrote of his experiences playing in a capture-the-flag tournament and then designing his own.

"A lot of these developers, before the [capture-the-flag], did not have a security mindset, but once they knew that their colleagues would be hacking their code, they started to really pay attention," he said. "I could have given a presentation on security for two hours, but this made a far greater difference."

Rankin presented his experiences with capture-the-flag games at the O'Reilly Security conference in Amsterdam last year. Hoopes will present his own experiences at the AppSec USA 2017 conference in Orlando, Florida, on September 21.

Image: Flickr