Get real on container security: 4 rules DevOps teams should follow

Containers are taking off as a fundamental way to deploy applications to the cloud. More than three-quarters (78%) of companies are adopting continuous integration and continuous deployment (CI/CD) processes, the best known being DevOps. Increasingly, that means adopting containers as a way of easily deploying applications to the cloud.

Yet security continues to be a concern with cloud deployment. In the past two months, misconfigurations of Amazon AWS S3 buckets—used to store persistent data for containerized applications—led to the leak of client information from financial-services firm Dow Jones, voter information from a third-party provider to the Republican National Committee, and Verizon customer data from a storage server set up by third-party provider NICE Systems.

Using containers is a tradeoff: Unlike virtual machines, containers are not isolated via a hypervisor, but through the operating system's security, with additional security modules added. Containers are similar to a sandbox, but fall short for the most part, because most containers are not customized to the extent necessary to be application-specific.

Nonetheless, applications running in containers with support from security technologies such as AppArmor and Seccomp are much safer than applications running on a common operating system without such limits, Google engineer and container guru Jessie Frazelle wrote on her blog.

"Containers are not going to be the answer to preventing your application from being compromised, but they can limit the damage from a compromise."
—Jessie Frazelle

Security has been a focus of companies supplying container technology for the past three years. The result: Containers are secure as long as developers and operations follow some basic rules. Container security experts share four.

1. Trust no one

Developers should not use just any container image.

A 2016 report by Banyan Ops, a security firm focused on containers, found that more than 30% of the container images in official public repositories of Docker Hub were vulnerable to security threats. A broader study conducted this year found the situation to be even worse: About 85% of the latest official images had a high-severity vulnerability, according to researchers at North Carolina State University.

The numbers underscore that paying attention to the packages and components that make up a container image is extremely important, especially if the container is from a public repository. Managing the software supply chain requires that companies regularly test their container images for vulnerabilities and vulnerable dependencies.

The first lesson is for developers to use container images from sources that they trust, said Anders Wallgren, chief technology officer at Electric Cloud, a software deployment company. "Use images of well-known provenance. If you are going to use Ubuntu, use the published machine instance."

In addition, any container image—whether sourced or homegrown—should be frequently tested for vulnerabilities and unwanted software. Luckily, software testing can be easily automated, and should be.

[ Learn how to secure enterprise container environments on TB Learn. ]

2. Be a control freak

Moving fast in DevOps has advantages but can also lead to misconfigurations and software errors being quickly published to production applications.

"Human error and misconfiguration are the largest sources of risk," said Vikram Kapoor, co-founder and CTO of Lacework. "People are busy. … These things happen, and they will keep happening. The important part is that when it happens, you need to find it fast."

One way to reduce the chance of a misconfiguration is to use a container-focused operating system as the container host, limiting the code that has to be vetted for security vulnerabilities.

"What many of these organizations are finding is that once they have a host, there is a bunch of stuff that I don't need—it is just vestigial code," said Nathan McCaulley, head of security for Docker. "There are many ways to that idea to create a minimal operating system for hosting containers."

Docker released Linux Kit at its recent user conference. CoreOS and other companies have released their own stripped-down versions of Linux to host containers.

3. Keep your secrets

Another key part of the development process is to control the API keys, database credentials, and other secrets that are necessary to work with other parts of the application. Keeping these secrets from being part of the container is important.

"The main important part is to make sure that these are not baked into any container, because these containers get passed around to many hosts, and you don't want your secrets everywhere," said Rob Szumski, a product manager with CoreOS, which makes management and orchestration systems for containers.

Using a secrets vault or service can reduce the possibility that these extremely important bits of information may be inadvertently leaked.

"You want the secrets to move to wherever the workload is moving, but you don't want them to be hard-coded," Docker's McCaulley said. "You want secret injection, so they arrive just in time for the application to use them."

4. Patch quickly

An agile software development process married to automated testing can result in finding and patching bugs quickly. While some software errors will find their way into production code, fixing them just as quickly is a good antidote for vulnerability.

"Security is fundamentally broken if you can't update software," CoreOS's Szumski said. "If you can't react to these changes when a bug comes out, that's a broken system that you cannot scale."

A lot of the security in containers comes right out the box. For most companies, being careful which container images they use, verifying changes and their software pipeline by signing off along the way, and securely managing secrets go a long way toward security.

"When you look at all these recommendations, it comes down to organizational hygiene," Docker's McCaulley said. "It is not as if moving to [containers] means that your security concerns go away, but you do get a lot of elements right out of the box."

Understanding the risk

Containers themselves can be vulnerable to a variety of attacks, such as kernel exploits, resource consumption attacks, vulnerabilities that allow breaking out of the container, and poisoned images.

In January, for example, Docker patched a privilege escalation vulnerability that could give an attacker the ability to "break out" of the container and masquerade as the host.

"In a lot of cases, what we are seeing is that folks move to a DevOps approach, and the problems that end up happening when you take an agile approach, many of the problems are solved by containers," said McCaulley.