Application containers have emerged as one of the most important technologies for modern IT infrastructure, and for good reason. Containers and container orchestration tools such as Kubernetes enable administrators to package, deploy, run, and manage applications in a way that’s portable, reusable, and automatable. In so doing, they can dramatically reduce the amount of IT labor and resources needed to manage application infrastructure. Yet this model also introduces new security concerns, ones that traditional IT shops may not have considered.
As with all software, containerized applications can fall prey to security vulnerabilities of various kinds, including bugs, inadequate authentication and authorization, and misconfiguration. Furthermore, containerized applications tend to be complex, comprising many discrete components that communicate with one another over a network. As a result, the total attack surface of the environment can be large, with potential trouble spots in multiple layers of the architecture.
Fortunately, best practices for securing containerized infrastructure are emerging. The National Institute of Standards and Technology (NIST), a division of the US Department of Commerce, has published “NIST Special Publication 800-190: Application Container Security Guide”: a set of guidelines that can serve as a useful starting point and a baseline for security audits. It breaks down security concerns and mitigation strategies for each layer of the container stack. Topics include doing vulnerability scanning, using a container-centric host OS, and employing strong authentication and authorization.
Here's are the key best practices for securing containerized infrastructure in the NIST guidelines.
Container images: Time to reflect
Perhaps the most obvious source of security issues in a containerized environment is problems lurking within application images themselves. These can include outdated, insecure versions of software or libraries; buggy applications; or even hidden malware. Tools that can scan for these vulnerabilities are essential, but the NIST guidelines caution that these should be container-aware, including the ability to scan all layers of a multilayer containerized application.
Flawed or malicious software isn’t the only threat. Poorly configured images can also be a source of vulnerabilities. For example, an image might launch an extraneous daemon or service that allows unwanted access from the network, or it might be configured to run with more user privileges than are necessary. Secrets stored within images, such as authentication keys or certificates, are another danger to watch out for.
NIST recommends pulling images only from trusted sources, such as private container registries, but a poorly configured registry can also be a security issue. Access to the registry should require encrypted and authenticated connections, preferably using credentials that are federated with existing network security controls. Any efforts to secure container images can be rendered meaningless if the registry can be easily compromised. Also, the registry should undergo frequent maintenance to ensure that it doesn’t contain stale images with lingering vulnerabilities.
[ Special Coverage: KubeCon/CloudNativeCon ]
Security orchestration: Lock it down
Container orchestration tools—of which Kubernetes has become the leading example—are another potential target of attack. Pay strict attention to securing the administrative interface, especially in scenarios where a single orchestrator manages multiple applications. This may include such measures as strong, two-factor authentication and at-rest encryption of data. If you don't strictly scope access, a careless or malicious user could potentially do all sorts of mischief, from taking down apps to launching rogue ones.
NIST also recommends configuring orchestrators to separate network traffic into discrete virtual networks, based on the sensitivity of the traffic being transmitted. The idea is that low-sensitivity workloads, such as public-facing web apps, should be isolated from high-sensitivity workloads, such as tax-reporting software. In addition, workloads should be distributed such that each host runs containers only of a given security level. These measures make it much more difficult for a malicious actor to gain access to sensitive data when a low-sensitivity application such as a blog is compromised.
In general, NIST recommends deploying and orchestrating clusters in ways that are secure by default. Examples include end-to-end encryption of all network traffic between cluster nodes and mutually authenticated network connections between cluster members. The orchestrator should be able to introduce nodes to the cluster securely, maintain a persistent identity for each node throughout its lifecycle, and isolate and remove compromised nodes without affecting the overall security of the cluster. These measures are especially important in large-scale environments that span multiple network organizations and scale to hundreds of hosts and thousands of containers.
Contain the containers themselves
In addition to container images and the applications within them, containers themselves can potentially become security issues. One of the more serious concerns arises when the container runtimes that launch and manage containers—software such as containerd, CRI-O, and rkt—themselves contain vulnerabilities. NIST cautions that, left unpatched, such flaws can lead to “container escape” scenarios where an attacker could potentially gain access to other containers or the host operating system itself, so admins should make installing runtime security patches a high priority.
Beyond this, administrators must pay special attention to the many configurable options available with container runtimes. A misconfigured container might be able to access too many devices, for example, which could potentially affect all containers running on the host. Other runtime options could allow a container to make unsafe system calls, mount sensitive directories in read-write mode, and even compromise the host OS.
Containerized infrastructure also makes scanning network traffic for security threats more challenging. Containers deployed on multiple hosts typically communicate over a virtual, encrypted network, and they are assigned dynamic IP addresses that change continuously as applications are scaled and load balanced by the orchestrator. Detecting network traffic anomalies in such an environment requires specialized, application-aware network filtering tools.
Lock down the operating system
At the lowest level of the containerized stack, the host OS represents the most critical target for attacks. If compromised, it can expose all of the containers running on it. For this reason, NIST recommends running a pared-down, container-specific OS that limits the number of installed components to the bare minimum of software required to create and manage containers. Fewer components means fewer potential vulnerabilities that can be exploited.
Even a minimized OS, however, will not be immune to security vulnerabilities. As they would with any software, it is critical that administrators keep up with OS security patches and apply them promptly to all host instances in the cluster. This includes not just the OS kernel, but also the container runtime and any other system services or components recommended by the OS vendor.
Proper OS configuration is also essential. In addition to mounting sensitive file systems as read-only, NIST recommends running the host OS as immutable infrastructure, with no data stored uniquely and persistently on the host. In addition, the host should not provide any application-level dependencies except those that have been packaged and deployed as containers. These measures make the OS a more trustworthy environment, with far fewer avenues for attack.
Operational issues: Automation is key
By now it should be clear that securing a containerized environment is a complex task, and one to be taken seriously. Yet it need not be an overwhelming one. A persistent theme throughout the NIST guidelines is the need for automation of security processes, particularly as the environment scales to hundreds of hosts and thousands of containers. Container orchestrators provide some of this automation, but container admins should also seek to automate such functions as vulnerability scanning and software updates.
Another lesson learned is that software alone cannot guarantee security. Containerization also requires that organizations examine their processes and teams and potentially adjust to the new operational model. The ephemeral nature of containers may call for different procedures than those used with traditional servers. For example, incident response teams will need awareness of the roles, owners, and sensitivity levels of deployed containers before they can know the proper steps to take in the event of an ongoing attack.
Security threats and mitigations are ever-evolving, and no one resource can provide all the answers. Still, the NIST Application Container Security Guide offers a solid foundation and framework for security policy for containerized environments. It’s well worth a read for anyone involved in building, deploying, managing, and maintaining containers and containerized applications, and it's a must-read for security professionals as the industry transitions to this next phase of IT.
To learn more, join Elsie Phillips from Red Hat at KubeCon/CloudNativeCon in Copenhagen for an introductory discussion of the NIST container security guidelines. The 35-minute talk will take place on Friday, May 4. TechBeacon readers receive a 20% discount when they enter code KCCNEUTB.
Keep learning
Choose the right ESM tool for your needs. Get up to speed with the our Buyer's Guide to Enterprise Service Management Tools
What will the next generation of enterprise service management tools look like? TechBeacon's Guide to Optimizing Enterprise Service Management offers the insights.
Discover more about IT Operations Monitoring with TechBeacon's Guide.
What's the best way to get your robotic process automation project off the ground? Find out how to choose the right tools—and the right project.
Ready to advance up the IT career ladder? TechBeacon's Careers Topic Center provides expert advice you need to prepare for your next move.
