Container Security

Containers have offered an entirely new paradigm in software engineering. Modern container solutions are considered almost synonymous with cloud infrastructure and distributed systems, as they present numerous advantages in development and deployment of applications. Of course, as is the case with all new, paradigm-shifting technologies, containers have introduced their own unique challenges and threats from a security perspective.

What is container security?

Container security can be defined as the holistic application of processes, tools, and other relevant resources, with the aim of providing strong information security for any container-based system or workload.

For an engineering organization to achieve this, it needs to be pragmatic, constantly evolving and adapting its security posture at every stage of the software development lifecycle. It might seem like such an attitude towards security should be obvious, but it’s especially important to apply it when the context is containers. A key first step is recognizing that new technology necessarily requires new techniques to secure it.

Why is container security important?

Container security is important for the same reason traditional infrastructure security is important: a security compromise puts customer data at risk and potentially exposes a business to millions of dollars in fines, lost productivity, reduced sales, and potential dissolution.

It’s better to say that it’s important to think about container security differently. Containers empower developers and DevOps teams to build and iterate software much faster, but they also bring unique technical challenges and security idiosyncrasies that could catch potentially naive engineers off guard. Assuming that traditional security models will be effective could have dire consequences.

One of the more widely utilized container engines, Docker, provides an excellent example of the need to think differently when it comes to container security. Docker provides the Docker Hub, a public repository of images and container configurations. The scale of the repository provides insight into how popular Docker and containers have become: There are over six million repositories, from which over eight billion images were pulled in the last month alone. Most images, even custom configured ones, typically start from a base Hub image. Unfortunately, research has revealed that even seemingly secure, “approved” images can be rife with vulnerabilities and compromises.

Public software repositories, such as the npm registry, have long been a target of malicious actors, and the Docker Hub is no different. Crypto-minersbackdoors, and outdated, insecure software packages are common. According to the latest State of Open Source Research – even the official images curated and maintained by Docker can have operating system vulnerabilities.

graph of container security comparison 2019 versus 2018
Figure 1: Increase in vulnerabilities in some popular container-based images in 2019 compared to previous year

Four steps to creating secure container images

Maintaining strong container security starts with the source: the base container image. It might seem daunting, but taking a few steps and applying the right tools can have a measurable improvement on your security posture.

1. Secure your code and its dependencies

The first step to container security might be surprising, as it doesn’t actually involve containers. The code, the lifeblood of any software organization, also demands close evaluation and effective security controls. If the underlying code isn’t secure, any steps to secure the containers will have been in vain. Fortunately, this is the portion of the container that’s most directly controlled by developers and (usually) the best understood.

Snyk Open Source helps you automatically detect open-source vulnerabilities and accelerate remediation throughout your development process. Snyk makes fixing code simple with Fix PRs, powered by our proprietary security database for the most popular open-source languages and frameworks.

2. Start with a minimal base image from a trusted source

The base image is one of the most important considerations when it comes to container security. The best way to help ensure safe images is with internal hosting. If an organization lacks the acumen or resources for self-managed infrastructure, there are some different approaches to securing image sources that can help.

The best option for maintaining a secure source of images is to host an internal image repository. A strong DevSecOps culture should facilitate InfoSec and DevOps teams collaborating to provide the infrastructure, monitoring, and auditing of hosting infrastructure. If it’s not feasible to host internally, the next best thing is to ensure that only official images from Docker Hub are utilized as base images. Red Hat, Microsoft, many Linux vendors and VMware / Bitnami are also providing some level of certified container images. These cover almost every programming language and framework in widespread use today. Of course, you can elect to choose base images with just the operating system and package manager and customize the language support on your own as well. Almost all of these providers sign their images with tools like Notary so you can verify their source and authenticity. While this isn’t a guarantee that the image is totally immune to compromise, it at least provides some additional assurance, as well as publicly available vulnerability scan results.

With selecting images from trusted sources and verifying the digital signatures on the images, you should consider selecting images with minimal operating system footprints as well:

  • On Docker Hub you’ll find images that use Alpine, which is a small Linux distribution well suited to containers
  • You can also find images tagged ‘slim’ which generally remove tools like compilers which shouldn’t be needed in production
  • The Ubuntu images on Docker Hub are trimmed down versions of that distribution
  • Red Hat provides their own ‘minimal’ and ‘platform’ UBI images that strip things down to the basics.

Another consideration in securing the image is to make sure that the final image only ships with the bare minimum number of packages and dependencies needed to function correctly. As with traditional hosts, every additional port, daemon, and process that runs provides a broader attack surface for potential compromise. Smaller images also have the added benefits of being faster to compile, scan, and upload as well.

3. Use static analysis tools to provide additional protection

There is a variety of container security tools that can be applied to help maintain strong container security. You can add the tools you need to develop, test, and deploy your applications right after you’ve identified your preferred base image. At each stage of the software lifecycle, you might need different sets of tools, and in production, you likely want to remove everything that isn’t absolutely necessary.

One class of tools that can provide a deeper look into potential container vulnerabilities are static analysis tools. OWASP provides a cheatsheet for Docker security tools and techniques, and #9 is to utilize static analysis. Snyk offers one such tool, Snyk Container, which provides a suite of features aimed at securing containers. Additional tools are available to help detect misconfigurations in Docker, and even Kubernetes if that is being utilized for orchestration.

4. Manage your runtime configuration

Securing the code and underlying image of a container is vitally important, but overlooking the configuration of how the container will behave at runtime could easily undo all the work undertaken beforehand.

The OWASP Cheat Sheet has several guidelines that apply to runtime configuration. There are some relatively easy steps that can be taken to help mitigate and minimize the chance of a potential compromise, such as not allowing the container to run as root and limiting the amount of host resources that can be used. Policy agents such as Open Policy Agent can be employed to enforce configuration policy for Docker files, preventing invalid configurations from making it into production environments.

How to secure your container build pipelines and CI/CD

A good place to start integrating security objectives into the software development lifecycle is the CI pipeline. Although security should be a focus right from the first planning session, a CI pipeline provides a mechanism to automate some of the security process.

As mentioned in the previous sections, tools like policy agents can be employed at various stages of the CI pipeline, ensuring invalid configurations are not allowed into any live environments. Additional tools can be employed to perform scans for potential hard-coded secrets or sensitive data. Containers can also be launched into a sandbox environment as part of a specific CI stage, where they are carefully monitored for abnormal network calls or system behavior. Automating security in your CI pipeline helps guarantee that a baseline security posture is always maintained.

Monitoring containers at runtime

When your containers are deployed and running, you can’t assume they’ll remain safe. New vulnerabilities get reported daily, misconfigurations can permit attacks even if your code and containers are secure, and sometimes users manage to find a way to deploy workloads that go around security and quality gates.

When it comes to container security, both the underlying infrastructure and running code should be continuously, actively monitored. Static configuration monitoring might not alert you to minute changes in the runtime environment until it is too late and a compromise has occurred. Effective monitoring will alert on potential issues before they become actual problems.

monitoring container security at the runtime

The configuration of your deployments should also be checked, preferably in a manner that ties back to the code used to create the deployment so that if there are issues they can be addressed at the source.

Container Security FAQ

Are containers secure?

Absolutely. But, like any infrastructure, they rarely come that way by default. Security is a holistic process, and containers are no different. However, the same processes and tools that were once used on traditional infrastructure might not be adequate to provide strong container security. Containers have changed the landscape of distributed systems, and new methods must be employed to secure them. There is a broad spectrum of container security solutions that can and should be employed to help provide the best possible security when it comes to containerized workloads.

How do I fix security vulnerabilities in containers?

For container images it’s a four-step process: take care of the vulnerabilities in your code and dependencies; choose base images with only what you need – start slim and add rather than deducing what to remove; evaluate the extra tools and packages you add – as containers progress closer to production the number of extras should be zero; and ensure the container is configured to run with as few privileges as possible.

How to secure a Docker container image?

Securing a Docker container image is a multi-faceted process. During the planning phase of an application, it should be ensured that the software doesn’t require any security anti-patterns to function correctly, such as running as root. Docker’s own documentation also provides a great starting point for how to deal with securing an image, specifically addressing the matter of trust. Trust controls can be implemented even on private registries, and is highly recommended. From there, ensuring base images only maintain a minimum profile of packages and dependencies, and utilizing static analysis tools to continuously monitor for vulnerabilities will help to secure your Docker images.

What is container scanning?

In the context of security, container scanning refers to any tool or process that scans a container or container artifact for potential security compromises, misconfigurations, or vulnerabilities. Scanning tools can encompass code, code dependencies, container configuration, and container runtime configuration, as well as other potential domains.

Container scanning can begin before a container is ever introduced, using static analysis and software composition analysis to ensure code and related dependencies are free of vulnerabilities or issues. Next, scanning the configuration utilizes tools like Open Policy Agent, to prevent misconfiguration issues. Finally, runtime scanning ensures live workloads remain in a secure posture throughout their lifespan.

Think differently about container security

Securing containers isn’t about making broad changes to your security; it’s about taking existing tools and processes, and augmenting them with modern container security solutions. Containers enable exciting, disruptive changes to the way organizations develop and ship software, and the right tools will help ensure security can keep the pace.

Jim Armstrong
July 9, 2020
| By Jim Armstrong