Preventing good containers from going bad

Containers go bad everyday, and often without warning. All it takes is one CVE impacting an image, and now all containers deployed using this image are at an increased level of risk of compromise. As the use of containers becomes a standard practice, existing software development and security methodologies will need to better support developing, running, and managing applications made possible by containerization.

The security risks associated with containerized software delivery has become a hot topic in the DevOps community, where Operations teams are under pressure to identify security vulnerabilities in their production environments. But it can be difficult for organizations to see the components and dependencies in all their container images, a task made more challenging when security governance is factored in.

One important operational difference due to the nature of containers is that vulnerabilities identified within containerized applications aren’t patched using traditional patch management processes. Instead, patches to container images are made by rebuilding the Docker image, and then replacing the existing running containers with the updated image. This paradigm shift often requires enterprises to both reassess their patching processes and continuous onitoring requirements.

Managing container infrastructure in a production environment becomes even more challenging due to the scale of deployment. One of the biggest problems is trust—specifically trust of the application. That is, can you trust that all containers in your Kubernetes or OpenShift cluster are performing the tasks you expect?

Minimizing container security risk

Given the level of adoption of open source technologies in container infrastructure, a key to protecting your applications in production is maintaining visibility into your open source components and proactively patching vulnerabilities as they are disclosed. If you assume from the outset your containers will be compromised, you can make it much harder to mount an attack from a compromised container.

Key questions operations teams need to answer in order to minimize risk include:

• What security risks might be present in base images used for your applications, and how often are those images updated?
• If a patch is issued for a base image, what is the risk associated with consuming the patch?
• How many versions behind can a project or component be before it becomes too risky to consume?
• How quickly will I be informed of component updates for dependencies which directly impact my containers?
• Given the structure of a component or project, do malicious actors have an easy way to gain an advantage?

Adopt container-specific vulnerability management tools

You can’t rely on traditional security tools that aren’t designed to manage the security risks associated with hundreds—or thousands—of container images. Periodic infrastructure security scans don’t account for the pace of container application development, nor do they account for the rate of security disclosures. Organizations need to adopt container-specific vulnerability management tools and processes to minimize the potential for compromise.

One critical attribute of any container security solution will be in its ability to identify new pod deployments within the cluster and automatically attest to the security state of the containers within the pod. The desired security state will vary by application, and solutions need a policy framework which allows an “at a glance” identification of any containers violating policy.

The most advanced container security will enable this enforcement by providing a method to prevent containers with security vulnerabilities from being deployed and identify containers based on images which have become vulnerable subsequent to deployment. It will also include centralized reporting and monitoring of the compliance state of each image to prevent noncompliant images from being run when known vulnerabilities are present.

Container images should be monitored continuously because new security vulnerabilities are being discovered every day. But with hundreds or thousands of containers running at the same time, finding and remediating every newly discovered vulnerability in each container can be a challenge. Identifying an issue in a container image means that same issue will be present in all running containers replicated from that image.

An image created with fully up-to-date components may be free of known vulnerabilities after its creation, but at some future point vulnerabilities will be discovered in image components effectively turning a “good” container into a “bad” one. To ensure containers are secure against newly reported vulnerabilities, organizations should utilize a container-native security solution that can monitor the container environment and provide precise detection of malicious activity within.

Finally, organizations should ensure their approach to container security scales to their containerized environment. It only takes one vulnerable container out of thousands to cause a breach—which is why organizations need simultaneous visibility into all container images.

While containers help speed software delivery, they also pose new risks to application security. The most effective and proactive way of controlling that security risk is by finding and removing vulnerabilities in base images. Deploy and use a dedicated container security solution focused on preventing, detecting, and responding to threats specifically aimed at containers.