Security firm CyberArk reported on Jan. 14 that it discovered a security risk on the popular Play-with-Docker site that could have potentially enabled an attacker to get access to the host system’s resources.
CyberArk responsibly disclosed the issue to the Play-with-Docker maintainers and the issue has now been fixed. The purpose of the Play-with-Docker (PWD) site is to provide an online resource that easily enables anyone to learn about and play with the popular Docker container application technology. With containers, applications are isolated within an operating system, and the general idea is that users aren’t supposed to be able to escape the confines of the container isolation and see the underlying host operating system.
“We were able to take advantage of a weakness inherent in Linux containers to perform this research,” Nimrod Stoler, Cyber Labs security researcher at CyberArk, told eWEEK. “Unlike virtual machines, which load kernels in every instance, containers share the same kernel code.”
Stoler added that using the same kernel code is what makes Linux containers—and Docker in particular—fast and agile, but it is also their Achilles heel. Containers have become an increasingly popular approach in recent years for application deployment, providing organizations with an agile approach for running and managing workloads both on-premises and in the cloud.
CyberArk is in the business of helping organizations understand privileged access security. For the PWD site, Stoler explained that the CyberArk research team used debugfs, which is a Linux tool that comes preloaded with the operating system, to probe the underlying host’s filesystem for a glimpse into the host’s hard drive. From there, CyberArk’s researchers were able to obtain files from the host filesystem to prepare the attack.
Stoler said that beyond debugfs, CyberArk’s researchers simply read files that were on the container, such as inside proc/, dev/ and sys/ that pointed back to the kernel. He commented that by seeing what files the container had that pointed at the underlying Linux kernel, the researchers were able to chart the borders of the container and test just how privileged it was.
“For example, we could read information from within the container about the underlying host’s CPUs, the type of VM it was loaded on, whether the underlying operating system was patched for certain variants of Spectre and Meltdown, the Bios version and more,” he said. “We also charted the container’s borders by checking the container user’s Linux capabilities, and there we could see that the container user was allowed to load Linux kernel modules.”
The core issue that enabled CyberArk to escape the PWD containers and access the underlying operating system has to do with the use of what is known as a privileged container.
“Docker uses the privileged flag to create a privileged container, but according to our research, privileged containers are a spectrum of containers spanning semi-privileged to super-privileged containers,” Stoler said.
According to CyberArk’s researcher, PWD was using privileged containers but had not been properly securing them. Privileged containers have more rights and access than non-privileged containers. Docker container technology provides a number of mechanisms to help secure containers. One of the approaches is the use of what is known as a user namespace. With user namespaces, there is more visibility and control for individual applications and processes that run on Docker.
“Using a user namespaced container would have limited the impact of this attack, as it would have required us to find a way to elevate privileges on the container before launching the attack,” Stoler said.
That said, Stoler noted that implementation of user namespaces would not have allowed PWD to run its application on the container. He said that there are many cases where an application requires more privileges than a user namespaced container allows. Another approach to securing container isolation is with an additional layer of virtualization abstraction, with a virtual machine (VM) hypervisor. A VM brings along its own operating system and a more strict isolation boundary from the underlying operating system.
“If PWD had each container in a separate VM, this attack would have failed,” Stoler said. “Our research revealed that the difference between VMs and Linux containers directly relates to the vulnerability. The very thing that makes Linux containers more agile—using the same kernel code for each instance—is what creates the weakness.”
There are multiple ways that organizations can choose to run containers inside of VMs. Amazon Web Services (AWS), for example, announced its Firecracker technology in November 2018 as virtual machine manager (VMM) approach to isolating containers. Another option is Kata Containers, which is an effort led by the OpenStack Foundation. Containers can also be run inside traditional virtualization technologies, including VMware.
For the PWD container escape, CyberArk was able to get to the host by way of kernel module loading that abused the privileged container status. Stoler said that PWD now blocks kernel module loading from the container, which limits the risk. For organizations wondering if they have the same issue as the one CyberArk reported on PWD, Stoler said his company is working on a tool to help.
“We’re exploring how privileged containers expand the attack surface,” he said. “Based on upcoming findings, we expect to release a tool that will help organizations minimize and control risk.”
Sean Michael Kerner is a senior editor at eWEEK and InternetNews.com. Follow him on Twitter @TechJournalist.