The open-source Kubernetes container orchestration project has become increasingly important in recent years as organizations rely on it to deploy applications. With the increased reliance has come increased scrutiny on security, especially at Google, which hosts a managed Kubernetes service called Google Kubernetes Engine (GKE).
In a call with press ahead of the KubeCon conference that runs Dec. 11-13 in Seattle, Maya Kaczorowski, product manager, Security & Privacy, at Google, outlined the steps Google is taking to help secure Kubernetes now and into the future.
“Customers are asking mostly questions around configuration and setting up Kubernetes securely,” she said.
Kubernetes is an open-source effort originally created and led by Google; since 2015, Kubernetes has been hosted at the Cloud Native Computing Foundation (CNCF). Both Amazon Web Services and Microsoft Azure operate their own hosted Kubernetes services, and there are commercial offerings from multiple vendors including IBM, Red Hat, SUSE, Pivotal and Cisco, among others. The GKE service is based on the upstream Kubernetes project and provides Google’s view on how Kubernetes should run in the public cloud.
Kaczorowski said that among the questions that customers ask Google about GKE are ones about infrastructure security, with organizations curious about how Kubernetes security features can be used to protect user identities. Organizations are also curious about the software supply chain and whether or not a given container application image is safe to deploy. She noted that the safety of container application images has become a larger issue for many organizations in 2018, after reports of vulnerable applications in Docker Hub as well as a recent issue in the NPM event stream module.
“Users are worried about what’s coming up in their environment,” she said.
Kaczorowski added that the more sophisticated users are asking questions about runtime security and how to identify a container that’s acting maliciously. Users are also interested in understanding how to conduct forensics on a container that has been impacted by a security issue.
What Google Is Doing
Google isn’t just taking the upstream Kubernetes as is and deploying it as GKE. Rather, Kaczorowski said Google is implementing best practices for security by default.
“We go beyond what’s in open source and put additional restrictions in place to secure users,” she said.
One of the most prominent restrictions that GKE has is a restricted Kubernetes dashboard. Multiple organizations including Tesla and Weight Watchers have had their Kubernetes environments attacked in 2018, due to the simple fact that they left their Kubernetes dashboard open and exposed to the internet. A study from Lacework released on June 19 found 21,169 publicly facing Kubernetes dashboards, and of those, 300 deployments were found to have open administrative dashboards without any required access credentials.
Google also makes use of private clusters and authorized networks to help protect GKE users.
“This is about providing private IP addresses for nodes and then restricting the IP access to the control plane using a set of set of IP addresses from a user’s whitelist,” Kaczorowski said.
Kubernetes runs on top of an operating system; in Google’s case, it’s a minimal operating system that is hardened and has been purpose-built. Kaczorowski said that the minimal OS is based on Google’s Chromium OS, which powers Google Chromebooks. The GKE OS needs to be minimal to reduce the attack surface for potential vulnerabilities, she said.
“It doesn’t need to have a lot of stuff because you bring a lot of stuff with you and your containers, and so Google builds its own operating system for this layer called container-optimized OS, or COS, and it’s built on Chromium,” she said.
Upgrading for security patches is always a best practice for IT, and it’s one that GKE implements with its node auto-upgrade capability. Kaczorowski said GKE manages the Kubernetes control plane for users, including updating that control plane and patching it when required.
Another core Google container security capability was announced at the Google Next conference on July 24, with the launch of the Container Registry Vulnerability service, which provides automatic scans of container images to help identify known vulnerabilities. At Next, Google also announced its Binary Authorization, which verifies that an image meets certain requirements before it can be deployed into production. At the KubeCon Europe event on May 3, Google announced its container runtime security effort, which involves partnerships with Aqua Security, Capsule8, StackRox, Sysdig and Twistlock. In a video interview with eWEEK, Kaczorowski detailed what the container security partnership is all about.
2019 Outlook
Looking into 2019, Kaczorowski sees two core trends playing out in the IT security space. The first one is simplifying everything.
“Right now, the burden on a user to get Kubernetes up and running with the right configuration is quite high,” she said. “So in GKE, we’ve done a lot of work to make that simpler, but in the open-source version, it’s just too much of a struggle.”
Kaczorowski is hopeful that the core open-source Kubernetes community moves toward simplifying Kubernetes and providing better defaults.
Kubernetes-specific attacks are another thing that Kaczorowski predicts are coming. To date, a lot of the attacks against containers can be classified as “drive-by” attacks, where an attacker is randomly scans environments looking for known vulnerabilities, she said.
“The attackers probably don’t even realize they’re attacking a containerized environment, and they probably don’t even care,” she said. “We will probably start to see people scanning more for Kubernetes vulnerabilities, realizing that they’re in a container trying to do something a little bit more interesting in that regard, or purposefully looking for containers to target because they might think that they’re misconfigured.”
Sean Michael Kerner is a senior editor at eWEEK and InternetNews.com. Follow him on Twitter @TechJournalist.