eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
At the core of the modern cloud-native application movement is an open-source technology that is perhaps not as well-known as it should be. That technology is the open-source etcd project, which provides a distributed key value store that is used by all the major public cloud providers and is at the core of the Kubernetes container orchestration system.
The etcd project is not new. In fact, it was started five years ago by a team of developers from CoreOS, a company that was acquired by Red Hat for $250 million on Jan. 30. As a distributed key value store, etcd provides a mechanism that enables data to be stored in a stable, reliable and consistent way across a cluster of different machines. As of Dec. 11, the etcd project is part of the Cloud Native Computing Foundation (CNCF), which is also home to the Kubernetes project along with 30 other cloud-native projects.
“Etcd has become successful beyond our wildest aspirations when we started the project five years ago,” Brandon Philips, co-founder of CoreOS and currently working in the office of the CTO at Red Hat, told eWEEK in a video interview. “Today every major cloud provider has etcd employed. It’s a critical part of Kubernetes, and it is becoming a critical part of a lot of cloud-native projects.”
At its core, etcd solves the technological requirement for cloud-native technologies that need a consistent global configuration for a cluster of servers. The etcd technology first found a home in CoreOS’ own Fleet project, which was an early effort at container orchestration that has been superseded by Kubernetes.
A primary element inside of etcd is the concept of distributed consensus for what is running, which is enabled via an implementation of the Raft fault-tolerant consensus algorithm.
Beyond Kubernetes
While etcd is a foundational element of Kubernetes, that’s not the only project that relies on it. Philips commented that over the past two years a number of other projects have embraced etcd. Among those projects is the open-source Vitesse effort, which provides a sharded cloud-native MySQL database implementation. In the database world, a shard is a data segment that is distributed to help improve reliability.
Uber is also making used of etcd as part of its open-source m3 metrics tracking technology. Additionally, Philips said he is aware of multiple organizations using etcd for internal uses.
“It has been really exciting to see etcd end up everywhere,” Philips said. “We always wanted etcd to be a data store for important data that couldn’t be lost in the face of various machine failures.”
Blockchain
There is no shortage of hype around blockchain in 2018, with advocates claiming all manner of uses for the cryptographically secure, digital ledger technology. With blockchain, the promise is that data is distributed and cryptographically verified. It’s a concept that isn’t all that dissimilar to what etcd already enables for Kubernetes and other adopters of the technology.
Philips explained that blockchain essentially has two core properties. The first is distributed trust, and the second is having a database that has a verifiable cryptographic hash.
“Etcd has actually had a verifiable hash feature for a long time. It’s a way of auditing that every machine in the cluster has the exact same copy of data,” he said.
What’s Next?
The current version of etcd is 3.3, and work on version 3.4 is currently ongoing, with a release set for mid-2019. Looking forward, Philips said there is a lot of work being done now to make etcd easier to deploy. Another core feature that is in development is the ability to more easily execute database downgrades.
“A lot of databases, including etcd, enable on the fly, live upgrading of a cluster,” Philips said. “Google and a number of the other larger operators have been working on a design document, so you can downgrade an etcd cluster in live time, as well.”
Watch the full video with Brandon Philips above.
Sean Michael Kerner is a senior editor at eWEEK and InternetNews.com. Follow him on Twitter @TechJournalist.