Deploying CockroachDB on Kubernetes through StatefulSets
For most users, we recommend deploying CockroachDB through StatefulSets; it’s straightforward and behaves like all of your other orchestrated services. (If you’re interested in DaemonSets, we have some guidance in our documentation.)
This guide covers deploying a stateful application with CockroachDB using Kubernetes’ StatefulSets feature. Topics include:
- An overview of options to deploy stateful applications
- Details about how StatefulSets and DaemonSets work in Kubernetes
- Steps to get your stateful application into production using CockroachDB
How CockroachDB fits with Kubernetes
CockroachDB’s origin story parallels Kubernetes’: both have their roots in Google’s infrastructure. CockroachDB is modeled after Spanner: Google’s scalable and consistent database. Kubernetes is a direct descendant of Borg: Google’s orchestration system.
This shared ideological DNA makes it natural for the two too work well together. Kubernetes’ StatefulSets feature was a huge step towards simplifying support for stateful services. Using it, database pods that are rescheduled to other nodes are able to “keep” the same remote disk and simply re-attach to it on its new Kubernetes node.
CockroachDB was designed to be a highly-available, fault-tolerant database meant to withstand chaotic deployments, which is powered by the Multi-Active Availability model. This feature lets it accept reads and writes on any CockroachDB node without sacrificing serializable isolation. Through multi-active availability, CockroachDB handles rescheduling gracefully. Moving between Kubernetes nodes is no different from a temporary node outage, which CockroachDB is well equipped to handle.
Deployment Overview of CockroachDB on Kubernetes through StatefulSets
So, what does it look like to run CockroachDB on Kubernetes through StatefulSets? Here’s an overview of what your environment would look like:
- A Kubernetes cluster
- A Kubernetes node for each CockroachDB node you want to run, each running in the same datacenter/availability zone
- We recommend putting each CockroachDB node on a separate machine to optimize fault tolerance. The Kubernetes scheduler prefers doing this anyway and if a machine goes down, you want to minimize your loss of nodes.
- We recommend a single datacenter availability zone when using Kubernetes with CockroachDB. It’s possible to deploy CockroachDB on Kubernetes across multiple availability zones, but as of CockroachDB 2.0, it’s not recommended for most users because of the complexity in exposing internal network names across Kubernetes clusters.
- A load balancing service for your CockroachDB cluster
- For StatefulSets, you’ll also have a persistent volume for each CockroachDB node (at the time of writing, StatefulSet support for local disks is still in beta)
- Monitoring for your Kubernetes cluster through a tool like Prometheus
If you have additional questions about anything related to CockroachDB you can ask in our CockroachDB Community Slack Channel. See you in there!
