21–30 of 87 posts
Earlier quoted context omitted.
Poorly? Almost none. Well? Significant.
Unhelpful and arrogant. We are amateurs in the eyes of the gods.
It’s a bit complex to keep something running for a long time.
It’s very difficult to make something fault tolerant with no data loss or corruption.
Arrogance is thinking that these things are easy because you can read a quickstart.
Earlier quoted context omitted.
Because kubernetes is a fantastic platform that provides a lot of node and app management functionality out of the box. Stateful services like databases are not really a challenge anymore given recent developments in both K8S and database containerization.
> Stateful services like databases are not really a challenge anymore I've faced so many issues with the CSI driver which doesn't attach/deattach properly when you modify anything on the Statetfulset. Additionally, you need to pin your DBs ("statefulset workloads") to certain Nodes based on the AZs since a block storage like an EBS cannot really be attached from Zone 1 to Zone 2 automatically. Based on these limitati…
I’ve not had issues with EBS/AWS, and affinity configs meant it was really easy to spread nodes throughout AZ’s.
On Azure though, volume mounts/unmounts were an absolute dumpster fire.
I always thought it's better to host distributed database, like Cockroachdb and Yugabytedb, in multiple kubernetes clusters. Just in case one of the cluster goes down.
A K8S cluster is already designed to be highly available and self healing. Stretching across clusters is usually a special case of scaling across datacenters/regions and brings a lot more overhead and complexity.
Why would i want to run my database in k8s?
Earlier quoted context omitted.
Because kubernetes is a fantastic platform that provides a lot of node and app management functionality out of the box. Stateful services like databases are not really a challenge anymore given recent developments in both K8S and database containerization.
> Stateful services like databases are not really a challenge anymore I've faced so many issues with the CSI driver which doesn't attach/deattach properly when you modify anything on the Statetfulset. Additionally, you need to pin your DBs ("statefulset workloads") to certain Nodes based on the AZs since a block storage like an EBS cannot really be attached from Zone 1 to Zone 2 automatically. Based on these limitati…
Having Postgres replicate the data across AZs or run Ceph on top of EBS to do it at the storage level are options. The complexity of setup is made simpler by running their respective operators.
Why would i want to run my database in k8s?
if you run your stateless services on an orchestration system like k8s, you have two options: a) run your stateful services (such as Postgres) on the same orchestration system b) choose a second orchestration system (often a more "traditional" infrastructure provisioning system like Chef/Puppet/Ansible/SaltStack) that will only manage your stateful services, and deal with the care & feeding & mental overhead of havin…
The main force that's pushing us to move our storage into K8s isn't simplifying our node configuration, but to keep parity between our prod and more dynamic staging environments. For the latter k8s storage operators are great because we don't generally care about data durability or performance.
Earlier quoted context omitted.
Because kubernetes is a fantastic platform that provides a lot of node and app management functionality out of the box. Stateful services like databases are not really a challenge anymore given recent developments in both K8S and database containerization.
With a database I generally want to have it on its own dedicated host with fast NVME drives, memory, cpu etc, maybe some OS specific system level configuration options for performance etc. I don't want to binpack it on a cluster of nodes each contending for system resources. Also RDBMS databases are not really designed to be ephemeral/elastic, they are stateful/persistent by nature. What does Kubernetes offer in this…
A kubernetes node can advertise those local block devices as persistent volumes. Your StatefulSet can request that class of local storage and the kubernetes scheduler will then find a volume for the pods on a node that the pod can fit on.
You then get all the features of kubernetes operators plus the performance of dedicated nodes and local storage. When a local storage node disappears, your pod gets a new volume on a new node and can replicate from its peers, or start from a backup.
Kubernetes then offers metrics, certificate management, secret storage and management, load balancing, healthchecks, debugging tools, rich firewalling, RBAC, and a huge ecosystem of plugins. It gives you a unified API for all your workloads. Requesting a database then is as simple as a single API call to kubernetes with a spec for your database (ex: creating a MysqlCluster object).
The company I work for runs MySQL, ZooKeeper, memcached, hbase, hadoop, kafka, elasticsearch, and more in kubernetes with great success. Scale here in one prod region is ~10,000 pods on a clusters of ~600 nodes. There are about 900 separate MySQL clusters consisting of 3 replicas each, 150 separate zk clusters of 5 replicas each, etc.
Earlier quoted context omitted.
Poorly? Almost none. Well? Significant.
Unhelpful and arrogant. We are amateurs in the eyes of the gods.
I would like to see a comparison between this operator and the one from Zalando.