Live data from Hacker News

PostgreSQL 14 on Kubernetes

blog.crunchydata.com

51–60 of 87 posts

Re: PostgreSQL 14 on Kubernetes

#51
post #3

Why would i want to run my database in k8s?

if you run your stateless services on an orchestration system like k8s, you have two options: a) run your stateful services (such as Postgres) on the same orchestration system b) choose a second orchestration system (often a more "traditional" infrastructure provisioning system like Chef/Puppet/Ansible/SaltStack) that will only manage your stateful services, and deal with the care & feeding & mental overhead of havin…

Aka: k8s is the new hammer and every workload looks like a nail.

I'll be downvoted to hell for this.

Re: PostgreSQL 14 on Kubernetes

#52

Earlier quoted context omitted.

StatefulSets, Node Groups, and Node Affinity/Taints solve some of those issues.

A database is not a service well suited for ephemeral, quick scale down and scale up container workloads that K8s really excells at, and if you have nodes specifically configured to run databases and nothing else then you should simply run them there directly and not mangle K8s to replicate what will essentially be equivalent to running them there directly.

K8S runs workloads across servers in a declarative fashion. Ephemeral or stateful doesn't make a difference anymore.

Whatever you use to manage the nodes directly is basically what K8S provides (and more) already, so you're really just replicating effort and complexity instead.

Re: PostgreSQL 14 on Kubernetes

#53

Earlier quoted context omitted.

Because kubernetes is a fantastic platform that provides a lot of node and app management functionality out of the box. Stateful services like databases are not really a challenge anymore given recent developments in both K8S and database containerization.

> Stateful services like databases are not really a challenge anymore I've faced so many issues with the CSI driver which doesn't attach/deattach properly when you modify anything on the Statetfulset. Additionally, you need to pin your DBs ("statefulset workloads") to certain Nodes based on the AZs since a block storage like an EBS cannot really be attached from Zone 1 to Zone 2 automatically. Based on these limitati…

This is where complexity comes in. Let Kubernetes abstract the resources for your workloads.

For example, use a storage service (like Rook/openEBS/portworx/etc) that uses node-attached disks to create a data layer. Then you can assign volumes to your individual workloads created and managed by the storage service.

This provides reliable and portable storage for your other services without having to tying them to lower-level details.

Re: PostgreSQL 14 on Kubernetes

#54
post #45
post #35

Earlier quoted context omitted.

Kubernetes is great but not everything fits in kubernetes, this is a bad ops team if this is the case.

Note the "except if you have a very good reason" caveat. I think this is a good ops team mantra. PostgreSQL unless you have a very good reason otherwise, and k8s unless you have a very good reason otherwise. Decisions like this allow for economies of scale with tooling, monitoring/observability, etc. Saying no to snowflakes is an important part of having a good ops team.

Guess it depends on who is defining “very”.

A persistent connection is antithetical to the design of kubernetes.

Persistent storage is hard to get right in kubernetes and puts a lot of pressure on the cluster being perfect, which is something a good ops team will tell you is hard.

If your application is stateless http services then kubernetes is great, but there are common things that don’t fit well, and forcing people into a solution which may require a complete rethinking is not very pragmatic.

Re: PostgreSQL 14 on Kubernetes

#55

Earlier quoted context omitted.

A database is not a service well suited for ephemeral, quick scale down and scale up container workloads that K8s really excells at, and if you have nodes specifically configured to run databases and nothing else then you should simply run them there directly and not mangle K8s to replicate what will essentially be equivalent to running them there directly.

K8S runs workloads across servers in a declarative fashion. Ephemeral or stateful doesn't make a difference anymore. Whatever you use to manage the nodes directly is basically what K8S provides (and more) already, so you're really just replicating effort and complexity instead.

Indeed.

k8s orchestration is also useful for running multiple replicas of a database and using leader election to promote a secondary to a primary if the primary goes down. Doing this without k8s would require some kind of clustering anyway, to coordinate the leader election. So if you're already using k8s for everything else, you might as well also use it for this instead of introducing something else like pacemaker.

The one disadvantage is that leader election under k8s does need to be implemented from scratch, because it isn't a first-party concept in k8s. All it gives you is the first-party implementation of compare-and-swap for resources. So you have to make a StatefulSet for your replicas, make a ConfigMap to store the leader election data (which is where the CAS ensures atomicity), and write a sidecar container to handle the healthchecks and leader election and dynamic scale-up/down of replicas. You'll probably end up making an operator to do all that for you so that you can deploy a custom resource for your replicas and they're converted to StatefulSets with Pods containing the sidecars, etc automatically.

Of course, for the big popular DBs, someone has probably already written such operators for you.

Re: PostgreSQL 14 on Kubernetes

#56
The truth is Kubernetes already has all the primitives to run database or any other stateful applications. What is challenging is how to make it reliable, performant and scalable for this kind of apps. If you manage to do it properly, then it will be far less costly and manageable than any DBaaS out there. There are 2 dimensions to the problem: 1) How to easily deploy and operate a "production" DB in K8S. As the toolset evolves, we've seen a lot of different operators and customization tools to facilitate this. Today it is rather trivial to write the manifests that will deploy a Redis cluster fit to your needs. 2) You need a REALLY good kubernetes-native data service layer that guarantees fast failover, sync replication and performance for your persistent volumes. StatefulSets alone won't really help in case of node failure...and don't even think about using NFS. Ondat/StorageOS are leaders in that space, it's really simple to deploy, so you can give it a try in less than 5min. Once you've made sure that your stateful configuration is right and that the network/data layer is reliable, then you can leverage all the visibility tools native to K8S (ELK/EFK, Prometheus, etc). Your stateful apps will be ready for CI and other DevOps integration using the K8S tools (Tekton, ArgoCD etc). This will make your devs happy as infra is not the blocker anymore...

Re: PostgreSQL 14 on Kubernetes

#58
post #3

Why would i want to run my database in k8s?

We run all our databases in K8S, approx 80 DBs. We're a fintech that requires HA so all databases run with at least 3 notes. Maging DB clusters with K8S only requires 1 file.

If we need to increase storage this can be achieved in minutes, if we need to take one node down we basically can do it anytime during the day..

Every now and then we discuss moving DBs to AWS RDS (or equivalent) however for our transaction volume RDS costs are prohibitive.

Re: PostgreSQL 14 on Kubernetes

#59

Best way to run database is in static pods. You'll get all the benefit of kubernetes ecosystem (monitoring, logs, inventory, access control) without any drawbacks.

You mean without a deployment or statefulset? How would you connect to the DB without a service providing a persistent DNS name?

You can create a service point to the `pod` tag. Or a manifest.

Re: PostgreSQL 14 on Kubernetes

#60
post #24
post #3

Why would i want to run my database in k8s?

At least in my case, because your ops team has decided that everything that gets deployed, gets deployed in k8s (unless you have a Very! good reason not to)

A middle ground would be provisioning managed services through Kubernetes resources using a controller like Config Connector (GCP) or ACK (AWS)

https://cloud.google.com/config-connector/docs/reference/res...

https://aws-controllers-k8s.github.io/community/reference/rd...

Post reply on HN