Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Postgres in Kubernetes [pdf]

static.sched.com

Post reply on HN
Print view View original

31–40 of 101 posts

Re: Running Postgres in Kubernetes [pdf]

#31

Post by renewiltord »

I much prefer just using RDS Aurora. Far fewer headaches. If I don't need low latency I'd use RDS Aurora no matter which cloud I'm hosted on. Otherwise I'll use hosted SQL.

The reason I mention this is that Kubernetes requires a lot of management to run so the best solution is to use GKE or things like that. If you're using managed k8s, there's little reason to not use managed SQL.

The advantages of k8s are not that valuable for a SQL server cluster. You don't even really get colocation of data because you're realistically going to use a GCE Persistent Disk or EBS volume and those are network attached anyway.

Re: Running Postgres in Kubernetes [pdf]

#33

Post by georgebarnett »

random3 wrote: post #26
sasavilic wrote: post #23

Unless you have a really good shared storage, I don't see any advantage for running Postgres in Kubernetes. Everything is more complicated without any real benefit. You can't scale it up, you can't move pod. If pg fails to start for some reason, good luck jumping into container to inspect/debug stuff. I am neither going to upgrade PG every 2 weeks nor it is my fresh new microservice that needs to be restarted when it…

GCP volumes are over network already. You can deploy stateful workloads using StatefulSets. We've run an HBase workloads for development purposes (about 20-30x cheaper than BigTable) and it worked great (no issues for over 12 months). While Postgres is hardly a distributed database, there may be some advantages to ensure availability and perhaps even more in replicated setup.

Why are you comparing HBase with Postgres? They are very different technologies with completely different architectural constraints?

Re: Running Postgres in Kubernetes [pdf]

#34

Post by jonfw »

sasavilic wrote: post #23

Unless you have a really good shared storage, I don't see any advantage for running Postgres in Kubernetes. Everything is more complicated without any real benefit. You can't scale it up, you can't move pod. If pg fails to start for some reason, good luck jumping into container to inspect/debug stuff. I am neither going to upgrade PG every 2 weeks nor it is my fresh new microservice that needs to be restarted when it…

The nice thing about running a DB inside a cluster is running your entire application, end to end, through one unified declarative model. It's really really easy to spin up a brand new dev or staging environment.

generally though in production, you're not going to be taking down DBs on purpose. If it's not supposed to be ephemeral, it doesn't fit the model

Re: Running Postgres in Kubernetes [pdf]

#35

Post by GordonS »

random3 wrote: post #27
GordonS wrote: post #25

So, a bit OT, but I'm looking for some advice on building a Postgres cluster, and I'm pretty sure k8s is going to add a lot of complexity with no benefit. I'm a Postgres fan, and use it a lot, but I've never actually used it in a clustered setup. What I'm looking at clustering for is not really for scalability (still at the stage where we can scale vertically), but for high availability and backup - if one node is do…

The main advantage with Kubernetes (especially in low ops environments like GKE) is not scalability, but availability and ease of development (spinning things up and down is super-easy). The learning curve to stand something up is not very high and pays of over time compared to SSH-ing into VMs.

I'm very comfortable with containers (less so specifically with k8s), but generally for stateless or stateless'ish services. What are the advantages of k8s specifically for a database?

Re: Running Postgres in Kubernetes [pdf]

#36

Post by qeternity »

sasavilic wrote: post #23

Unless you have a really good shared storage, I don't see any advantage for running Postgres in Kubernetes. Everything is more complicated without any real benefit. You can't scale it up, you can't move pod. If pg fails to start for some reason, good luck jumping into container to inspect/debug stuff. I am neither going to upgrade PG every 2 weeks nor it is my fresh new microservice that needs to be restarted when it…

You don't run shared/network storage. You run PVCs on local storage and you run an HA setup. You ship log files every 10s/30s/1m to object storage. You test your backups regularly which k8s is great for.

All of this means that I don't worry about most things you mention. PG upgrade? Failover and upgrade the pod. Upgrade fails? wal-g clone from object storage and rejoin cluster. Scaling up? Adjust the resource claims. If resource claims necessitate node migration, see failover scenario. It's so resilient. And this is all with raid 10 nvme direct attached storage just as fast as any other setup.

You mention etcd but people don't run etcd the way you're describing postgres. You run a redundant cluster that can achieve quorum and tolerate losses. If you follow that paradigm, you end up with postgres on k8s.

Re: Running Postgres in Kubernetes [pdf]

#37

Post by dilyevsky »

sasavilic wrote: post #23

Unless you have a really good shared storage, I don't see any advantage for running Postgres in Kubernetes. Everything is more complicated without any real benefit. You can't scale it up, you can't move pod. If pg fails to start for some reason, good luck jumping into container to inspect/debug stuff. I am neither going to upgrade PG every 2 weeks nor it is my fresh new microservice that needs to be restarted when it…

Ime most pg deployments don’t need insanely high iops and become cpu bound much quicker. So running ebs or gcp pd ssd or even ceph pd is usually enough.

Re: Running Postgres in Kubernetes [pdf]

#38

Post by penagwin »

GordonS wrote: post #25

So, a bit OT, but I'm looking for some advice on building a Postgres cluster, and I'm pretty sure k8s is going to add a lot of complexity with no benefit. I'm a Postgres fan, and use it a lot, but I've never actually used it in a clustered setup. What I'm looking at clustering for is not really for scalability (still at the stage where we can scale vertically), but for high availability and backup - if one node is do…

Compared to many databases, postgres HA is a mess. It has builtin streaming, but no fail over of any kind, all of that has to be managed by another application.

We've had the best luck with patron, but even then you'll find the documentation confusing, have weird issues, etc. You'll need to setup etcd/Consul to use it. That's right you need a second database cluster to setup your database cluster.... Great...

I have no clue how such a community favorite database has no clear solution to basic HA.

Re: Running Postgres in Kubernetes [pdf]

#39

Post by qeternity »

jashmatthews wrote: post #14
mamon wrote: post #4

Earlier quoted context omitted.

It actually is not that complex. I'm using Crunchy Postgres Operator at my current employer. You get an Ansible playbook to install an Operator inside Kubernetes, and after that you get a commandline administration tool that let's you create a cluster with a simple pgo create cluster command. Most administrative tasks like creating or restoring backups (which can be automatically pushed to S3) are just one or two pgo…

Crunchy PGO is super cool but I'm not sure how we got to the idea that it's not that complex compared to a managed service like RDS.

I don't think anyone would argue that RDS isn't vastly simpler. If it weren't, there'd be no reason to pay such a premium for it.

Re: Running Postgres in Kubernetes [pdf]

#40

Post by deathanatos »

caiobegotti wrote: post #30

Please don't. It's not because it's possible that it is a good idea. The PDF itself clearly shows how it can get complex quickly. The great majority of people won't ever be able to do this properly, securely and with decent reliability. Of course I may have to swallow my words in the future in case a job requires it but unless you REALLY REALLY REALLY need PostgreSQL inside Kubernetes IMHO you should just stick with…

I've done MySQL RDS, and I've seen k8s database setups. (But not w/ PG.)

RDS is okay, but I would not dismiss the maintenance work required; RDS puts you at the mercy of AWS when things go wrong. We had a fair bit of trouble with failovers taking 10x+ longer than they should. We also set up encryption, and that was also a PITA: we'd consistently get nodes with incorrect subjectAltNames. (Also, at the time, the certs were either for a short key or signed by a short key, I forget which. It was not acceptable at that time, either; this was only 1-2 years ago, and I'm guessing hasn't been fixed.) Getting AWS to actually investigate, instead of "have you tried upgrading" (and there's always an upgrade, it felt like). RDS MySQL's (maybe Aurora? I don't recall fully) first implementation of spatial indexes was flat-out broken, and that was another lengthy support ticket. The point is that bugs will happen, and cloud platform support channels are terrible at getting an engineer in contact with an engineer who can actually do something about the problem.

Post reply on HN