Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Postgres in Kubernetes [pdf]

static.sched.com

Post reply on HN
Print view View original

81–90 of 101 posts

Re: Running Postgres in Kubernetes [pdf]

#81

Post by qeternity »

sunshinekitty wrote: post #75
coding123 wrote: post #69

Earlier quoted context omitted.

It sounds pretty simple and enticing. Any problems convincing the team of this route?

Speaking from experience.. This is fine if you have a small single node database that can have some downtime. Once you need replicas, to fit your database in ram for performance, or reliability with a hot standby for failovers it becomes a lot more complicated. You should consider what (if anything) you miss out on by just running in a single vm that you can scale out much easier down the road should you need to. Alt…

> single vm that you can scale out much easier

I’m not sure how experience could lead you to this conclusion. This wouldn’t work for any of our production needs.

Re: Running Postgres in Kubernetes [pdf]

#82

Post by dang »

redis_mlc wrote: post #77
deathanatos wrote: post #71

Earlier quoted context omitted.

> there's no maintenance work required on your part. The post of mine you are replying to outlines maintenance work we had to do on an actual RDS instance. My point is that you shouldn't weigh managed solutions as maintenance-free: they're not (and examples of why and how they are not are in the first post). They might win out, and they do have a place, but if you're evaluating them as "hassle-free", you will be disa…

I bet your life is tough, being as dumb as a box of rocks. The above answers are from somebody who doesn't know what they're talking about, either about database administration or compliance.

Ok, that's enough. Given that https://news.ycombinator.com/item?id=23670678 was just a couple days ago, we've banned this account.

Re: Running Postgres in Kubernetes [pdf]

#83

Post by imtringued »

ianhowson wrote: post #46
dijit wrote: post #44

Earlier quoted context omitted.

this is very contra to my own experience, we're IOPS bound far more than we're CPU bound. This is true in Video Games (current job) and e-commerce (what became part of Oracle Netsuite)

IOPS limitations and network latency are the reason I want to run my Postgres replicas in k8s. Every machine gets NVMe and a replica instance. They're still cattle, but they can serve a lot of requests. Database CPU usage is negligible. It was easier to put a replica on every host than try to rearchitect things to tolerate single-digit millisecond RTT to a cloud database.

One thing I dislike about SQL databases is the synchronous protocol. It really feels like I should be able to send a batch of queries all at once and then process the response in bulk instead of waiting for each individual query to respond incurring a round trip every single time. You can obviously rewrite your queries so that they fetch everything upfront in one single huge query but that is a lot of work and the end result isn't straight forward code that everyone will understand.

This is probably half the reason why GraphQL exists. Your request can execute multiple queries asynchronously and it's the default mode of operation, not something you have to explicitly work on. Extreme example. Round trip latency is 100ms. Your GraphQL server is on the same node as the database and sends four SQL queries at the same time. The entire response arrives after 100ms.

Re: Running Postgres in Kubernetes [pdf]

#84

Post by imtringued »

redis_mlc wrote: post #77
deathanatos wrote: post #71

Earlier quoted context omitted.

> there's no maintenance work required on your part. The post of mine you are replying to outlines maintenance work we had to do on an actual RDS instance. My point is that you shouldn't weigh managed solutions as maintenance-free: they're not (and examples of why and how they are not are in the first post). They might win out, and they do have a place, but if you're evaluating them as "hassle-free", you will be disa…

I bet your life is tough, being as dumb as a box of rocks. The above answers are from somebody who doesn't know what they're talking about, either about database administration or compliance.

I have no compliance requirements and I use encryption even when the database is on the same node as the application just to eliminate that excuse. There is no need to justify encryption. Just use it. Setting it up won't take more than an hour.

Re: Running Postgres in Kubernetes [pdf]

#85

Post by devit »

ci5er wrote: post #50
jonfw wrote: post #34

Earlier quoted context omitted.

The nice thing about running a DB inside a cluster is running your entire application, end to end, through one unified declarative model. It's really really easy to spin up a brand new dev or staging environment. generally though in production, you're not going to be taking down DBs on purpose. If it's not supposed to be ephemeral, it doesn't fit the model

My biases are probably 5+ years old, but it used to be that running PostGres on anything other than bare metal (for example, running it in Docker) was fraught with all sorts of data/file/io sync issues that you might not be able to recover from. So, I just got used to running the databases on metal (graph, postgres, columnar) with whatever reliability schemes myself and leaving docker and k8s outside of that. Has tha…

Assuming you run kubelet/Docker on bare metal, containers will of course run on bare metal as well.

A container is just a collection of namespaces.

Re: Running Postgres in Kubernetes [pdf]

#86

Post by jchw »

sasavilic wrote: post #23

Unless you have a really good shared storage, I don't see any advantage for running Postgres in Kubernetes. Everything is more complicated without any real benefit. You can't scale it up, you can't move pod. If pg fails to start for some reason, good luck jumping into container to inspect/debug stuff. I am neither going to upgrade PG every 2 weeks nor it is my fresh new microservice that needs to be restarted when it…

> If pg fails to start for some reason, good luck jumping into container to inspect/debug stuff.

kubectl exec, attach and cp all make this trivial. Whatever type of inspection you want should be relatively doable.

Putting stuff in kubernetes also lets you take advantage of the ecosystem, including network policies, monitoring and alerting systems, health and liveness testing, load balancing, deployment and orchestration, etc.

Re: Running Postgres in Kubernetes [pdf]

#87

Post by majewsky »

kgraves wrote: post #3

What's the use-case for running databases in k8s, is this a widely accepted best practice?

Conway's Law. The hardware team deals with the lower parts of the stack: hardware, OS, up to Kubernetes. The applications team(s) deal exclusively with Kubernetes payloads.

Re: Running Postgres in Kubernetes [pdf]

#88

Post by kristofarkas »

chvid wrote: post #72

To all the commenters in this thread. If kubernetes cannot run a database then what good is it? (And I suppose the same issues pop up for things like a persistent queue or a full text indexer.) The end goal of Kubernetes is to able to create and recreate environments and scale them up and down at will all based a declarative configuration. But if you take databases out of it; then you are not really achieving that go…

Some say that running stateful applications on K8S is not a good idea anyways, and K8S is best used for stateless applications. Sure you can connect to a stateful DB but the app itself is stateless.

Re: Running Postgres in Kubernetes [pdf]

#89

Post by how_gauche »

Postgres + stolon + k8 is the easiest time I've ever had bootstrapping a DB for high availability. I'm not sure I'd use it for extremely high throughput apps, but for smallish datasets that NEED to be online, it was amazing. The biggest reason it's amazing? The dev, staging, and prod environments look exactly the same from a coder's perspective, and bringing a fresh one up is always a single command, because that's just how you work in kube-land.

Re: Running Postgres in Kubernetes [pdf]

#90

Post by 1337shadow »

ci5er wrote: post #50
jonfw wrote: post #34

Earlier quoted context omitted.

The nice thing about running a DB inside a cluster is running your entire application, end to end, through one unified declarative model. It's really really easy to spin up a brand new dev or staging environment. generally though in production, you're not going to be taking down DBs on purpose. If it's not supposed to be ephemeral, it doesn't fit the model

My biases are probably 5+ years old, but it used to be that running PostGres on anything other than bare metal (for example, running it in Docker) was fraught with all sorts of data/file/io sync issues that you might not be able to recover from. So, I just got used to running the databases on metal (graph, postgres, columnar) with whatever reliability schemes myself and leaving docker and k8s outside of that. Has tha…

It hasn't, if your DB is sufficiently large, or under a lot of writes, the shutdown time will be larger than the docker timeout before killing pg. Data loss occurs in that case.
Post reply on HN