Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we moved from AWS RDS to Postgres in Kubernetes

nhost.io

Post reply on HN
Print view View original

111–120 of 153 posts

Re: Why we moved from AWS RDS to Postgres in Kubernetes

#111

Post by SomaticPirate »

fmajid wrote: post #96
nunopato wrote: post #76

(Nhost) Sorry for not answering everyone individually, but I see some confusion duo to the lack of context about what we do as a company. First things first, Nhost falls into the category of backend-as-a-service. We provision and operate infrastructure at scale, and we also provide and run the necessary services for features such as user authentication and file storage, for users creating applications and businesses.…

Are you using a Kubernetes PostgreSQL operator like pgo or CloudNativePG? https://proopensource.it/blog/postgresql-on-k8s-experiences

Also would like to know this. This post is a bit light on content. It sounds like they just moved to K8s from RDS. In my experience, Postgres works decently but there are sharp edges running it containerized (OOMS in subprocesses might not be caught by the container runtime, shared memory is pitifully low in docker at 64 MB by default)

Re: Why we moved from AWS RDS to Postgres in Kubernetes

#112

Post by MuffinFlavored »

radimm wrote: post #4

Having recently heard a lot of about PostgreSQL in Kubernetes (cloudNativePG for example) it always makes me wonder about the actual load and the complexity of the cluster in the question. > This is the reason why we were able to easily cope with 2M+ requests in less than 24h when Midnight Society launched This gives the answer, while it's probably not evenly distributed gives 23 req/sec (guess peak 60 - 100 might be…

> Having recently heard a lot of about PostgreSQL in Kubernetes

I could never get a straight answer on whether running a database in a container (and mounting the storage volume through a bind mount/network drive or whatever) came with a performance hit compared to running it as a systemd service for example.

Re: Why we moved from AWS RDS to Postgres in Kubernetes

#113

Post by aledalgrande »

whoisthemachine wrote: post #78
kazen44 wrote: post #61

Earlier quoted context omitted.

> Few developers think about the cost of using the network. Developers do not seem to realise how slow the network is compared to everything else. Sure, 100gbit network itnerfaces do exist, but most servers are attached with 10gbit interfaces, and most of the actual implementations will not actually manage to hit something like 10gbit/s because of latency and window scaling. You cannot escape latency (without inventi…

> You cannot escape latency (without inventing another universe in which physics do not apply). And latency is detrimental to performance. This. So few people distinguish between bandwidth and latency. One can be increased arbitrarily and fairly easily with new encoding techniques (which generally only improves edge cases), and the other has a floor that is hard-coded into our universe. I've gotten into debates with…

> 10GB connection from the EU to Texas should be as fast as a connection from Texas to the Midwest

and that is even before you take into consideration network topology

Re: Why we moved from AWS RDS to Postgres in Kubernetes

#114

Post by ransom1538 »

I operate a large fleet of mysql db instances. We cannot use Cloudsql (RDS competitor) due mainly to cost. BUT, one thing left out, was the ability to have complex topologies. EG. MasterA <- SlaveA[1..n] <- MasterB <- SlaveB[1..n]. With extremely high writes, being able to cut and shard where you want if very powerful. In this example you could write to MasterB with different data. If i need to filter a table in replication: done. We don't need to beg AWS RDS team for the option to change a db variable (I have done this). Warning: Doing this stuff at scale with massive bills is very stressful. It took about a year to get everything ironed out [snapshots, autoscaling, sharding, custom monitoring, etc].

Re: Why we moved from AWS RDS to Postgres in Kubernetes

#116

Post by speedgoose »

MuffinFlavored wrote: post #112
radimm wrote: post #4

Having recently heard a lot of about PostgreSQL in Kubernetes (cloudNativePG for example) it always makes me wonder about the actual load and the complexity of the cluster in the question. > This is the reason why we were able to easily cope with 2M+ requests in less than 24h when Midnight Society launched This gives the answer, while it's probably not evenly distributed gives 23 req/sec (guess peak 60 - 100 might be…

> Having recently heard a lot of about PostgreSQL in Kubernetes I could never get a straight answer on whether running a database in a container (and mounting the storage volume through a bind mount/network drive or whatever) came with a performance hit compared to running it as a systemd service for example.

It does but it’s minimal. Especially compared to the high latency and low throughout network volumes provides (which are the defaults on cloud VMs).

Re: Why we moved from AWS RDS to Postgres in Kubernetes

#117

Post by elitan »

e-clinton wrote: post #100

Congrats on the launch. Curious to see what else is in store for this week. Do I have to manually upgrade my old instances?

Thank you. It's going to be a fun week!

We're working on a one-click migration from RDS to a dedicated Postgres instance for older projects. Should be live in the next week or so.

Re: Why we moved from AWS RDS to Postgres in Kubernetes

#118

Post by xyzzy_plugh »

If the cost of operating a postgres database is eating into your margins so much (and you can't simply adjust your prices to eat the difference) then I would suspect the wrong technology is in place.

Sure, RDS is expensive, but it's also quite well done. Almost every cloud platform service is more expensive than doing it yourself. No surprise here.

In the past I've deployed SQLite over Postgres for cost cutting reasons. It's not too difficult to swap out unless you're heavily bought into database features.

Re: Why we moved from AWS RDS to Postgres in Kubernetes

#119

Post by MikePlacid »

remram wrote: post #108
brand wrote: post #30

Earlier quoted context omitted.

I’ve personally deployed O(TBs) and O(10^4 TPS) Postgres clusters on Kubernetes with a CNPG-style operator based deployment. There are some subtleties to it but it’s not exceeding complicated, and a good project like CNPG goes a long way to shaving off those sharp edges. As other commenters have suggested it’s good to really understand Kubernetes if you want to do it, though.

> O(TBs) and O(10^4 TPS) What does this syntax mean? Surely you wouldn't use big-o notation with a constant in it, especially to convey the same meaning as the thing without the O?

Mathematically speaking the statement you are objecting to is correct: c1 is O(c2) for any constants c1, c2.

English-language-ly speaking the statement you are objecting to is also correct: both you and I managed to get it’s correct meaning.

No?

Re: Why we moved from AWS RDS to Postgres in Kubernetes

#120

Post by 10000truths »

jiggawatts wrote: post #99
kazen44 wrote: post #61

Earlier quoted context omitted.

> Few developers think about the cost of using the network. Developers do not seem to realise how slow the network is compared to everything else. Sure, 100gbit network itnerfaces do exist, but most servers are attached with 10gbit interfaces, and most of the actual implementations will not actually manage to hit something like 10gbit/s because of latency and window scaling. You cannot escape latency (without inventi…

It wouldn't matter if the links were 10,000 terabits! Because of the way TCP works, it has a bounded speed for small chatty transactions that is determined primarily by the latency , not the throughput. If you look at a network throughput graph from a packet capture, it looks like a sawtooth pattern. This is called slow start , and its a key feature of TCP and all similar protocols. So if a server A wants to talk to…

The slow start behavior you describe is not inherent to TCP proper, but rather, a detail of the congestion control algorithm in use by the endpoints' TCP stacks. Most such algorithms will have some kind of AIMD feedback loop to achieve some balance of fairness and efficiency. But for applications where you have control over the endpoints and the network in between them, you can minimize slow start by setting a high initcwnd/initrwnd and using a less aggressive window shrinking mechanism.
Post reply on HN