I didn't see "backups" mentioned in that, though I'm sure they have them. Depending on your needs, it's a big thing to keep in mind while weighing options. For a small startup or operation, a managed service having credible snapshots, PITR backups, failover, etc. is going to save a business a lot of ops cost, compared to DIY designing, implementing, testing, and drilling, to the same level of credibility. One recent…
Why we moved from AWS RDS to Postgres in Kubernetes
81–90 of 153 posts
Re: Why we moved from AWS RDS to Postgres in Kubernetes
#82Earlier quoted context omitted.
> 23 req/sec (guess peak 60 - 100 might be already stretching it) That kind of load is something a decent developer laptop with an NVME drive can serve, nothing to write home about. It is sad that the "cloud" and all these supposedly "modern" DevOps systems managed to redefine the concept of "performance" for a large chunk of the industry.
RDS tops out at about 18000 IOPS since it uses a single ebs volume. Any decent ssd will do much better. E.g. a 970 evo will easily do >100K IOPS and can do more like 400K in ideal conditions. You can get that many IOPS with aurora, but the cost is exorbitant.
Re: Why we moved from AWS RDS to Postgres in Kubernetes
#83(Nhost) Sorry for not answering everyone individually, but I see some confusion duo to the lack of context about what we do as a company. First things first, Nhost falls into the category of backend-as-a-service. We provision and operate infrastructure at scale, and we also provide and run the necessary services for features such as user authentication and file storage, for users creating applications and businesses.…
Re: Why we moved from AWS RDS to Postgres in Kubernetes
#84(Nhost) Sorry for not answering everyone individually, but I see some confusion duo to the lack of context about what we do as a company. First things first, Nhost falls into the category of backend-as-a-service. We provision and operate infrastructure at scale, and we also provide and run the necessary services for features such as user authentication and file storage, for users creating applications and businesses.…
And what are your cost savings from RDS perspective. I'd a similar problem where we'd to provision like 5 databases for 5 different teams. RDS is really expensive. And your solution is open source ? I would like to try.
We haven't open-sourced any of this work yet but we hope to do it soon. Join us on discord if you want to follow along (https://nhost.io/discord).
Re: Why we moved from AWS RDS to Postgres in Kubernetes
#85(Nhost) Sorry for not answering everyone individually, but I see some confusion duo to the lack of context about what we do as a company. First things first, Nhost falls into the category of backend-as-a-service. We provision and operate infrastructure at scale, and we also provide and run the necessary services for features such as user authentication and file storage, for users creating applications and businesses.…
And what are your cost savings from RDS perspective. I'd a similar problem where we'd to provision like 5 databases for 5 different teams. RDS is really expensive. And your solution is open source ? I would like to try.
Re: Why we moved from AWS RDS to Postgres in Kubernetes
#86Earlier quoted context omitted.
AWS RDS is 10x slower than BareMetal MySQL (both reads and writes). Slowness is mainly due to the reason that Storage is over network for RDS. Not bad to invest some extra time to get better performance. You are falling to “Appeal to antiquity” fallacy if you think something old is better.
What you describe is still a fallacy because it's assuming that just because you can get better performance with BareMetal, that somehow this is a cheaper or better option. In fact it will be either more error-prone, or more expensive, or both, because you are trying to reproduce from scratch what the whole RDS team has been doing for 10 years.
Re: Why we moved from AWS RDS to Postgres in Kubernetes
#87I've recently been spending a fair amount of time trying to improve query performance on RDS. This includes reviewing and optimizing particularly nasty queries, tuning PG configuration (min_wal_size, random_page_cost, work_mem, etc). I am using a db.t3.xlarge with general purpose SSD (gp2) for a web server that sees moderate writes and a lot of reads. I know there's no real way to know other than through testing, but…
I'd recommend hopping off of t3 asap if you're searching for performance gains - performance can be extremely variable (by design). M class will even you out. General storage IOPS is governed by your provisioned storage size. You can again get much more consistent performance by using provisioned IOPS. Feel free to email me if you want to chat through things specific to your env - email is in my about:
Re: Why we moved from AWS RDS to Postgres in Kubernetes
#88Having recently heard a lot of about PostgreSQL in Kubernetes (cloudNativePG for example) it always makes me wonder about the actual load and the complexity of the cluster in the question. > This is the reason why we were able to easily cope with 2M+ requests in less than 24h when Midnight Society launched This gives the answer, while it's probably not evenly distributed gives 23 req/sec (guess peak 60 - 100 might be…
Re: Why we moved from AWS RDS to Postgres in Kubernetes
#89These threads are always full of people who have always used an AWS/GCP/Azure service, or have never actually run the service themselves. Running HA Postgres is not easy...but at any sort of scale where this stuff matters, nothing is easy. It's not as if AWS has 100% uptime, nor is it super cheap/performant. There are tradeoffs for everyone's use-case but every thread is full of people at one end of the cloud / roll-…
Honestly, that's what I initially thought trying to run ha postgres on k8s, but zalando's postgres operator made things so much easier (maybe even easier than RDS). Very easy to rollout as many postgres clusters with whatever size you want. We've been running our production db on it for the last 6 months or so, no outage yet. Though I guess if you have to have a very custom setup, it might be more difficult.
Re: Why we moved from AWS RDS to Postgres in Kubernetes
#90These threads are always full of people who have always used an AWS/GCP/Azure service, or have never actually run the service themselves. Running HA Postgres is not easy...but at any sort of scale where this stuff matters, nothing is easy. It's not as if AWS has 100% uptime, nor is it super cheap/performant. There are tradeoffs for everyone's use-case but every thread is full of people at one end of the cloud / roll-…
I've been successfully running Postgres in Kubernetes with the Operator from Crunchy Data. It makes HA setup really easy with a tool called Patroni, which basically takes care of all the hard stuff. Running 1 primary and 2 replicas is really no harder than running single-node Postgres.