Weird, I'd imagine for kind of use they are getting it would be easy to shard the infrastructure to entirely separate instances
.
Scaling PostgreSQL to power 800M ChatGPT users
71–80 of 145 posts
Re: Scaling PostgreSQL to power 800M ChatGPT users
#72"This effort demonstrates that with the right design and optimizations, Azure PostgreSQL can be scaled to handle the largest production workloads." Sure, but choosing from the start a DB that can scale with ease would have taken far less time and effort. You can bend any software into doing anything, but is it worth it?
They could've just sharded it; their users are not interconnected, it would be easy to just have 128 shards and then assign user to one by org/user hash
Re: Scaling PostgreSQL to power 800M ChatGPT users
#73I honestly don't understand such negative response tone from the comments. Yes, it does promote Azure, but that's to be expected from a company with is part owned by Microsoft :).
The main point of the article is that it's actually not that hard to live with a single primary Postgres for your transactional workloads (emphasis on _transactional_), and if OpenAI with their 800M+ users can still survive on a single primary (with 50(!) read replicas), so could you, especially before you've reached your first 100M users.
Any non-distributed database or setup is orders of magnitude easier to design for, and it's also typically much more cost efficient too, both in terms of hardware and software too.
There are some curious details, e.g.:
- you can ship WAL to 50 read replicas simultaneously from a single primary and be fine - you can even be using an ORM and still get decent performance - schema changes are possible, and you can just cancel a slow ALTER to prevent production impact - pgbouncer is ok even for OpenAI scale
There are so many things that contradict current "conventional wisdom" based on the experience from what was possible with the hardware 10+ (or even 20+) years ago. Times finally changed and I really welcome articles like these that show how you can greatly simplify your production setup by leveraging the modern hardware.
Re: Scaling PostgreSQL to power 800M ChatGPT users
#74Did I miss it, or did they not say why they picked CosmoDB? Postgres has also sharding, so instead of moving to a different DB they could have added a new postgres instance with sharding for the new requests.
Re: Scaling PostgreSQL to power 800M ChatGPT users
#75From what I understand they basically couldn't scale writes in PostgreSQL to their needs and had to offload what they could to Azure's NoSQL database. I wonder, is there another popular OLTP database solution that does this better? > For write traffic, we’ve migrated shardable, write-heavy workloads to sharded systems such as Azure CosmosDB. > Although PostgreSQL scales well for our read-heavy workloads, we still enc…
I was thinking about the same paragraph because write-amplification is exactly the problem solved by LSM trees _and_ they already have a solution for that in-house - one of the first acquisitions that OpenAI made is Rockset - a company that actually built the RocksDb at scale.
So, this is the part that actually made me left wondering why.
Re: Scaling PostgreSQL to power 800M ChatGPT users
#76Article has so much fluff and only some very coarse information like (we sharded writes, yay!). Almost no detail just keywords for SEO, or whatever they’re aiming for. There’s also a lot of repetition. Maybe it was AI generated…?
Could even be seen as a disguised ad for their infrastructure partner too.
Re: Scaling PostgreSQL to power 800M ChatGPT users
#77The 'single primary with read replicas' pattern scaling to 800M users is the real insight here. Most startups reach for sharding or distributed databases way too early, adding complexity for scale they don't have. If OpenAI can serve hundreds of millions from one Postgres primary by offloading reads and pushing new write-heavy features elsewhere, that's a strong argument for simplicity.
Re: Scaling PostgreSQL to power 800M ChatGPT users
#78Earlier quoted context omitted.
> would slow down the primary since it has to wait for the TCP acks Other than keeping around more WAL segments not sure why it would slow down the primary?
If you use streaming replication (ie. WAL shipping over the replication connection), a single replica getting really far behind can eventually cause the primary to block writes. Some time back I commented on the behaviour: https://news.ycombinator.com/item?id=45758543 You could use asynchronous WAL shipping, where the WAL files are uploaded to an object store (S3 / Azure Blob) and the streaming connections are only u…
> If you use streaming replication (ie. WAL shipping over the replication connection), a single replica getting really far behind can eventually cause the primary to block writes. Some time back I commented on the behaviour: https://news.ycombinator.com/item?id=45758543
I'd like to know more, since I don't understand how this could happen. When you say "block", what do you mean exactly?
Re: Scaling PostgreSQL to power 800M ChatGPT users
#79TL;DR There is no secret sauce, it's the same set of techniques you’ve seen in most PostgreSQL scaling guides. Those techniques do work.
Re: Scaling PostgreSQL to power 800M ChatGPT users
#80Article has so much fluff and only some very coarse information like (we sharded writes, yay!). Almost no detail just keywords for SEO, or whatever they’re aiming for. There’s also a lot of repetition. Maybe it was AI generated…?
[dead]