Yeah that's an ad for Azure Cosmos DB
Scaling PostgreSQL to power 800M ChatGPT users
111–120 of 145 posts
Re: Scaling PostgreSQL to power 800M ChatGPT users
#112Earlier quoted context omitted.
> would slow down the primary since it has to wait for the TCP acks Other than keeping around more WAL segments not sure why it would slow down the primary?
If you use streaming replication (ie. WAL shipping over the replication connection), a single replica getting really far behind can eventually cause the primary to block writes. Some time back I commented on the behaviour: https://news.ycombinator.com/item?id=45758543 You could use asynchronous WAL shipping, where the WAL files are uploaded to an object store (S3 / Azure Blob) and the streaming connections are only u…
There is no backpressure from replication and streaming replication is asynchronous by default. Replicas can ask the primary to hold back garbage collection (off by default), which will eventually cause a slow down, but not blocking. Lagging replicas can also ask the primary to hold onto WAL needed to catch up (again, off by default), which will eventually cause disk to fill up, which I guess is blocking if you squint hard enough. Both will take considerable amount of time and are easily averted by monitoring and kicking out unhealthy replicas.
Re: Scaling PostgreSQL to power 800M ChatGPT users
#113Article has so much fluff and only some very coarse information like (we sharded writes, yay!). Almost no detail just keywords for SEO, or whatever they’re aiming for. There’s also a lot of repetition. Maybe it was AI generated…?
Lol, yes, it was very vague. Very generic paragraphs on Caching, Connection pooling, Query Optimization wrt Joins, etc.
Re: Scaling PostgreSQL to power 800M ChatGPT users
#114Earlier quoted context omitted.
Are you saying this because OpenAI didnt choose SQL Server?
In 2026 is SQL Server ever the answer?
My sentiments exactly. Anyone at the low side of scale thinking about MS SQL, should seriously do a current survey of things in the dbms space.. there is absolutely no NEED to pay for dbms in 2026. Those old dinosours only still exist, because of the data hijacking nature of past db designs and coding. Everybody and their grandmother were obfuscating code and designs in order to bake in customer loyalty and repetitive patronage. Those old projects are keeping the lights on at proprietary DB Inc. AT the high end of things, you're gonna need db engineers, and if you get yourself Microsoftie hammersharks disguised as professional engineers, they gonna see everything as a nail.
Re: Scaling PostgreSQL to power 800M ChatGPT users
#115Earlier quoted context omitted.
Why is it a good database? Integration with Entra? I've heard arguments in favor of Oracle DB, but I've never heard anything good about MSSQL besides integration with the MS ecosystem.
The SQL Server query planner is head and shoulders above what Postgres offers in the types of optimizations it will apply to your queries. It also properly caches query plans. It offers heap tables, as well as index organized tables depending on what you need. The protocol supports running multiple queries and getting multiple resultsets back at once saving some round-trips and resources. Also supports things like gl…
TBF, there's a price to be paid for speed on threads... no isolation, lower tolerance to failures, complex synchronization, painful debugging.
Re: Scaling PostgreSQL to power 800M ChatGPT users
#116Someone ask Microsoft what does it feel to be bested by an open source project on their very own cloud platform!!! Lol.
Azure offers Postgres “DBaaS”, so I’m pretty sure they are no where near that stage. It’s more likely that we should watch out for the Microsoft E-E-E strategy.
Amen to that.. those tripple-E bastards are likely to use that playbook again. Best advise is to seek fertile grounds where freedom grows. I can't wait for Europe's cloud offering, I believe they're gonna serve as the middle ground between greedy tech-bros and china's fake free as in free beer products. Pack up your bags IT HOBBITS, we're moving to middle earth.
Re: Scaling PostgreSQL to power 800M ChatGPT users
#117The 'single primary with read replicas' pattern scaling to 800M users is the real insight here. Most startups reach for sharding or distributed databases way too early, adding complexity for scale they don't have. If OpenAI can serve hundreds of millions from one Postgres primary by offloading reads and pushing new write-heavy features elsewhere, that's a strong argument for simplicity.
When people spend their entire careers in AWS land, it's easy to forget just how much power a single beefy bare metal server brings to bear. You can scale far and wide simply by getting a bigger server.
True. But “big beef” is complicated and difficult to make reliable. Horizontal scaling of unreliable servers is dirt simple to stay up through almost anything except sudden load spikes. And then it’s largely a matter of configuring your auto scaling and retries.
That said big beef is so simple to start with. And this story is a strong example that YAGNI is a practical reality for almost everybody wrt “distributed everything”.
Re: Scaling PostgreSQL to power 800M ChatGPT users
#118Regarding schema changes and timeouts - while having timeouts in place is good advice, you can go further. While running the schema rollout, run a script alongside it that kills any workload conflicting with the aggressive locks the schema change is trying to take. This will greatly reduce the pain caused by lock contention, and prevent you from needing to repeatedly rerun statements on high-throughput tables. This w…
Doesn't Postgres support transactional schema changes already? Why would you want to proactively kill work that's just going to complete after the schema change is done? Load balancing, throttling etc. is a different matter that has little to do with what you're proposing.
It supports transactional schema changes, but that's not what I'm talking about. Most schema changes require heavyweight locks on the tables they're altering. The locks might be short lived (for example, just a catalogue update to add a column), but they are nevertheless heavyweight and both block and are blocked by other work.
SELECT or DML operations take a lightweight lock on the table that doesn't block most other work, but it does block these schema changes. While the schema change is waiting to acquire the table lock, all new operations (like new SELECTs, for example) get blocked until the schema change completes.
So the following scenario can be pretty disastrous:
* Start a long-running SELECT operation on table
* Attempt to apply schema change to the table
* All new work on the table is blocked until the SELECT completes and the schema change can apply.
* Production outage
What the ChatGPT folks do is set a lock timeout when applying the schema change to make it 'give up' after a few seconds. This works to avoid truly excessive impact (in their case, they may have micro-outages of up to 5s while trying to apply schema), but has problems - firstly, they then need to retry, which may lead to more micro-outages, and secondly there's no guarantee on a system with mixed workload that they will be able to force the change through, and the schema change just ends up getting starved out.
A better alternative for most workloads is to build a system that detects what workload is blocking your schema change and kills it, allowing the schema change to go through quickly and unblock all the work behind it. You'd still use a lock timeout with this to be on the safe side, but it shouldn't be necessary in most cases.
Side note on transactional DDL - for Postgres systems with high throughput, most people just use autocommit. Table level locks that get taken to perform the schema change get held for the duration of the transaction, and you generally want to really minimize the amount of time you hold them for.
Re: Scaling PostgreSQL to power 800M ChatGPT users
#119> It’s also common to find long-running idle queries in PostgreSQL. Configuring timeouts like idle_in_transaction_session_timeout is essential to prevent them from blocking autovacuum. Idle transactions have been a huge footgun at $DAYJOB… our code base is full of “connect, start a transaction, do work, if successful, commit.” It means you’re consuming a connection slot for all work, even while you’re not using the d…
How did you implement this runtime check? Is it a lint rule, or using the type system?
Re: Scaling PostgreSQL to power 800M ChatGPT users
#120The 'single primary with read replicas' pattern scaling to 800M users is the real insight here. Most startups reach for sharding or distributed databases way too early, adding complexity for scale they don't have. If OpenAI can serve hundreds of millions from one Postgres primary by offloading reads and pushing new write-heavy features elsewhere, that's a strong argument for simplicity.
If you need so many tricks to support the infra, it will eventually come back to bite you. I am pretty sure that Google in year 2000 could have supported their workloads with existing technologies (Yahoo could, and it was a much larger company). But they did GFS and Bigtable, and the rest is history. Other companies struggled to catch up due to inferior infrastructure. A visionary company needs to be prepared and should not be hindered by infrastructure. Can you scale the single primary system another 10x or more? Because their CEO said that they will scale their revenue by that much within just a couple of years.