Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling PostgreSQL to power 800M ChatGPT users

openai.com

Post reply on HN
Print view View original

81–90 of 145 posts

Re: Scaling PostgreSQL to power 800M ChatGPT users

#81

Post by londons_explore »

zozbot234 wrote: post #65
EB66 wrote: post #16

Earlier quoted context omitted.

Sharding can be made mostly transparent, but it's not purely a DB-level concern in practice. Once data is split across nodes, join patterns, cross-shard transactions, global uniqueness, certain keys hit with a lot of traffic, etc matter a lot. Even if partitioning handles routing, the application's query patterns and its consistency/latency requirements can still force application-level changes.

> mostly transparent, but it's not purely a DB-level concern in practice ... But how would any of that change by going outside Postgres itself to begin with? That's the part that doesn't make much sense to me.

When sharded, anything crossing a shard boundary becomes non-transactional.

Ie. if you shard by userId, then a "share" feature which allows a user to share data with another user by having a "SharedDocuments" table cannot be consistent.

That in turn means you're probably going to have to rewrite the application to handle cases like a shared document having one or other user attached to it disappear or reappear. There are loads of bugs that can happen with weak consistency like this, and at scale every very rare bug is going to happen and need dealing with.

Re: Scaling PostgreSQL to power 800M ChatGPT users

#82

Post by zozbot234 »

AlisdairO wrote: post #39

Regarding schema changes and timeouts - while having timeouts in place is good advice, you can go further. While running the schema rollout, run a script alongside it that kills any workload conflicting with the aggressive locks the schema change is trying to take. This will greatly reduce the pain caused by lock contention, and prevent you from needing to repeatedly rerun statements on high-throughput tables. This w…

Doesn't Postgres support transactional schema changes already? Why would you want to proactively kill work that's just going to complete after the schema change is done? Load balancing, throttling etc. is a different matter that has little to do with what you're proposing.

Re: Scaling PostgreSQL to power 800M ChatGPT users

#83

Post by londons_explore »

9rx wrote: post #63
everfrustrated wrote: post #3

This is why I love Postgres. It can get you to being one of the largest websites before you need to reconsider your architecture just by throwing CPU and disk at it. At that point you can well afford to hire people who are deep experts at sharding etc.

> At that point you can well afford to hire people who are deep experts at sharding etc. Can you, though? OpenAI is haemorrhaging money like it is going out of style and, according to the news cycle over the last couple of days, will likely to be bankrupt by 2027.

And typically the bigger the company gets, the harder it is to migrate to a new data model.

You suddenly have literally thousands of internal users of a datastore, and "We want to shard by userId, nobody please don't do joins on user Id anymore" becomes an impossible ask.

Re: Scaling PostgreSQL to power 800M ChatGPT users

#84

Post by zozbot234 »

londons_explore wrote: post #81
zozbot234 wrote: post #65

Earlier quoted context omitted.

> mostly transparent, but it's not purely a DB-level concern in practice ... But how would any of that change by going outside Postgres itself to begin with? That's the part that doesn't make much sense to me.

When sharded, anything crossing a shard boundary becomes non-transactional. Ie. if you shard by userId, then a "share" feature which allows a user to share data with another user by having a "SharedDocuments" table cannot be consistent. That in turn means you're probably going to have to rewrite the application to handle cases like a shared document having one or other user attached to it disappear or reappear. There…

> When sharded, anything crossing a shard boundary becomes non-transactional.

Not necessarily? You can have two-phase commit for cross-shard writes, which ought to be rare anyway.

Re: Scaling PostgreSQL to power 800M ChatGPT users

#85

Post by londons_explore »

winterrx wrote: post #19

First OpenAI Engineering blog? I'm definitely interested in seeing more and how they handled the rapid growth.

There was a lot of downtime...

I think they handled the massive growth by a lot of 2am emergencies and editing config files directly in production in the hope of fixing fires.

Re: Scaling PostgreSQL to power 800M ChatGPT users

#86

Post by londons_explore »

CodeCompost wrote: post #46
huksley wrote: post #45

"... If a new feature requires additional tables, they must be in alternative sharded systems such as Azure CosmosDB rather than PostgreSQL...." So it is not really scaling too much now, rather maintaining current state of things and new features go to a different DB?

Azure CosmosDB is insanely expensive. I can't imagine anybody using it unless you have OpenAI money.

We don't know the profit margins on it... Might not be very expensive if you're an internal user as openAI effectively is for microsoft.

Re: Scaling PostgreSQL to power 800M ChatGPT users

#87

Post by everfrustrated »

beoberha wrote: post #33
csto12 wrote: post #26

Earlier quoted context omitted.

In 2026 is SQL Server ever the answer?

That’s kind of my point. They’re not really in competition. I bet they’d have an easier time with this scale if they were on SQL Server, but obviously that migration isn’t happening and startups don’t reach for it for many reasons.

The software licencing of 50 read replicas alone would make sqlserver a non-starter

Re: Scaling PostgreSQL to power 800M ChatGPT users

#88

Post by Etheryte »

lighthouse1212 wrote: post #77

The 'single primary with read replicas' pattern scaling to 800M users is the real insight here. Most startups reach for sharding or distributed databases way too early, adding complexity for scale they don't have. If OpenAI can serve hundreds of millions from one Postgres primary by offloading reads and pushing new write-heavy features elsewhere, that's a strong argument for simplicity.

When people spend their entire careers in AWS land, it's easy to forget just how much power a single beefy bare metal server brings to bear. You can scale far and wide simply by getting a bigger server.

Re: Scaling PostgreSQL to power 800M ChatGPT users

#89

Post by killingtime74 »

ramraj07 wrote: post #55
killingtime74 wrote: post #49

Why a single postgres? Why not shard by users?

They literally answer that in the post. They started with a single instance and realized sharding the existing tables will be too much work (they'll slowly migrate to new tables instead).

Right, should have read it

Re: Scaling PostgreSQL to power 800M ChatGPT users

#90

Post by jll29 »

PUSH_AX wrote: post #76
szastamasta wrote: post #69

Article has so much fluff and only some very coarse information like (we sharded writes, yay!). Almost no detail just keywords for SEO, or whatever they’re aiming for. There’s also a lot of repetition. Maybe it was AI generated…?

Could even be seen as a disguised ad for their infrastructure partner too.

Yeah, right - when PostgreSQL starts to struggle, Microsoft Azure CosmoDB[tm] comes to the rescue (mentioned 3x).
Post reply on HN