Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling PostgreSQL to power 800M ChatGPT users

openai.com

Post reply on HN
Print view View original

41–50 of 145 posts

Re: Scaling PostgreSQL to power 800M ChatGPT users

#41

Post by manquer »

dmix wrote: post #13
zozbot234 wrote: post #10

Earlier quoted context omitted.

PostgreSQL actually supports sharding out of the box, it's just a matter of setting up the right table partitioning and using Foreign Data Wrapper (FDW) to forward queries to remote databases. I'm not sure what the post is referencing when they say that sharding requires leaving Postgres altogether.

This is specifically what they said about sharding > The primary rationale is that sharding existing application workloads would be highly complex and time-consuming, requiring changes to hundreds of application endpoints and potentially taking months or even years

> potentially taking months or even years

On one hand OAI sell coding agents and constantly hype how easy it will replace developers and most of the code written is by agents, on the other hand they claim it will take years to refactor

Both cannot be true at the same time.

Re: Scaling PostgreSQL to power 800M ChatGPT users

#42

Post by Ozzie_osman »

simonw wrote: post #20
dmix wrote: post #13

Earlier quoted context omitted.

This is specifically what they said about sharding > The primary rationale is that sharding existing application workloads would be highly complex and time-consuming, requiring changes to hundreds of application endpoints and potentially taking months or even years

Genuinely sounds like the kind of challenge that could be solved with a swarm of Codex coding agents. I'm surprised they aren't treating this as an ideal use-case to show off their stack!

Getting the sharing in-place, yes, but maintaining it operationally would still be a headache. Things like schema migrations across shards, resharding, and even observability.

Re: Scaling PostgreSQL to power 800M ChatGPT users

#43

Post by antonkochubey »

jiggawatts wrote: post #36
ggregoire wrote: post #35

> scaled up by increasing the instance size I always wondered what kind of instance companies at that level of scalability are using. Anyone here have some ideas? How much cpu/ram? Do they use the same instance types available to everyone, or does AWS and co offer custom hardware for these big customers?

The major hyperscalers all offer a plethora of virtual machines SKUs that are essentially one entire two-socket box with many-core CPUs. For example, Azure Standard_E192ibds_v6 is 96 cores with 1.8 TB of memory and 10 TB of local SSD storage with 3 million IOPS. Past those "general purpose" VMs you get the enormous machines with 8, 16, or even 32 sockets.[1] These are almost exclusively used for SAP HANA in-memory da…

> Something like this will set you back $30K-$60K per year

lol, no, cloud is nowhere near that good value. It’s $3.5M annually.

> The Standard_HX176rs HPC VM size gives you 176 cores and 1.4 TB of memory

This one is $124k per year.

Re: Scaling PostgreSQL to power 800M ChatGPT users

#44

Post by cuu508 »

jiggawatts wrote: post #36
ggregoire wrote: post #35

> scaled up by increasing the instance size I always wondered what kind of instance companies at that level of scalability are using. Anyone here have some ideas? How much cpu/ram? Do they use the same instance types available to everyone, or does AWS and co offer custom hardware for these big customers?

The major hyperscalers all offer a plethora of virtual machines SKUs that are essentially one entire two-socket box with many-core CPUs. For example, Azure Standard_E192ibds_v6 is 96 cores with 1.8 TB of memory and 10 TB of local SSD storage with 3 million IOPS. Past those "general purpose" VMs you get the enormous machines with 8, 16, or even 32 sockets.[1] These are almost exclusively used for SAP HANA in-memory da…

Are there any pictures around of these 8, 16, 32 socket boards? Just curious how they look like.

Re: Scaling PostgreSQL to power 800M ChatGPT users

#45

Post by huksley »

"... If a new feature requires additional tables, they must be in alternative sharded systems such as Azure CosmosDB rather than PostgreSQL...."

So it is not really scaling too much now, rather maintaining current state of things and new features go to a different DB?

Re: Scaling PostgreSQL to power 800M ChatGPT users

#46

Post by CodeCompost »

huksley wrote: post #45

"... If a new feature requires additional tables, they must be in alternative sharded systems such as Azure CosmosDB rather than PostgreSQL...." So it is not really scaling too much now, rather maintaining current state of things and new features go to a different DB?

Azure CosmosDB is insanely expensive. I can't imagine anybody using it unless you have OpenAI money.

Re: Scaling PostgreSQL to power 800M ChatGPT users

#48

Post by tormeh »

Tostino wrote: post #29
csto12 wrote: post #26

Earlier quoted context omitted.

In 2026 is SQL Server ever the answer?

It really is a good database. Give it lots of room. If you can distribute your workload on multiple machines though, you can't beat Postgres' licencing terms vs SQL Server.

Why is it a good database? Integration with Entra? I've heard arguments in favor of Oracle DB, but I've never heard anything good about MSSQL besides integration with the MS ecosystem.

Re: Scaling PostgreSQL to power 800M ChatGPT users

#50

Post by awesome_dude »

EB66 wrote: post #16
zozbot234 wrote: post #14

Earlier quoted context omitted.

I know they said that, but in fact sharding is entirely a database-level concern. The application need not be aware of it at all.

Sharding can be made mostly transparent, but it's not purely a DB-level concern in practice. Once data is split across nodes, join patterns, cross-shard transactions, global uniqueness, certain keys hit with a lot of traffic, etc matter a lot. Even if partitioning handles routing, the application's query patterns and its consistency/latency requirements can still force application-level changes.

> Once data is split across nodes, join patterns, cross-shard transactions, global uniqueness, certain keys hit with a lot of traffic

If you're having trouble there then a proxy "layer" between your application and the sharded database makes sense, meaning your application still keeps its naieve understanding of the data (as it should) and the proxy/database access layer handles that messiness... shirley

Post reply on HN