Live data from Hacker News

Making 768 servers look like 1

planetscale.com

51–60 of 89 posts

Re: Making 768 servers look like 1

#51

Earlier quoted context omitted.

Spreading requests out across hundreds, thousands, and in some cases even more is precisely what is done in the industry for big databases! Good examples: cashapp: https://code.cash.app/planetscale-metal github: https://github.blog/engineering/infrastructure/partitioning-... etsy: https://www.etsy.com/codeascraft/migrating-etsyas-database-s... These companies could not realistically operate off a single database serv…

> These companies could not realistically operate off a single database server. I want to see it fail first. I suggest their org chart has more to do with their architecture than database performance. Even so you’re in the category I said. Hundreds of expensive engineers maintain this stuff.

You should read through those articles.

Re: Making 768 servers look like 1

#52

Earlier quoted context omitted.

Let's hypothesize this is true. I worked at a startup that had _money_ - a lot of it - flowing in. Stakeholders wanted to see growth, which meant features. They did not want to see us making platform improvements; those didn't show well to clients. Best throw money at the problem, rather than sacrifice the all-important features. We had a number of architectural quirks that reflected this. One was that we ran our dat…

What if there was someone involved who had built web applications than run on laptops before and made different architectural decisions than the path your company went down.

If my grandma had wheels she would have been a bike. Most of these decisions are made by someone very early on in the process, either a cofounder or an underpaid junior engineer, when the company has zero revenue and no PMF.

It's unrealistic to expect a startup to build a product (or at least it was prior to LLMs generating a lot of boilerplate) with best practices in place.

Re: Making 768 servers look like 1

#53

What about sequences? The example shows an auto-incrementing user ID. How’s that possible without contention between all shards? Is the proxy responsible for sequences? What about foreign keys? Do they all have to live on the same shard? How do you do distributed transactions? On cross-shard reads: how do you do sorting? And cross-shard joins? I’d love to be proven wrong, but I suspect the 768 servers look like 1 onl…

A lot of what you're referring to is dictated by the sharding strategy. Vitess and Neki both let you configure this via the VSchema / data topology (That's what I'm getting at here)

https://planetscale.com/blog/making-768-servers-look-like-1#...

This decision matters a ton. I drilled into this specific point on sharding in one of my articles from awhile back:

https://planetscale.com/blog/database-sharding

This is something you face any time you shard data, no matter the system. Single-shard queries are preferred to cross-shard ones. We want 99% of what we do to be single shard, but occasionally cross-shard work is unavoidable.

Re: Making 768 servers look like 1

#54

What about sequences? The example shows an auto-incrementing user ID. How’s that possible without contention between all shards? Is the proxy responsible for sequences? What about foreign keys? Do they all have to live on the same shard? How do you do distributed transactions? On cross-shard reads: how do you do sorting? And cross-shard joins? I’d love to be proven wrong, but I suspect the 768 servers look like 1 onl…

Of course 768 servers NEVER behave as 1. This is physically impossible. Global services using relational dbs typically severely restrict queries that run against the cluster. So no joins, no intervals, no grouping, etc. Transactional queries are usually limited to something like "get a single record, preferably from cache". For many typical web services this can go VERY FAR. Only a handful of global services needs mo…

> Transactional queries are usually limited to something like "get a single record, preferably from cache".

simply wrong

Re: Making 768 servers look like 1

#55

Earlier quoted context omitted.

Spreading requests out across hundreds, thousands, and in some cases even more is precisely what is done in the industry for big databases! Good examples: cashapp: https://code.cash.app/planetscale-metal github: https://github.blog/engineering/infrastructure/partitioning-... etsy: https://www.etsy.com/codeascraft/migrating-etsyas-database-s... These companies could not realistically operate off a single database serv…

> These companies could not realistically operate off a single database server. I want to see it fail first. I suggest their org chart has more to do with their architecture than database performance. Even so you’re in the category I said. Hundreds of expensive engineers maintain this stuff.

> I want to see it fail first.

your lack of experience at scale doesn't really invalidate this architecture

Re: Making 768 servers look like 1

#57

I disagree with the opening premise: > A single database server cannot handle such demand, so we must spread the queries and data out across many servers with database sharding Did you max out the capacity of the best server you can buy? Such a database can serve millions of customers (the numbers given). You always want to scale up the other parts first, request handlers, caching, etc. The day you can no longer insp…

Well, they are selling this thing so they don't want you to buy a big server (with a read replica) as that's much cheaper.

We have tons of customers who do exactly this. It's great. Sharding is for customers who out grow this path.

Re: Making 768 servers look like 1

#58
post #3

Looks like the GIF is fully built out in code. It's really nice to look at, well made, and easy to understand too. I wonder what program or code they used. I'd love to know. p.sI thought it was a GIF, but it's an iframe. That was a nice little surprise.

Author here, thank you.

Technically, they are using js + gsap + svg embedded i the article with iframes.

Process-wise, I drafted most of them as static images in excalidraw, passed the images along to cursor for a first draft, applied styling rules, and then did a bunch of fine-tuning.

Re: Making 768 servers look like 1

#59
I should start by disclosing that I work part time in the Oracle Database group, but - of course - my HN account is entirely my own, despite occasional mild shilling. The article itself is shilling for PlanetScale so that seems OK.

The author - certainly not deliberately! - says some untrue things about relational databases. The most important one is this:

> To understand why sharding is a necessary part of scaling relational databases...

But sharding isn't a necessary part of scaling relational databases. It's really just a requirement of simple databases like Postgres and MySQL.

If you want a relational database cluster that really does make 768 servers look like one, then you want what Oracle calls RAC ("real application clusters"). Your cloud will probably rent you access to one under a name like Autonomous Database. Self hosted it may be called ExaData, which is a unified hardware/software "rent a rack" style offering. You may be surprised to discover that it's not much more expensive than many managed Postgres offerings.

RAC can scale a non-sharded relational database horizontally. That means all queries can access all data, all SQL features like sequences and joins work, any server can take part in transactions with any other and in general it looks exactly like a really big single machine would. In other words, it's a synchronous multi-write-master system.

RAC scales very well. 768 servers is well within reach as long as your query patterns scale too i.e. don't all contend on writing to one row. Behind the scenes it uses a dedicated high speed RDMA network with lock coordination to transfer data blocks directly between nodes, never hitting disk for memory that's already in the buffer cache.

Additionally RAC is fully HA and supports rolling upgrades of the cluster whilst live. You don't need any NLBs or routers either. The client drivers automatically discover and load balance between nodes without needing intermediaries, transactions can start on one node and fail over to another without applications noticing, and so on. There's plenty of opportunity for caching and replication. You can run asynchronously replicated failover clusters, run multiple clusters in a Raft-driven globally coordinated super-cluster and can deploy coherent read-through caches anywhere; the main clusters will inform them the moment data in them becomes stale.

In other words, it can do a lot.

If for some reason you do need sharding then that's also supported with features like automatic sharding key distribution to client drivers that transparently route queries correctly, but most apps don't need this.

In case you're wondering why I say all this, firstly, obviously, I have a financial conflict of interest. But the database hasn't driven Oracle's stock price for a long time, so it's not a big one. These days it's all about cloud and AI.

No, the main reason is that HN fills up every month with blog posts where engineers talk about the incredible pain involved with scaling and running Postgres. And almost always, it's clear that they don't realize there's any alternative to that pain. It's not that they considered the options and then explain why they picked this one, it's that they think - as claimed in this article - that it's almost some fundamental limitation of reality itself, imposed by the laws of computer science.

There are lots of startups that lose time and money due to database problems they simply don't need to have. And that sucks. If they'd prefer to spend that time, pain and money to avoid using a DB from Oracle, fine, so be it. I won't argue with random devs about lawnmower memes. But if it's because they don't realize what's possible.... well, maybe someone will be helped by being aware of this. Database scaling problems are a choice, not an inevitability.

Re: Making 768 servers look like 1

#60

I should start by disclosing that I work part time in the Oracle Database group, but - of course - my HN account is entirely my own, despite occasional mild shilling. The article itself is shilling for PlanetScale so that seems OK. The author - certainly not deliberately! - says some untrue things about relational databases. The most important one is this: > To understand why sharding is a necessary part of scaling r…

Lol about buying Oracle for performance!

But anyway, do you have benchmarks from anybody not related to Oracle showing that performance? Because Oracle forbids talking about it...

Post reply on HN