Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making 768 servers look like 1

planetscale.com

Post reply on HN
Print view View original

21–30 of 89 posts

Re: Making 768 servers look like 1

#21

Post by anichhangani »

groundzeros2015 wrote: post #13

I disagree with the opening premise: > A single database server cannot handle such demand, so we must spread the queries and data out across many servers with database sharding Did you max out the capacity of the best server you can buy? Such a database can serve millions of customers (the numbers given). You always want to scale up the other parts first, request handlers, caching, etc. The day you can no longer insp…

Surely the cost of running sql server on premium hardware with replication would be more than running on commodity hardware with sharding?

Re: Making 768 servers look like 1

#22

Post by citrin_ru »

drdexebtjl wrote: post #5

What about sequences? The example shows an auto-incrementing user ID. How’s that possible without contention between all shards? Is the proxy responsible for sequences? What about foreign keys? Do they all have to live on the same shard? How do you do distributed transactions? On cross-shard reads: how do you do sorting? And cross-shard joins? I’d love to be proven wrong, but I suspect the 768 servers look like 1 onl…

> The example shows an auto-incrementing user ID. How’s that possible without contention between all shards?

Why not divide ID range (63bit?) by the maximum planned number of shards and then set on each shard it's min/max value so ranges will not overlap?

Re: Making 768 servers look like 1

#23

Post by groundzeros2015 »

anichhangani wrote: post #21
groundzeros2015 wrote: post #13

I disagree with the opening premise: > A single database server cannot handle such demand, so we must spread the queries and data out across many servers with database sharding Did you max out the capacity of the best server you can buy? Such a database can serve millions of customers (the numbers given). You always want to scale up the other parts first, request handlers, caching, etc. The day you can no longer insp…

Surely the cost of running sql server on premium hardware with replication would be more than running on commodity hardware with sharding?

Only if your engineering resources and free and unbounded.

Even then I would probably use those resources to optimize software instead.

In the physical world of trucks and cranes no company would make that mistake to try to save 30-80k.

Re: Making 768 servers look like 1

#24

Post by vkazanov »

ahk-dev wrote: post #12
vkazanov wrote: post #11

Earlier quoted context omitted.

Of course 768 servers NEVER behave as 1. This is physically impossible. Global services using relational dbs typically severely restrict queries that run against the cluster. So no joins, no intervals, no grouping, etc. Transactional queries are usually limited to something like "get a single record, preferably from cache". For many typical web services this can go VERY FAR. Only a handful of global services needs mo…

This seems like the important distinction: making the infrastructure look like one database to the application is different from making it behave like one unrestricted relational database. At what point does hiding the sharding become counterproductive? I imagine teams still need a fairly deep understanding of shard keys, query routing, and failure modes to avoid accidentally expensive cross-shard operations.

Yes!

Distributed systems introduce severe restrictions on what can be reasonably done at scale. Having a single connection string is one thing, being able to do a massive JOIN is another (and should only be ever done in analytical databases).

The question is not "when sharding becomes counter-productive" but "when it starts making sense".

With sharding something somewhere has to know how to route queries to subsets of data. So it is a complexity price paid for being able to scale. If one can avoid paying this cost then he should.

And USUALLY cross-shard queries are not just expensive but simply impossible in operational clusters. Like, if you do COUNT on a table, you only count within a single db shard table.

Re: Making 768 servers look like 1

#26

Post by javier2 »

drdexebtjl wrote: post #5

What about sequences? The example shows an auto-incrementing user ID. How’s that possible without contention between all shards? Is the proxy responsible for sequences? What about foreign keys? Do they all have to live on the same shard? How do you do distributed transactions? On cross-shard reads: how do you do sorting? And cross-shard joins? I’d love to be proven wrong, but I suspect the 768 servers look like 1 onl…

low-traffic auto increment like user creation can be relatively performant by reserving batches up front, then using those in each shard.

But the use-case for this is mostly if you do not need, or have very limited use for anything cross-shard.

Re: Making 768 servers look like 1

#28

Post by skeptic_ai »

groundzeros2015 wrote: post #13

I disagree with the opening premise: > A single database server cannot handle such demand, so we must spread the queries and data out across many servers with database sharding Did you max out the capacity of the best server you can buy? Such a database can serve millions of customers (the numbers given). You always want to scale up the other parts first, request handlers, caching, etc. The day you can no longer insp…

Hetzner has this for 100usd a month, 256gb ram can handle quite a lot of traffic already

€84.70 max. per month €0.1357 per hour CPU Intel Xeon E5-1650V3 RAM 256 GB Drives 2 × 6.0 TB Enterprise HDD Location #FSN1-DC1 Information IPv4 ECC iNIC

Re: Making 768 servers look like 1

#30

Post by hilariously »

anichhangani wrote: post #21
groundzeros2015 wrote: post #13

I disagree with the opening premise: > A single database server cannot handle such demand, so we must spread the queries and data out across many servers with database sharding Did you max out the capacity of the best server you can buy? Such a database can serve millions of customers (the numbers given). You always want to scale up the other parts first, request handlers, caching, etc. The day you can no longer insp…

Surely the cost of running sql server on premium hardware with replication would be more than running on commodity hardware with sharding?

It depends on who you are paying for it, but generally a distributed system is harder to reason about, harder to fix, has weirder edge cases, and much more easily get into situations where it requires even MORE expertise to fix than just having a big honking server.

When you start calculating things that are not just the server, the single server looks cheaper and cheaper. How do you get a consistent backup? How do you do DR? How do you tune queries when it could go to this node or that node? Now writes are going to be significantly slower if you need multi-node commit because no matter what you are racing the speed of light on the network.

Post reply on HN