Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making 768 servers look like 1

planetscale.com

Post reply on HN
Print view View original

71–80 of 89 posts

Re: Making 768 servers look like 1

#71

Post by prirun »

chiph wrote: post #43
citrin_ru wrote: post #22

Earlier quoted context omitted.

> The example shows an auto-incrementing user ID. How’s that possible without contention between all shards? Why not divide ID range (63bit?) by the maximum planned number of shards and then set on each shard it's min/max value so ranges will not overlap?

Because if the business grows, then you will need to add servers, and reallocate each existing shard's min/max values. If you left room to grow (divide amongst 2^31 instead of 2^63) you can go a long way before you have to solve that problem. If the business turns out to be wildly successful, you'd probably set up a whole new pool anyway and migrate over (and get a new set of pre-allocated ID ranges). This might be t…

Consistent hashing helps solve this. You don't divide into shards based on the id, but on the consistent hash of the id. Then when you have to add shards, it's easier to distribute data to the new shards. If you have 3 shards and expand to 4, only 1/4th of the data from each shard gets moved to the new shard.

Re: Making 768 servers look like 1

#72

Post by drdexebtjl »

vkazanov wrote: post #24
ahk-dev wrote: post #12

Earlier quoted context omitted.

This seems like the important distinction: making the infrastructure look like one database to the application is different from making it behave like one unrestricted relational database. At what point does hiding the sharding become counterproductive? I imagine teams still need a fairly deep understanding of shard keys, query routing, and failure modes to avoid accidentally expensive cross-shard operations.

Yes! Distributed systems introduce severe restrictions on what can be reasonably done at scale. Having a single connection string is one thing, being able to do a massive JOIN is another (and should only be ever done in analytical databases). The question is not "when sharding becomes counter-productive" but "when it starts making sense". With sharding something somewhere has to know how to route queries to subsets o…

Parent commenter was not asking about when sharding becomes counterproductive, it was asking about when hiding sharding from the application becomes counterproductive.

The point is that it’s a leaky abstraction. It should not be at the database proxy level.

Instead, the application should be responsible to route queries to the correct shard. That way it can’t make cross-shard queries or cross-shard transactions accidentally, it must do so explicitly.

Re: Making 768 servers look like 1

#73

Post by marcosdumay »

mike_hearn wrote: post #67
marcosdumay wrote: post #60

Earlier quoted context omitted.

Lol about buying Oracle for performance! But anyway, do you have benchmarks from anybody not related to Oracle showing that performance? Because Oracle forbids talking about it...

Not forbidden. You can email a specific address to share results/setup and get permission to publish if you want. Other commercial databases also do that, because there's so many ways to misconfigure a database to make it slow and competitors are strongly incentivized to do so. The question is what you'd want to benchmark. For example, imagine testing Postgres with a write load that goes well beyond what a single mac…

> Other commercial databases also do that

That's why you won't see many serious comments bragging about the performance of SQL Server either.

Even though is has way fewer performance land-mines that you must design your entire architecture around than Oracle.

Anyway, you are correct that RAC is a very impressive piece of software. If Postgres had something like it, it would be a beast. It's not enough to save Oracle, though.

Re: Making 768 servers look like 1

#74

Post by drdexebtjl »

bddicken wrote: post #53
drdexebtjl wrote: post #5

What about sequences? The example shows an auto-incrementing user ID. How’s that possible without contention between all shards? Is the proxy responsible for sequences? What about foreign keys? Do they all have to live on the same shard? How do you do distributed transactions? On cross-shard reads: how do you do sorting? And cross-shard joins? I’d love to be proven wrong, but I suspect the 768 servers look like 1 onl…

A lot of what you're referring to is dictated by the sharding strategy. Vitess and Neki both let you configure this via the VSchema / data topology (That's what I'm getting at here) https://planetscale.com/blog/making-768-servers-look-like-1#... This decision matters a ton. I drilled into this specific point on sharding in one of my articles from awhile back: https://planetscale.com/blog/database-sharding This is som…

> We want 99% of what we do to be single shard, but occasionally cross-shard work is unavoidable.

Why optimize for the 1% and put the logic in a database proxy, then? It’s a leaky abstraction.

Require the application to explicitly and loudly make a query for each shard and collate the results.

Re: Making 768 servers look like 1

#75

Post by pixl97 »

groundzeros2015 wrote: post #70
pixl97 wrote: post #61

Earlier quoted context omitted.

And what if they cost more than your entire AWS bill? You seem to have misunderstanding on what scales some things run at, and expense curves of getting them to run at those levels. A poorly built app running on hundreds of servers can be massively expensive. A well built app running on a single server can also be massively expensive because people that can build apps like that tend to command whatever price they wan…

It’s not deep magic or expensive. It’s simply asking hmm, how does my operating systems textbook solve this? Before asking how AWS solves this.

So, no, you don't actually work anywhere with a global team running tons of projects.

Re: Making 768 servers look like 1

#76

Post by mike_hearn »

marcosdumay wrote: post #73
mike_hearn wrote: post #67

Earlier quoted context omitted.

Not forbidden. You can email a specific address to share results/setup and get permission to publish if you want. Other commercial databases also do that, because there's so many ways to misconfigure a database to make it slow and competitors are strongly incentivized to do so. The question is what you'd want to benchmark. For example, imagine testing Postgres with a write load that goes well beyond what a single mac…

> Other commercial databases also do that That's why you won't see many serious comments bragging about the performance of SQL Server either. Even though is has way fewer performance land-mines that you must design your entire architecture around than Oracle. Anyway, you are correct that RAC is a very impressive piece of software. If Postgres had something like it, it would be a beast. It's not enough to save Oracle,…

What sort of landmines are you referring to? This is quite interesting.

Still it's not just SQL Server and Oracle. PlanetScale - the author of this blog post - also had a DeWitt clause until literally just days ago. They replaced it with a slightly different clause but I wouldn't be willing to benchmark under their new rules.

https://planetscale.com/blog/transparency-in-benchmarking

It says benchmarking is allowed only if "the Benchmark is conducted in good faith and uses a fair and transparent methodology". What does good faith mean, or fair and transparent? You still effectively need their permission because otherwise they might decide you weren't benchmarking in "good faith" retroactively.

It's no surprise. The stuff that makes database benchmarking useless is the variety of features. You can always design a benchmark that makes a specific vendor look good or bad.

Like, Oracle can add missing indexes on the fly. If someone designed a benchmark in which the core table was huge but missing an index, PostgreSQL would die and Oracle would recover then fly. But how many would accept that as a fair test?

Re: Making 768 servers look like 1

#77

Post by g8oz »

mike_hearn wrote: post #59

I should start by disclosing that I work part time in the Oracle Database group, but - of course - my HN account is entirely my own, despite occasional mild shilling. The article itself is shilling for PlanetScale so that seems OK. The author - certainly not deliberately! - says some untrue things about relational databases. The most important one is this: > To understand why sharding is a necessary part of scaling r…

Does something similar to RAC exist for other commercial databases like Microsoft SQL Server or IBM DB2?

Re: Making 768 servers look like 1

#78

Post by marcosdumay »

mike_hearn wrote: post #76
marcosdumay wrote: post #73

Earlier quoted context omitted.

> Other commercial databases also do that That's why you won't see many serious comments bragging about the performance of SQL Server either. Even though is has way fewer performance land-mines that you must design your entire architecture around than Oracle. Anyway, you are correct that RAC is a very impressive piece of software. If Postgres had something like it, it would be a beast. It's not enough to save Oracle,…

What sort of landmines are you referring to? This is quite interesting. Still it's not just SQL Server and Oracle. PlanetScale - the author of this blog post - also had a DeWitt clause until literally just days ago. They replaced it with a slightly different clause but I wouldn't be willing to benchmark under their new rules. https://planetscale.com/blog/transparency-in-benchmarking It says benchmarking is allowed on…

> What sort of landmines are you referring to?

For example the Oracle's incapacity of reordering tests of different kinds (as in constants, functions, rows...) when optimizing a query; or the quadratic run time of triggers that change data in large updates.

Re: Making 768 servers look like 1

#79

Post by anonzzzies »

farslan wrote: post #39
anonzzzies wrote: post #14

Earlier quoted context omitted.

Well, they are selling this thing so they don't want you to buy a big server (with a read replica) as that's much cheaper.

That's not true! We have demand for bigger machines and we also sell them. You can go checkout the PlanetScale website. See: https://x.com/samlambert/status/2077197049129587150 There are truly customers that bigger machine no longer cuts. Disclaimer: I'm an Engineer at PS.

I did not say there is no market, but I am willing to bet whatever that the majority of your client base would be much cheaper off with a simple setup where they can scale the entirety of their company’s existence with one machine. Not even a large one. This cloud stuff has its place but not for most. For them it’s either themselves overspending or wasting VC money.

Re: Making 768 servers look like 1

#80

Post by mike_hearn »

g8oz wrote: post #77
mike_hearn wrote: post #59

I should start by disclosing that I work part time in the Oracle Database group, but - of course - my HN account is entirely my own, despite occasional mild shilling. The article itself is shilling for PlanetScale so that seems OK. The author - certainly not deliberately! - says some untrue things about relational databases. The most important one is this: > To understand why sharding is a necessary part of scaling r…

Does something similar to RAC exist for other commercial databases like Microsoft SQL Server or IBM DB2?

DB2 yes, SQL Server no.

There are some cloud databases that scale horizontally but they often only support a subset of SQL, or require careful design to avoid creating hot tablets due to using a shared nothing design e.g. due to hot read nodes.

Post reply on HN