Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making 768 servers look like 1

planetscale.com

Post reply on HN
Print view View original

81–89 of 89 posts

Re: Making 768 servers look like 1

#81

Post by vkazanov »

samlambert wrote: post #54
vkazanov wrote: post #11

Earlier quoted context omitted.

Of course 768 servers NEVER behave as 1. This is physically impossible. Global services using relational dbs typically severely restrict queries that run against the cluster. So no joins, no intervals, no grouping, etc. Transactional queries are usually limited to something like "get a single record, preferably from cache". For many typical web services this can go VERY FAR. Only a handful of global services needs mo…

> Transactional queries are usually limited to something like "get a single record, preferably from cache". simply wrong

Well, operational dbs in practical circumstances are used for queries that take predictable time to complete.

"Get a row" "Get ten rows"

Etc, etc. All constant, all predictable, all not requiring unpredictable time to run. Also caches nicely.

What is "simply wrong" here?

Re: Making 768 servers look like 1

#82

Post by groundzeros2015 »

bddicken wrote: post #51
groundzeros2015 wrote: post #48

Earlier quoted context omitted.

> These companies could not realistically operate off a single database server. I want to see it fail first. I suggest their org chart has more to do with their architecture than database performance. Even so you’re in the category I said. Hundreds of expensive engineers maintain this stuff.

You should read through those articles.

Thanks for sharing. They validated exactly what I said was of concern:

- loss of atomicity and transactions.

- tons of custom tooling to work around limitation of custom sql (knowing which operations are allowed and consistent).

- GitHub was able to operate for almost a decade on a single MySQL instance, and it’s not clear they really hard to change.

Etsy said this was a huge problem and took 5 years and thousands of PRs to resolve.

Re: Making 768 servers look like 1

#83

Post by groundzeros2015 »

bddicken wrote: post #57
anonzzzies wrote: post #14

Earlier quoted context omitted.

Well, they are selling this thing so they don't want you to buy a big server (with a read replica) as that's much cheaper.

We have tons of customers who do exactly this. It's great. Sharding is for customers who out grow this path.

Why would multiple machines be cheaper than one? Don’t you pay an overhead per machine?

Re: Making 768 servers look like 1

#84

Post by groundzeros2015 »

pixl97 wrote: post #75
groundzeros2015 wrote: post #70

Earlier quoted context omitted.

It’s not deep magic or expensive. It’s simply asking hmm, how does my operating systems textbook solve this? Before asking how AWS solves this.

So, no, you don't actually work anywhere with a global team running tons of projects.

I’m available to consult on these secret special methods If you’ve got a project.

Re: Making 768 servers look like 1

#85

Post by amazingamazing »

groundzeros2015 wrote: post #13

I disagree with the opening premise: > A single database server cannot handle such demand, so we must spread the queries and data out across many servers with database sharding Did you max out the capacity of the best server you can buy? Such a database can serve millions of customers (the numbers given). You always want to scale up the other parts first, request handlers, caching, etc. The day you can no longer insp…

and if your one beefy server goes down? oh, now you want a replica. so much for one beefy machine.

Re: Making 768 servers look like 1

#86

Post by kbrajesh176 »

drdexebtjl wrote: post #74
bddicken wrote: post #53

Earlier quoted context omitted.

A lot of what you're referring to is dictated by the sharding strategy. Vitess and Neki both let you configure this via the VSchema / data topology (That's what I'm getting at here) https://planetscale.com/blog/making-768-servers-look-like-1#... This decision matters a ton. I drilled into this specific point on sharding in one of my articles from awhile back: https://planetscale.com/blog/database-sharding This is som…

> We want 99% of what we do to be single shard, but occasionally cross-shard work is unavoidable. Why optimize for the 1% and put the logic in a database proxy, then? It’s a leaky abstraction. Require the application to explicitly and loudly make a query for each shard and collate the results.

Exactly. sharding is for scaling usecase where queries can be handled within a shard(co-partitioning). If sharding strategy can assign keys from two tables where join is needed you can get achieve same performance.

In another usecase where joins are not needed realtime, you can build another system where results are precomputed and cached. I consider sharding as methodology for scaling instead of onefit all solution. Overall it depends on usecase and sound sharding strategy for performance.

Re: Making 768 servers look like 1

#87

Post by marysol5 »

chaps wrote: post #64
samlambert wrote: post #54

Earlier quoted context omitted.

> Transactional queries are usually limited to something like "get a single record, preferably from cache". simply wrong

Care to explain? Not getting a good vibe from you, Mr. CEO.

Getting developers to work with the infrastructure we have, and not make everyone's lives substantially harder is near impossible.

Re: Making 768 servers look like 1

#88

Post by hilariously »

marginalia_nu wrote: post #66
hilariously wrote: post #30

Earlier quoted context omitted.

It depends on who you are paying for it, but generally a distributed system is harder to reason about, harder to fix, has weirder edge cases, and much more easily get into situations where it requires even MORE expertise to fix than just having a big honking server. When you start calculating things that are not just the server, the single server looks cheaper and cheaper. How do you get a consistent backup? How do y…

Another thing worth thinking about when running anything in parallel is the rather massive increase in error rate. When you're essentially rolling 768x the number of dice, failure outcomes will happen a lot more often than if you just roll the one or two. It's a scale that transforms a 10 year MTBF to a ~5 day MTBF.

A great point, you will also eventually be in a constant state of failure, and have to turn that into something you can manage, automatically.

Re: Making 768 servers look like 1

#89

Post by dizhn »

amazingamazing wrote: post #85
groundzeros2015 wrote: post #13

I disagree with the opening premise: > A single database server cannot handle such demand, so we must spread the queries and data out across many servers with database sharding Did you max out the capacity of the best server you can buy? Such a database can serve millions of customers (the numbers given). You always want to scale up the other parts first, request handlers, caching, etc. The day you can no longer insp…

and if your one beefy server goes down? oh, now you want a replica. so much for one beefy machine.

In the last 10 year or so my 2 beefy machines had much better uptime than the top clouds. :)

I am not suggesting you do this but some businesses can actually be down a little bit, like my non-IT one. I've had issues in the past with email. If the issue happens after business hours and resolves before start of business next day, nobody will even notice.

Post reply on HN