Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we built a serverless SQL database

cockroachlabs.com

Post reply on HN
Print view View original

51–60 of 122 posts

Re: How we built a serverless SQL database

#51

Post by zkldi »

bitwize wrote: post #45
phoboslab wrote: post #37

> If you’ve created a database before, you probably had to estimate how many servers to use based on the expected traffic. The answer is "one". If you have less than 10k req/s you shouldn't even start to think about multiple DB servers or migrating from bog-standard MySQL/MariaDB or Postgres. I will never understand this obsession with "scaling". Modern web dev seriously over-complicates so many things, it's not even…

(robot voice) Does /dev/null support sharding? Sharding is the secret ingredient in the web scale sauce.

it's impressive how well that video has aged with modern web dev.

Re: How we built a serverless SQL database

#53

Post by dcposch »

phoboslab wrote: post #37

> If you’ve created a database before, you probably had to estimate how many servers to use based on the expected traffic. The answer is "one". If you have less than 10k req/s you shouldn't even start to think about multiple DB servers or migrating from bog-standard MySQL/MariaDB or Postgres. I will never understand this obsession with "scaling". Modern web dev seriously over-complicates so many things, it's not even…

This is a good argument against running a k8s cluster when u don't need one, but not a good argument against this new serverless Cockroach product.

Serverless is not just about auto scaling up from 1 to n, it's about autoscaling down from 1 to 0.

If Cockroach provides a robust SQL DB at a marginal cost of ~$0/mo for small request volumes, that is a real value add over running your own pg server.

Not having to deal with administration or backups is another big value add.

This offering looks like it compares very nicely to say running an rds instance with auto backups enabled.

Re: How we built a serverless SQL database

#54

Post by [deleted] »

phoboslab wrote: post #37

> If you’ve created a database before, you probably had to estimate how many servers to use based on the expected traffic. The answer is "one". If you have less than 10k req/s you shouldn't even start to think about multiple DB servers or migrating from bog-standard MySQL/MariaDB or Postgres. I will never understand this obsession with "scaling". Modern web dev seriously over-complicates so many things, it's not even…

[deleted]

Re: How we built a serverless SQL database

#55

Post by GuB-42 »

brokencode wrote: post #42
phoboslab wrote: post #37

> If you’ve created a database before, you probably had to estimate how many servers to use based on the expected traffic. The answer is "one". If you have less than 10k req/s you shouldn't even start to think about multiple DB servers or migrating from bog-standard MySQL/MariaDB or Postgres. I will never understand this obsession with "scaling". Modern web dev seriously over-complicates so many things, it's not even…

The question isn’t whether you have 10k req/s now, but whether you expect to in the future. If you are designing a blog, then yeah, you probably don’t need to worry about it. If you are starting a social network or SAAS business application, then you probably do.

The future will be different.

A lot of successful businesses start with things that are not scalable, and it is a strength, not a weakness. If you start a social network for instance, you can't beat Facebook at its own game. You have to do something that Facebook can't do because it is too big. Scalability problems will be tackled as you grow.

Among the many things Facebook can't do is running its service on a single database. It makes things much harder on them, thankfully, you are much smaller than Facebook and you can. Take that advantage.

Re: How we built a serverless SQL database

#56

Post by dboreham »

Groxx wrote: post #48
mjibson wrote: post #43

Earlier quoted context omitted.

The setup you describe is very much not simple. I worked at a place with very good DBAs and our replication setup caused us more downtime than anything else. Cockroach and Spanner exist because many programmers observed that what you describe is hard.

As a counter-anecdote: multiple startup projects I've worked on with separate MySQL setups where each had just a single master + two slaves (one warm for fast failover in case of hardware failure or upgrades, one cold for slow analytics-style queries) did just fine with millions (to tens of millions) of users. No downtime at all for years on end. MySQL and Postgres are massively more widely-used than Cockroach and Sp…

Very few deployments experience actual failures. Could be some fridge-door/light situation going on.

Re: How we built a serverless SQL database

#57

Post by andydb »

dilyevsky wrote: post #35
nonameiguess wrote: post #27

Earlier quoted context omitted.

It's understandly not something a lot of vendors consider, especially early stage products, but classified systems don't have Internet access and can't use managed cloud services (unless offered by AWS via C2S, and possibly in the future by whoever wins JEDI if they resurrect that), leaving no choice but to self-host anything. That's the extremest example, but lots of enterprises airgap part of their network for othe…

In addition to security/legal stuff there are also concerns around slo/business continuity and costs

Sure, those are all good points. We'll definitely keep an eye on customer demand in these areas so we can make good prioritization decisions. It can be frustrating when there are so many things that we'd like to do, but we can only pick a small subset to actually do.

Re: How we built a serverless SQL database

#58

Post by lumost »

phoboslab wrote: post #37

> If you’ve created a database before, you probably had to estimate how many servers to use based on the expected traffic. The answer is "one". If you have less than 10k req/s you shouldn't even start to think about multiple DB servers or migrating from bog-standard MySQL/MariaDB or Postgres. I will never understand this obsession with "scaling". Modern web dev seriously over-complicates so many things, it's not even…

The answer is unfortunately less clear cut. Particularly if you assume that whoever is tasked with scaling this hypothetical DB doesn't know what they are doing a-priori.

The following questions are likely to come up

1) My t3.xl DB is down, how much bigger can I make it?

2) My r3.24xl DB can only handle 100 TPS and now my site is down, what can I do?

3) My 2x r3.24xl DB cluster costs a lot of money, Are other solutions cheaper?

4) My latency is high, are other solutions faster?

For someone who hasn't dealt with these questions before, these will become long and painful lessons with massive material impacts to the business.

It's appealing to use Dynamo as it takes the magic out of scaling. It's appealing to use serverless RDBMS as you don't have to think about it anymore unless it has high costs/latency.

Re: How we built a serverless SQL database

#60

Post by aaaaaaaaaaab »

psadri wrote: post #44
phoboslab wrote: post #37

> If you’ve created a database before, you probably had to estimate how many servers to use based on the expected traffic. The answer is "one". If you have less than 10k req/s you shouldn't even start to think about multiple DB servers or migrating from bog-standard MySQL/MariaDB or Postgres. I will never understand this obsession with "scaling". Modern web dev seriously over-complicates so many things, it's not even…

Unfortunately, easy to hit that with say GraphQL where each client request can resolve to dozens of db selects vs a single hand written/tuned SQL select.

Maybe that's a good reason to avoid GraphQL then?
Post reply on HN