Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we built a serverless SQL database

cockroachlabs.com

Post reply on HN
Print view View original

101–110 of 122 posts

Re: How we built a serverless SQL database

#101

Post by ketanip »

dboreham wrote: post #56
Groxx wrote: post #48

Earlier quoted context omitted.

As a counter-anecdote: multiple startup projects I've worked on with separate MySQL setups where each had just a single master + two slaves (one warm for fast failover in case of hardware failure or upgrades, one cold for slow analytics-style queries) did just fine with millions (to tens of millions) of users. No downtime at all for years on end. MySQL and Postgres are massively more widely-used than Cockroach and Sp…

Very few deployments experience actual failures. Could be some fridge-door/light situation going on.

> fridge-door/light situation going on

what does it mean ?

Re: How we built a serverless SQL database

#102

Post by maybevain »

ketanip wrote: post #101
dboreham wrote: post #56

Earlier quoted context omitted.

Very few deployments experience actual failures. Could be some fridge-door/light situation going on.

> fridge-door/light situation going on what does it mean ?

I think that is meant to be parsed as: Just like you can't check if the fridge light is on without opening the door (which of course turns it on), it's hard to know if a system is resilient to failure without having one. It just might be that there hasn't been a situation that would cause a failure.

Re: How we built a serverless SQL database

#103

Post by anamax »

webmaven wrote: post #94
YZF wrote: post #86

Earlier quoted context omitted.

This will be somebody else's problem? Ehm, no it's not. - When the bridge collapses I'll be managing someone else. - When they find out this airplane has critical design flaws I'll be managing someone else. - When this software I'm working on is hacked I'll be managing someone else.

Your examples aren't equivalent. You're comparing the failure of a prototype or proof-of-concept under stress to the failure of production models under expected conditions. The real risk you're incurring by deploying a prototype to production until traction is demonstrated and scaling becomes an issue is that deferring the design of the 'real' system risks succumbing to second-system syndrome and prematurely trying t…

The second doesn't (necessarily) have to scale.

It has to handle the projected load, with some safety margin.

"projected until when?" you ask?

Well, your projections will be wrong. And, your needs will change, which means that you're going to be changing the system anyway.

That's where judgment comes in.

Suppose that each customer is worth $10/month and that the "not scalable" version can handle 1M customers. That's at least $5M MRR when the system is at 50%.

If the scalable version takes significantly longer to develop, you might choose to put it off until you have more money/resources, especially since there will probably be other changes. (And, you don't know where the actual bottlenecks are.)

Re: How we built a serverless SQL database

#104

Post by chillfox »

phoboslab wrote: post #37

> If you’ve created a database before, you probably had to estimate how many servers to use based on the expected traffic. The answer is "one". If you have less than 10k req/s you shouldn't even start to think about multiple DB servers or migrating from bog-standard MySQL/MariaDB or Postgres. I will never understand this obsession with "scaling". Modern web dev seriously over-complicates so many things, it's not even…

I feel like you forgot SQLite, and for really small scale you could just use whatever storage/serialization solution comes built in to your favorite language.

Re: How we built a serverless SQL database

#105

Post by chillfox »

andydb wrote: post #2

To any who might see this, I'm the author of the blog post, and led the engineering team that built CockroachDB Serverless. I'll be monitoring this thread in case there are any questions you'd like to ask me about it.

How low can CocroachDB go with resource usage?

Is the open source version viable for small hobby projects?

Re: How we built a serverless SQL database

#106

Post by ranguna »

andydb wrote: post #70
WatchDog wrote: post #66

This offering seems a lot more compelling than the serverless databases offered by AWS directly. At least if cost and scaling to zero is important to you. Dynamodb addresses the cost issue, but it's so painful to use, and is unsuitable for a lot of use-cases. Aurora Serverless takes 30 seconds to cold-start, so it's also usually a non-starter, other than for batch type workloads. So great work, but I have a couple of…

Our product roadmap will be heavily influenced by customer asks. So if there are things about CockroachDB Serverless that prevent you from using it (like requiring ingress through a public IP), we definitely want to hear about it. Regarding a cluster limit, we currently allow up to 5 clusters per customer account. I'd like to hear what kind of use-cases you have in mind for having a lot more clusters. One I've though…

The main thing that's keeping me from moving to cockroachDB is that it doesn't support triggers. Once triggers are supported, I'll jump out of aws in a millisecond.

Re: How we built a serverless SQL database

#107

Post by [deleted] »

maybevain wrote: post #102
ketanip wrote: post #101

Earlier quoted context omitted.

> fridge-door/light situation going on what does it mean ?

I think that is meant to be parsed as: Just like you can't check if the fridge light is on without opening the door (which of course turns it on), it's hard to know if a system is resilient to failure without having one. It just might be that there hasn't been a situation that would cause a failure.

[deleted]

Re: How we built a serverless SQL database

#109

Post by [deleted] »

jungturk wrote: post #77
gkop wrote: post #69

Earlier quoted context omitted.

Tradeoffs are tradeoffs. In your k8s example, by running a k8s cluster when you don't need one, a cost you pay is the overhead. In the Cockroach serverless case, costs that come to mind include vendor lock-in when you evolve a pattern of production traffic that is hard to migrate to other solutions, and security and compliance challenges due to the virtualized instances running on shared clusters. In many cases these…

In the case you mention, you've made a tradeoff to defer developing an in-house solution for supporting a potential future pattern of production traffic, and that can be a huge accelerator. Further, once you're experiencing the types of success that demands a superior solution the priority to invest in such a solution is clear. It isn't "lock-in" - there's enough experience and capability in the market to solve those…

[deleted]

Re: How we built a serverless SQL database

#110

Post by Tanjreeve »

anamax wrote: post #87
YZF wrote: post #86

Earlier quoted context omitted.

This will be somebody else's problem? Ehm, no it's not. - When the bridge collapses I'll be managing someone else. - When they find out this airplane has critical design flaws I'll be managing someone else. - When this software I'm working on is hacked I'll be managing someone else.

All bridges have a capacity limit. "Doesn't scale" doesn't imply "inadequate." Overbuilding is waste.

Software isn't bridges though. If you run into scaling issues if you're lucky it's just a case of swapping out a database for a bigger one. More likely it isn't just that though and being able to do things like scale up distributed workers coherently with a change of a config file requires upfront thought and design that YAGNI would say isn't necessary. People just breezily saying "we can optimise it later" for a scale up of orders of magnitude are nearly always wrong. I'd argue Reddit is a perfect example of a site that's clearly running into scaling issues but are locked into an architecture with few escape hatches built in

When a bridge fails it'll just fail. When software runs into a scaling limit it'll degrade and fall on its arse constantly and be absolutely terrible as long as it takes for the software team to completely rework their entire architecture often having to learn completely new technologies.

Post reply on HN