Live data from Hacker News

How we built a serverless SQL database

cockroachlabs.com

21–30 of 122 posts

Re: How we built a serverless SQL database

#21
post #12

Earlier quoted context omitted.

Haven’t yet got time to read the whole thing so sorry if it’s already answered but is it possible to run this sql pod/storage pod separation setup yourself with crdb community/enterprise? We run enterprise crdb but it’s all in one process (with replicas)

It's not currently possible, partly because it complicates the deployment model quite a bit. Dynamically bringing SQL pods up and down requires a technology like Kubernetes. It takes some serious operational know-how to keep it running smoothly, which is why we thought it would be perfect for a managed Cloud service. What would be your company's reasons for wanting this available in self-hosted CRDB? What kinds of us…

I’m actually thinking for side proj (I’m very familiar with running production crdb in k8s via my current job) - the usecase is to bring isolated “virtual” crdb cluster per customer to save on operational overdhead of managing many crdb clusters in kubernetes (which is doable but requires extra automation). That’s not how we handle customer mutitenancy at my current job but I’m sure this could be common usecase. Now I could use your cloud but I sure am interested in diy option as well

Re: How we built a serverless SQL database

#22
post #16

I really see the greatness of the serverless option. Congratulations! What I can't really understand is why would someone use the dedicated cluster in AWS at that price.

- Pricing

- storage and db access is still mostly through hoops

- long running processes

- pain to develop, test and debug

Re: How we built a serverless SQL database

#23
post #2

To any who might see this, I'm the author of the blog post, and led the engineering team that built CockroachDB Serverless. I'll be monitoring this thread in case there are any questions you'd like to ask me about it.

Do you have a getting started guide on CockroachDB Serverless? I couldn’t find one in your docs [1]. This looks very interesting.

[1] https://www.cockroachlabs.com/docs/v21.1/example-apps.html

Re: How we built a serverless SQL database

#24
post #3
post #2

To any who might see this, I'm the author of the blog post, and led the engineering team that built CockroachDB Serverless. I'll be monitoring this thread in case there are any questions you'd like to ask me about it.

What's your elevator pitch for why my organization should use CockroachDB Serverless vs. something like AWS Aurora Serverless, particularly if we're already relatively invested in the AWS ecosystem?

Not CDB employee but CDB scales beyond what Aurora can support.

Re: How we built a serverless SQL database

#25
post #2

To any who might see this, I'm the author of the blog post, and led the engineering team that built CockroachDB Serverless. I'll be monitoring this thread in case there are any questions you'd like to ask me about it.

Do you have a getting started guide on CockroachDB Serverless? I couldn’t find one in your docs [1]. This looks very interesting. [1] https://www.cockroachlabs.com/docs/v21.1/example-apps.html

The Quickstart [1] is what you're looking for. Examples for NodeJS, Python, Go, and Java.

[1]https://www.cockroachlabs.com/docs/cockroachcloud/quickstart...

Re: How we built a serverless SQL database

#26
post #3
post #2

To any who might see this, I'm the author of the blog post, and led the engineering team that built CockroachDB Serverless. I'll be monitoring this thread in case there are any questions you'd like to ask me about it.

What's your elevator pitch for why my organization should use CockroachDB Serverless vs. something like AWS Aurora Serverless, particularly if we're already relatively invested in the AWS ecosystem?

Oh boy, I'm an engineer, but I'll do my best to pretend I'm on the sales or marketing team for a minute...

First of all, CockroachDB Serverless is available on AWS, and should integrate quite well with that ecosystem, including with Serverless functions offered by AWS Lambda.

Here are a few advantages of CockroachDB Serverless that Aurora will struggle to match (note that we're still working on Serverless multi-region support):

1. Free-forever tier. We offer a generous "free forever" tier that doesn't end after a month or a year. As the blog post outlines, our architecture is custom-built to make this economical.

2. No ceiling on write scalability. Even non-Serverless Aurora runs into increasing trouble as the number of writes / second increases past what a single machine can handle. CockroachDB just keeps going. We've had multiple high-scale customers who hit Aurora limits and had to move over to Cockroach to support business growth.

3. True multi-region support. Aurora only allows read-only, stale replicas in other regions, while CRDB allows full ACID SQL transactions. If you want to move into other regions of the world and have latency concerns or GDPR concerns, CRDB is custom-built to make the full SQL experience possible.

4. No Cloud lock-in. Perhaps this is not a concern for you company, but many companies don't like getting completely locked in to a single Cloud provider. CockroachDB works on multiple cloud providers and doesn't have a monetary interest in locking you in to just one.

5. Online schema changes. CockroachDB supports operations like adding/removing columns, renaming tables, and adding constraints without any downtime. You can perform arbitrary schema changes without disturbing your running application workloads. SQL DDL "just works".

6. Cold start in an instant. CockroachDB clusters automatically "scale to zero" when they're not in use. When traffic arrives, they resume in a fraction of a second. Compare that to Aurora, where you need to either have a minimum compute reservation, or you need to endure multi-second cold starts.

7. Great support. We've got a friendly Slack room where you can get free support and rub shoulders with fellow CockroachDB users, as well as CockroachDB folks like myself. We also have 24/7 paid support for deeper problems you might encounter.

Taken altogether, CockroachDB can go wherever your business needs it to go, without all the constraints that traditional SQL databases usually have. Do you want thousands of clusters for testing/development/tiny apps at a reasonable cost? Could your business take off and need the scale that CRDB offers? Could your business need to expand into multiple geographic regions? Are some of your workloads erratic or periodic, but still should start up instantly when needed? It's not just about what you need now, but what you may need in the future. It makes sense to plan ahead and go with a database that has "got you covered" wherever you need to go.

Re: How we built a serverless SQL database

#27
post #12

Earlier quoted context omitted.

Haven’t yet got time to read the whole thing so sorry if it’s already answered but is it possible to run this sql pod/storage pod separation setup yourself with crdb community/enterprise? We run enterprise crdb but it’s all in one process (with replicas)

It's not currently possible, partly because it complicates the deployment model quite a bit. Dynamically bringing SQL pods up and down requires a technology like Kubernetes. It takes some serious operational know-how to keep it running smoothly, which is why we thought it would be perfect for a managed Cloud service. What would be your company's reasons for wanting this available in self-hosted CRDB? What kinds of us…

It's understandly not something a lot of vendors consider, especially early stage products, but classified systems don't have Internet access and can't use managed cloud services (unless offered by AWS via C2S, and possibly in the future by whoever wins JEDI if they resurrect that), leaving no choice but to self-host anything. That's the extremest example, but lots of enterprises airgap part of their network for other reasons, and private companies that don't handle officially classified data may still be restricted in where they can host data based on PII/PHI legal regulations, and may not be able to use your data center.

Re: How we built a serverless SQL database

#28
post #26
post #3

Earlier quoted context omitted.

What's your elevator pitch for why my organization should use CockroachDB Serverless vs. something like AWS Aurora Serverless, particularly if we're already relatively invested in the AWS ecosystem?

Oh boy, I'm an engineer, but I'll do my best to pretend I'm on the sales or marketing team for a minute... First of all, CockroachDB Serverless is available on AWS, and should integrate quite well with that ecosystem, including with Serverless functions offered by AWS Lambda. Here are a few advantages of CockroachDB Serverless that Aurora will struggle to match (note that we're still working on Serverless multi-regio…

That’s a nice pitch, and certainly catched my curiosity! Going to check out CRDB.

Re: How we built a serverless SQL database

#29
post #5

Does CockroachDB Serverless expose an HTTP api? This sounds like a great fit for use with Cloudflare Workers, but that requires an http api.

I kind of assumed this came with an HTTP api, woops :) I've been using Airtable with Workers for tiny side projects. This seemed like a nice alternative to Airtable's lack of support for vanilla SQL.

Re: How we built a serverless SQL database

#30
post #26
post #3

Earlier quoted context omitted.

What's your elevator pitch for why my organization should use CockroachDB Serverless vs. something like AWS Aurora Serverless, particularly if we're already relatively invested in the AWS ecosystem?

Oh boy, I'm an engineer, but I'll do my best to pretend I'm on the sales or marketing team for a minute... First of all, CockroachDB Serverless is available on AWS, and should integrate quite well with that ecosystem, including with Serverless functions offered by AWS Lambda. Here are a few advantages of CockroachDB Serverless that Aurora will struggle to match (note that we're still working on Serverless multi-regio…

I'm sold. What's your migration story from decades old MySQL Database over to CockroachDB?

I'm only half joking there.

Post reply on HN