The database ruins all good ideas
squarism.com
The database ruins all good ideas
1–10 of 165 posts
Re: The database ruins all good ideas
#2Am I just dense, or was the idea the CEO had just not a good idea and simply garden variety premature optimization?
Also, it seems unfair to say that the database ruins all good ideas when there continues to be significant impressive innovation in the space in cloud services. Like Amazon Aurora for one. If you really want to treat your RDBMS like an elastically scalable (with significant caveats) resource priced by the request, you can.
Re: The database ruins all good ideas
#3Use CockroachDB.
Re: The database ruins all good ideas
#4No it doesn’t. They scale amazingly well if you throw money at the problem. Most people never get there. When you do you will know. I’ve been there. When you’re spending $3 million on hardware and licenses a year you either have a viable business or fucked up badly. That’s the real decider.
The answer is to start siloing customers or application concerns out into separate clusters depending on your operating model.
If you spent the last decade writing shitty cross database SQL then you’ll have to silo by customer. That’s the only real constraint.
Re: The database ruins all good ideas
#5The mistake is thinking half the system is doing nothing. It’s a bit like saying I’m paying to store my backups offsite but most of the time it’s not really used so it’s a waste of money. It just doesn’t hold.
Most SQL problems that most of us have to deal with stem from inadequate indexes and/or poorly written queries. No fancy active-active setup will ever solve those issues.
Re: The database ruins all good ideas
#6Can’t be bothered. Aurora in Postgres/MySQL and then just scale that vertically till it stops.
Re: The database ruins all good ideas
#7No it doesn’t. They scale amazingly well if you throw money at the problem. Most people never get there. When you do you will know. I’ve been there. When you’re spending $3 million on hardware and licenses a year you either have a viable business or fucked up badly. That’s the real decider. The answer is to start siloing customers or application concerns out into separate clusters depending on your operating model. I…
I'd phrase it differently: the database is the litmus test of your supposed "good idea". If your good idea doesn't survive referential integrity or even data consistency, your idea doesn't deserve the label "good".
Of course, the next "good idea" is to sacrifice data integrity in the name of performance. That can work, but usually it's just a technical form of growth hacking. Sacrificing data integrity without understanding the problem space is a disaster waiting to happen (but luckily, it may not happen on your watch so you can continue hacking).
Re: The database ruins all good ideas
#8If you really want or need to go active-active but also don’t want to think about it much and can live with MySQL my preference would probably be AWS Aurora multi-master if you can live with its limitations:
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...
As noted there and elsewhere though often you are better off just scaling up.
If you do want to really think about it then NewSQL (Cloud Spanner or others) could be the way to go but the cost, performance, compatibility/portability and other concerns make this a much more complex decision that isn’t easy to capture in a blog post (or HN comment ;).
Re: The database ruins all good ideas
#9No it doesn’t. They scale amazingly well if you throw money at the problem. Most people never get there. When you do you will know. I’ve been there. When you’re spending $3 million on hardware and licenses a year you either have a viable business or fucked up badly. That’s the real decider. The answer is to start siloing customers or application concerns out into separate clusters depending on your operating model. I…
I'd phrase it differently: the database is the litmus test of your supposed "good idea". If your good idea doesn't survive referential integrity or even data consistency, your idea doesn't deserve the label "good". Of course, the next "good idea" is to sacrifice data integrity in the name of performance. That can work, but usually it's just a technical form of growth hacking. Sacrificing data integrity without unders…
Very good points there.
I have directly experienced the problems associated with blindly assuming that you can write your own consistency guarantees in Redis. I didn't do it but I had to write some tooling to rewrite the AOF to get some data back...
Re: The database ruins all good ideas
#10Use CockroachDB.
To solve what problem?