I’m not convinced the author knows what they’re talking about. The answer to the question they posed is basically “ACID”, so unclear what all the verbiage is. Also unclear why they expect all readers to have an architecture featuring multiple application servers all sharing a single database, since that is neither classic monolith nor classic microservices.
The database ruins all good ideas
31–40 of 165 posts
Re: The database ruins all good ideas
#32Earlier quoted context omitted.
I'd phrase it differently: the database is the litmus test of your supposed "good idea". If your good idea doesn't survive referential integrity or even data consistency, your idea doesn't deserve the label "good". Of course, the next "good idea" is to sacrifice data integrity in the name of performance. That can work, but usually it's just a technical form of growth hacking. Sacrificing data integrity without unders…
I'm sorry, are you saying that a good idea should survive the loss of referential integrity or data consistency? I don't feel that's what you mean, and it's probably my fault for misreading your comment.
Yeah, I probably could have phrased that better. The sibling comment is correct, I meant that if your idea requires abandoning referential integrity or data consistency, it's probably not a good idea. Since it's the database that actually enforces those constraints, it may seem that the database causes the problem. But in most cases, the problem is the data model or the idea itself.
Re: The database ruins all good ideas
#33Earlier quoted context omitted.
> The answer is to start siloing customers or application concerns out into separate clusters depending on your operating model. This is such an underrated solution (siloing or sharding your data in some way). I think people don't do it because: 1. The tooling doesn't make it super-easy (e.g. good luck sharding Postgres unless you're willing to pay for Citus) 2. "Trendy" companies in the past decade have been network…
For your "life would be easier" version, isn't that pretty much what Citus does? And Citus is open source so paying for the managed version is optional.
saying "Citus is open source" doesn't mean its as simple as running `yarn install citus; yarn run citus`. Its is much much more complicated than typical open source. You pay for support for a very good reason.
Re: The database ruins all good ideas
#34The article is talking about Oracle RAC, and the "crossover" he is talking about was usually really quick Infiniband, not a crossover cable (so, a separate NIC for good reason!). The person that wrote the article doesn't seem to understand RAC enough to actually comment properly. A properly tuned RAC instance will scale horizontally very very well.
Former (recovering?) Exadata user here. I'll go as far as to say that configuring a best-practices RAC cluster is easier than configuring a best-practices MySQL cluster. RAC solves some additional problems, like discovery and load balancing, that MySQL kicks out to the user (e.g. you have to manage DNS and run a tool like ProxySQL separately).
Re: The database ruins all good ideas
#35The title of this piece is great: very catchy. But I don't think the content supports the title - by the end of it I wasn't at all clear /why/ the database ruins all good ideas. A few other points. First, horizontally scaling database reads is actually reasonably straight-forward these days: one leader, multiple replicas, load balance reads to the replicas and use a mechanism such that users that have just performed…
horizontally scaling database reads is actually reasonably straight-forward these days: one leader, multiple replicas
There's one feature that I'd like to see in that area: partitioned writable replicas. In the same vein that you can partition the table storage across an index, I'd like it to be possible to assign different writers to different parts of a table/database. Of course, you'd still need a single primary replica to handle the transactions that transcend the configured partitions, but we already have routing engines that can transparently redirect an incoming query to any available replica, so it's partly there.
There's probably corner cases lurking that I can't even think of, but in my mind it's the only thing missing from building a truly web-scale (multiple zones, multiple datacenters) ACID-preserving relational database.
Re: The database ruins all good ideas
#36The title of this piece is great: very catchy. But I don't think the content supports the title - by the end of it I wasn't at all clear /why/ the database ruins all good ideas. A few other points. First, horizontally scaling database reads is actually reasonably straight-forward these days: one leader, multiple replicas, load balance reads to the replicas and use a mechanism such that users that have just performed…
horizontally scaling database reads is actually reasonably straight-forward these days: one leader, multiple replicas There's one feature that I'd like to see in that area: partitioned writable replicas. In the same vein that you can partition the table storage across an index, I'd like it to be possible to assign different writers to different parts of a table/database. Of course, you'd still need a single primary r…
This is essentially the idea of VoltDB [1]:
All tables are partitioned and each partition is assigned to a single thread. As long as each query only touches a single partition, you get linear horizontal scalability and serializable isolation. The drawback is that if a query touches multiple partitions, only that one query can be executed on the entire cluster, so this architecture only works for very specific OLTP workloads.
Re: The database ruins all good ideas
#37Aren't there quite some solutions today?
Fauna, Upstash, DynamoDB, CosmoDB, Firestore, Planetscale.
Re: The database ruins all good ideas
#38Use CockroachDB.
CockroachDB is a terrible technology that causes almost guaranteed data corruption due to its lack of ACID guarantees and is written in a language with a GC which contirbutes to GC pauses. The dev team refuses to listen to feedback to port their code to C ;(.
> its lack of ACID guarantees
Our transactions implementation is our crown jewel. You might want to check your sources.
Re: The database ruins all good ideas
#39weird, nobody mention nuodb.
Re: The database ruins all good ideas
#40Databases scale just fine. Its called sharding. Azure has HyperScale (with is Citus), AWS has Aurora. The actual problem with databases is upgrading to major versions without downtime. All the other problems are just noise
Agreed. But one impediment to sharding is bad DB design, which is rife right now because everyone starts with Django/Rails/Laravel and starts treating the DB like a document store that happens to have some relational features.