The database ruins all good ideas

Post by AdrianB1 » Sat, Jul 17, 2021, 10:48 PM UTC

kevml wrote: post #24

AdrianB1 wrote: post #16

A dockerized database server is something I cannot understand; I understand bare metal, I can accept virtualized, but I cannot find a good used case for a mid sized or large server (dozens of gigabytes to dozens of terabytes) dockerized and I don't know why a smaller server is a problem.

A docker image is simply packaging. There’s a small, almost always negligible, performance hit. But the value comes in being able to ship a consistent application and configuration pairing. This becomes really valuable when shipping multiple databases for multiple services.

Post by klodolph » Sat, Jul 17, 2021, 11:15 PM UTC

tremon wrote: post #35

simonw wrote: post #29

The title of this piece is great: very catchy. But I don't think the content supports the title - by the end of it I wasn't at all clear /why/ the database ruins all good ideas. A few other points. First, horizontally scaling database reads is actually reasonably straight-forward these days: one leader, multiple replicas, load balance reads to the replicas and use a mechanism such that users that have just performed…

horizontally scaling database reads is actually reasonably straight-forward these days: one leader, multiple replicas There's one feature that I'd like to see in that area: partitioned writable replicas. In the same vein that you can partition the table storage across an index, I'd like it to be possible to assign different writers to different parts of a table/database. Of course, you'd still need a single primary r…

Post by ngrilly » Sat, Jul 17, 2021, 11:44 PM UTC

pkolaczk wrote: post #25

Guvante wrote: post #20

Earlier quoted context omitted.

Because "just rewrite your program in C to avoid GC pauses" is such a flawed argument when it comes to any production system that it isn't even worth discussing. The reality is there are limited resources to work on any given project and "rewrite" is generally not the correct way to fix a given problem. Especially since the first thing you are going to need to do is show that a network system isn't resilient to GC pa…

GC pauses are not the only GC issue. Thrashing the caches and blowing up memory usage by an order of magnitude can be also very bad for performance. In a database system memory is very precious - the more of it you can use for caching / buffering users data, the better the performance. As for the subsecond spikes in latency, these tend to multiply in a distributed system. If serving a client request takes N internal…

Post by kevml » Sat, Jul 17, 2021, 11:49 PM UTC

AdrianB1 wrote: post #41

kevml wrote: post #24

Earlier quoted context omitted.

A docker image is simply packaging. There’s a small, almost always negligible, performance hit. But the value comes in being able to ship a consistent application and configuration pairing. This becomes really valuable when shipping multiple databases for multiple services.

I know what docker does, I never saw a database server deployed in mass. One of my teams manage over 250 database servers, but nobody ever supported the idea of using docker for that (all the servers run on Windows). Shipping multiple databases for multiple services does not necessarily mean multiple database servers. In the database world I saw large servers with many databases more often than multiple servers of on…

Post by cortesoft » Sat, Jul 17, 2021, 11:53 PM UTC

samatman wrote: post #22

SQLite is a remarkably good solution to most of these problems, if deployed correctly. For your main line-of-business database? Of course not. But a deployment of rqlite[0] for your service workers in a read-heavy workload? cuts out a round-trip out of the VM, mocking is trivial, there's a lot to like there. [0]: https://github.com/rqlite/rqlite