The Inner Workings of Distributed Databases • Hacker News Board

Re: The Inner Workings of Distributed Databases

#11

Post by tabtab » Mon, Apr 17, 2023, 11:55 PM UTC

moomoo11 wrote: post #10

MuffinFlavored wrote: post #3

When is the right time to "level up" from "I'm good with just plain old Postgres" to QuestDB, InfluxDB, Patroni, etc.? > Unfortunately, automatic failover is solved neither by PostgreSQL nor TimescaleDB, but there are 3rd-party solutions like Patroni that add support for that functionality. PostgreSQL describes the process of failover as STONITH (Shoot The Other Node In The Head), meaning that the primary node has to…

You don’t choose a database to “level up”. It’s a tool. Use the right tool for the right job. I’ve migrated rdbms to wide column databases like Cassandra or dynamo because we had specific requirements that rdbms were not fulfilling. I’ve also migrated from document database to rdbms because the document store didn’t meet our specific requirements. I wouldn’t just use any random database because I want to appear cool…

> I’ve migrated rdbms to wide column databases like Cassandra or dynamo because we had specific requirements that rdbms were not fulfilling.

I'm curious of a common situation. Could RDBMS be improved on that area, or do they inherently lack some necessary property?

I will agree that current RDBMS tend to lack dynamism, and that should be remedied: https://www.reddit.com/r/CRUDology/comments/12ari2l/dynamic_...

Re: The Inner Workings of Distributed Databases

#12

Post by fzliu » Mon, Apr 17, 2023, 11:59 PM UTC

moomoo11 wrote: post #10

MuffinFlavored wrote: post #3

When is the right time to "level up" from "I'm good with just plain old Postgres" to QuestDB, InfluxDB, Patroni, etc.? > Unfortunately, automatic failover is solved neither by PostgreSQL nor TimescaleDB, but there are 3rd-party solutions like Patroni that add support for that functionality. PostgreSQL describes the process of failover as STONITH (Shoot The Other Node In The Head), meaning that the primary node has to…

You don’t choose a database to “level up”. It’s a tool. Use the right tool for the right job. I’ve migrated rdbms to wide column databases like Cassandra or dynamo because we had specific requirements that rdbms were not fulfilling. I’ve also migrated from document database to rdbms because the document store didn’t meet our specific requirements. I wouldn’t just use any random database because I want to appear cool…

Here's a page on the architecture of a distributed vector database (Milvus) for anybody interested: https://milvus.io/docs/architecture_overview.md

Re: The Inner Workings of Distributed Databases

#13

Post by moomoo11 » Tue, Apr 18, 2023, 12:07 AM UTC

tabtab wrote: post #11

moomoo11 wrote: post #10

Earlier quoted context omitted.

You don’t choose a database to “level up”. It’s a tool. Use the right tool for the right job. I’ve migrated rdbms to wide column databases like Cassandra or dynamo because we had specific requirements that rdbms were not fulfilling. I’ve also migrated from document database to rdbms because the document store didn’t meet our specific requirements. I wouldn’t just use any random database because I want to appear cool…

> I’ve migrated rdbms to wide column databases like Cassandra or dynamo because we had specific requirements that rdbms were not fulfilling. I'm curious of a common situation. Could RDBMS be improved on that area, or do they inherently lack some necessary property? I will agree that current RDBMS tend to lack dynamism, and that should be remedied: https://www.reddit.com/r/CRUDology/comments/12ari2l/dynamic_...

It’s less to do with rdbms specifics and more to do with things like globally available replication and super low latency.

As for features I mean you could use Postgres and some extensions and you’re good to go. If you want a document db you can just use jsonb type.

You need to know your access patterns and what queries you will be running beforehand when working with something like dynamo. You can also do relational modeling with dynamo following single table design and basically pre join.

Re: The Inner Workings of Distributed Databases

#14

Post by gregwebs » Tue, Apr 18, 2023, 1:30 AM UTC

TiDB and CRDB handles all these scenarios. They are designed for synchronized distributed replication from the ground up and a tremendous amount of engineering work has gone into these systems.

Re: The Inner Workings of Distributed Databases

#15

Post by foodoos » Tue, Apr 18, 2023, 2:39 AM UTC

> we chose our goal to be achieving multi-master replication with Async consistency. We believe that this approach strikes the best balance of fault tolerance and transaction throughput.

"SLOG: Serializable, Low-latency, Geo-replicated Transactions"

https://par.nsf.gov/servlets/purl/10126332

Re: The Inner Workings of Distributed Databases

#16

Post by jimbokun » Tue, Apr 18, 2023, 3:27 AM UTC

moomoo11 wrote: post #10

MuffinFlavored wrote: post #3

When is the right time to "level up" from "I'm good with just plain old Postgres" to QuestDB, InfluxDB, Patroni, etc.? > Unfortunately, automatic failover is solved neither by PostgreSQL nor TimescaleDB, but there are 3rd-party solutions like Patroni that add support for that functionality. PostgreSQL describes the process of failover as STONITH (Shoot The Other Node In The Head), meaning that the primary node has to…

You don’t choose a database to “level up”. It’s a tool. Use the right tool for the right job. I’ve migrated rdbms to wide column databases like Cassandra or dynamo because we had specific requirements that rdbms were not fulfilling. I’ve also migrated from document database to rdbms because the document store didn’t meet our specific requirements. I wouldn’t just use any random database because I want to appear cool…

I want to thank you for the advice to “use the right tool for the job” because it’s certainly not banal, prosaic advice that is invoked in every technology discussion.

Re: The Inner Workings of Distributed Databases

#17

Post by jimbokun » Tue, Apr 18, 2023, 3:29 AM UTC

tabtab wrote: post #11

moomoo11 wrote: post #10

Earlier quoted context omitted.

You don’t choose a database to “level up”. It’s a tool. Use the right tool for the right job. I’ve migrated rdbms to wide column databases like Cassandra or dynamo because we had specific requirements that rdbms were not fulfilling. I’ve also migrated from document database to rdbms because the document store didn’t meet our specific requirements. I wouldn’t just use any random database because I want to appear cool…

> I’ve migrated rdbms to wide column databases like Cassandra or dynamo because we had specific requirements that rdbms were not fulfilling. I'm curious of a common situation. Could RDBMS be improved on that area, or do they inherently lack some necessary property? I will agree that current RDBMS tend to lack dynamism, and that should be remedied: https://www.reddit.com/r/CRUDology/comments/12ari2l/dynamic_...

Transactions and joins are the parts that are tricky to implement in a performant way in distributed systems. No SQL systems generally scale well by leaving those features out. Or at least heavily restricting them.

Re: The Inner Workings of Distributed Databases

#18

Post by Dylan16807 » Tue, Apr 18, 2023, 3:49 AM UTC

moomoo11 wrote: post #10

MuffinFlavored wrote: post #3

When is the right time to "level up" from "I'm good with just plain old Postgres" to QuestDB, InfluxDB, Patroni, etc.? > Unfortunately, automatic failover is solved neither by PostgreSQL nor TimescaleDB, but there are 3rd-party solutions like Patroni that add support for that functionality. PostgreSQL describes the process of failover as STONITH (Shoot The Other Node In The Head), meaning that the primary node has to…

You don’t choose a database to “level up”. It’s a tool. Use the right tool for the right job. I’ve migrated rdbms to wide column databases like Cassandra or dynamo because we had specific requirements that rdbms were not fulfilling. I’ve also migrated from document database to rdbms because the document store didn’t meet our specific requirements. I wouldn’t just use any random database because I want to appear cool…

> I wouldn’t just use any random database because I want to appear cool

Obviously! "level up" does not imply you used the wrong thing on purpose and you're switching to another solution that's better in every way, as you seem to have read it.

Re: The Inner Workings of Distributed Databases

#19

Post by diarrhea » Tue, Apr 18, 2023, 4:32 AM UTC

MuffinFlavored wrote: post #3

When is the right time to "level up" from "I'm good with just plain old Postgres" to QuestDB, InfluxDB, Patroni, etc.? > Unfortunately, automatic failover is solved neither by PostgreSQL nor TimescaleDB, but there are 3rd-party solutions like Patroni that add support for that functionality. PostgreSQL describes the process of failover as STONITH (Shoot The Other Node In The Head), meaning that the primary node has to…

> Aren't all distributed databases basically really clever wrappers around write-ahead log + really tight timestamp/clock syncing?

As far as I know, the second requirement is often solved differently. Google’s Spanner has tight clock synchronisation via GPS and/or atomic clocks, and will even report uncertainties. Knowing these uncertainties allows it to simply wait them out before committing, for example.

But in general, exact time keeping and clock syncing is often too hard and costly. Luckily, it’s often not required and one can do with logical clocks, such as version vectors or Lamport time stamps. These order events by causality (A before B, B before A, A and B happened concurrently), which eventually allows the WAL to be sorted deterministically.

Things like multi leader with async replication will inevitably run into conflicts though. These will need some sort of resolution (manually or automatically via CRDTs). There’s no way around it due to the builtin, inherent possibility of concurrent writes.

Note that concurrent in these scenarios has essentially nothing to do with time. It’s not about “happened at the same time”. It’s a question of “did A know about B?”. No? Then A can’t be causally dependent on B and they are concurrent events. Exactly like two “parallel” branches in git. They’ll need to be merged later on, and conflicts will need to be resolved.

Lastly, if we can deterministically order events, every node can reach the same conclusions. This is equivalent to consensus.

So my take would be: distributed databases are often about a log of (write) events, and some consensus mechanism to agree upon the exact order in that log. Logical clocks are a good solution for that, but physical clocks ca be made to work as well (Google Spanner).

This is all taken from the book “Designing Data Intensive Applications”, a great read!

Re: The Inner Workings of Distributed Databases

#20

Post by Andys » Tue, Apr 18, 2023, 5:09 AM UTC

MuffinFlavored wrote: post #3

When is the right time to "level up" from "I'm good with just plain old Postgres" to QuestDB, InfluxDB, Patroni, etc.? > Unfortunately, automatic failover is solved neither by PostgreSQL nor TimescaleDB, but there are 3rd-party solutions like Patroni that add support for that functionality. PostgreSQL describes the process of failover as STONITH (Shoot The Other Node In The Head), meaning that the primary node has to…

CRDB is almost a drop-in replacement at this point. I personally found it easier to run locally than postgres.