YugabyteDB (open source) uses the Postgres code, to provide all PostgreSQL features, plugged on top of a Spanner-like distributed storage and transactions to scale horizontally:
https://docs.yugabyte.com/preview/architecture/layered-archi...
The Inner Workings of Distributed Databases
21–28 of 28 posts
Re: The Inner Workings of Distributed Databases
#22Is there any book/textbook course out there that goes through how to write a database or dbms from scratch up to something useful, think something like nand to tetris style? I have been looking but there is not much on this topic out there I feel like.
Re: The Inner Workings of Distributed Databases
#23Earlier quoted context omitted.
You don’t choose a database to “level up”. It’s a tool. Use the right tool for the right job. I’ve migrated rdbms to wide column databases like Cassandra or dynamo because we had specific requirements that rdbms were not fulfilling. I’ve also migrated from document database to rdbms because the document store didn’t meet our specific requirements. I wouldn’t just use any random database because I want to appear cool…
> I wouldn’t just use any random database because I want to appear cool Obviously! "level up" does not imply you used the wrong thing on purpose and you're switching to another solution that's better in every way, as you seem to have read it.
I’ve met more people who have used the wrong tool for the job than people who are pragmatic and knowledgeable about WHY to use certain tech.
Re: The Inner Workings of Distributed Databases
#24When is the right time to "level up" from "I'm good with just plain old Postgres" to QuestDB, InfluxDB, Patroni, etc.? > Unfortunately, automatic failover is solved neither by PostgreSQL nor TimescaleDB, but there are 3rd-party solutions like Patroni that add support for that functionality. PostgreSQL describes the process of failover as STONITH (Shoot The Other Node In The Head), meaning that the primary node has to…
> failover as STONITH (Shoot The Other Node In The Head) What functional consensus protocol doesn't mandate attempted murder? When a node becomes incoherent it can't be relied upon to notice that it has done so and bow out gracefully. Like cancer, there is always a change that 'cell death' will fail and leave you in a pathological state.
Most consensus algorithms assume some subset of possible behaviors from the misbehaving nodes. The algorithms that don't are called "Byzantine" and are a very short list (e.g. the situation where a node can lie and maliciously try to misinform other nodes about the state of the system).
If you can tell that a node failed, there are usually other opportunities for circuit-breaking than shooting it, such as at the hypervisor, load-balancer, or even clients.
Re: The Inner Workings of Distributed Databases
#25Is there any book/textbook course out there that goes through how to write a database or dbms from scratch up to something useful, think something like nand to tetris style? I have been looking but there is not much on this topic out there I feel like.
You can take CMU's 15-445 which is available online with lectures uploaded to youtube: https://15445.courses.cs.cmu.edu/fall2022/.
Re: The Inner Workings of Distributed Databases
#26Is there any book/textbook course out there that goes through how to write a database or dbms from scratch up to something useful, think something like nand to tetris style? I have been looking but there is not much on this topic out there I feel like.
There is a book that is exactly this, "Database Design and Implementation" by Edward Sciore.
You write a database in Java while having the principles explained along the way.
https://www.amazon.com/Database-Design-Implementation-Data-C...
Re: The Inner Workings of Distributed Databases
#27Earlier quoted context omitted.
> I wouldn’t just use any random database because I want to appear cool Obviously! "level up" does not imply you used the wrong thing on purpose and you're switching to another solution that's better in every way, as you seem to have read it.
I’ve met more people who have used the wrong tool for the job than people who are pragmatic and knowledgeable about WHY to use certain tech.
Many people use the wrong tool. I don't disagree with that, but also I think you misinterpreted "level up". If they thought the other tool was unqualified better, they would have used it from the start. The reason to ask about the "right time" is because they already know different situations need different tools.
Re: The Inner Workings of Distributed Databases
#28Is there any book/textbook course out there that goes through how to write a database or dbms from scratch up to something useful, think something like nand to tetris style? I have been looking but there is not much on this topic out there I feel like.
There is a book that is exactly this, "Database Design and Implementation" by Edward Sciore. You write a database in Java while having the principles explained along the way. https://www.amazon.com/Database-Design-Implementation-Data-C...
This looks amazing thank you for the recommendation!