51–60 of 69 posts
It's fascinating how Raft has democratized distributed computing -- writing a consistent, distributed state machine now can be done in a few lines of code, assuming you have a Raft implementation lying around, like Hashicorp's excellent Go library. What this project (and similar projects such as Rqlite) doesn't address is the hard problem -- sharding. Raft makes it quite trivial to write a master/slave system where e…
Earlier quoted context omitted.
No, Raft doesn't deal with the faults, it deals with consensus and can help tolerate certain faults. You then deal with the faults by other means. And even on a fast and reliable local network where Raft can work well, once you have a significant amount of data replacing failing nodes, healing broken records, resyncing, rebalancing all without affecting operations is not trivial and unlikely to be done properly.
> Raft doesn't deal with the faults I'm not sure what this means. The Raft paper explicitly states that Raft is a fault-tolerant system -- and by definition this means it deals with faults. To quote the paper: "Replicated state machines are used to solve a variety of fault tolerance problems in distributed systems." Raft is a type of replicated state machine. To say that "Raft doesn't deal with faults" is not correct…
As far as Google went with their paper; they were incredibly correct by saying that actually implementing a distributed system is hard because of the failure modes; rather than the success modes of the running system.
Earlier quoted context omitted.
> Raft doesn't deal with the faults I'm not sure what this means. The Raft paper explicitly states that Raft is a fault-tolerant system -- and by definition this means it deals with faults. To quote the paper: "Replicated state machines are used to solve a variety of fault tolerance problems in distributed systems." Raft is a type of replicated state machine. To say that "Raft doesn't deal with faults" is not correct…
Well, if you’re being smart - do please tell us how Raft solves the problem of online faulty node replacement: how how does it re-sync the current, up-to-date state to the new node, how does it make sure that when one of the nodes fails - that all of the clients do not overwhelm the rest of the system. As far as Google went with their paper; they were incredibly correct by saying that actually implementing a distribu…
I never said that it did. That is what I mean by my statement that more work is required to build a real system in the real world. Much of the code of rqlite is about cluster management, built on a Raft substrate.
>how how does it re-sync the current, up-to-date state to the new node
I suggest you read the Raft paper, and study the Hashicorp Go implementation (https://github.com/hashicorp/raft) to see how this is done. It's all there.
>As far as Google went with their paper; they were incredibly correct by saying that actually implementing a distributed system is hard because of the failure modes; rather than the success modes of the running system.
I couldn't agree more.
Earlier quoted context omitted.
> Raft doesn't deal with the faults I'm not sure what this means. The Raft paper explicitly states that Raft is a fault-tolerant system -- and by definition this means it deals with faults. To quote the paper: "Replicated state machines are used to solve a variety of fault tolerance problems in distributed systems." Raft is a type of replicated state machine. To say that "Raft doesn't deal with faults" is not correct…
Well, if you’re being smart - do please tell us how Raft solves the problem of online faulty node replacement: how how does it re-sync the current, up-to-date state to the new node, how does it make sure that when one of the nodes fails - that all of the clients do not overwhelm the rest of the system. As far as Google went with their paper; they were incredibly correct by saying that actually implementing a distribu…
- faulty node replacement: you can take any node off (or any node can crash) at any time. As long as there are enough nodes left to reach a quorum, your system will be available. If there are not enough nodes left, your system will be unavailable (but keep consistency)
- re-syncs, snapshots and the rest are all covered in the raft paper
- clients wanting to perform writes always talk to the node that is currently the leader, that node fails, clients will look for next leader (which will be eventually elected as long as there's a quorum of surviving nodes)
- overwhelming a system is a different concern, raft writes are serialized so the goal is usually not throughput (for that you might look at AP/AC storage solutions in the CAP spectrum, raft is CP). Designs that need high write throughput might still use raft as internal building block for coordination.
Earlier quoted context omitted.
> Raft doesn't deal with the faults I'm not sure what this means. The Raft paper explicitly states that Raft is a fault-tolerant system -- and by definition this means it deals with faults. To quote the paper: "Replicated state machines are used to solve a variety of fault tolerance problems in distributed systems." Raft is a type of replicated state machine. To say that "Raft doesn't deal with faults" is not correct…
Well, if you’re being smart - do please tell us how Raft solves the problem of online faulty node replacement: how how does it re-sync the current, up-to-date state to the new node, how does it make sure that when one of the nodes fails - that all of the clients do not overwhelm the rest of the system. As far as Google went with their paper; they were incredibly correct by saying that actually implementing a distribu…
it is a core feature of the protocol.
> how does it make sure that when one of the nodes fails - that all of the clients do not overwhelm the rest of the system.
for write access (proposals), in a typical implementation (defined below [1]), one failed nodes actually slightly speed up the system with the cost of reduced reliability. Consider a typical setup with 3 nodes, the leader normally replicate state to 2 followers, the replication cost get cut in half once your cluster loses one member.
given that you can implement linearizable read by going through the write procedure, one can argue the above "speed up" can obviously be achieved on reads as well.
[1] independent replication to followers, for N followers, the same entry will be serialised and sent N times by the leader.
again - just talking about typical implementation and the described "speed up" comes with degraded reliability.
It's fascinating how Raft has democratized distributed computing -- writing a consistent, distributed state machine now can be done in a few lines of code, assuming you have a Raft implementation lying around, like Hashicorp's excellent Go library. What this project (and similar projects such as Rqlite) doesn't address is the hard problem -- sharding. Raft makes it quite trivial to write a master/slave system where e…
is raft a fancy name for distinguished leader multipaxos?
How does it compare to RocksDB? edit: I'm sorry, I was thinking of BedrockDB.
BedrockDB requires to operate a separate process, whereas you can embed dqlite in your Go application (pretty much in the SQLite philosophy). Also, afaik BedrockDB patches upstream SQLite with some more intrusive changes than dqlite (e.g. for supporting concurrent writes). The SQLite patch that dqlite requires is pretty minimal and just adds hooks to internal WAL events.
It's fascinating how Raft has democratized distributed computing -- writing a consistent, distributed state machine now can be done in a few lines of code, assuming you have a Raft implementation lying around, like Hashicorp's excellent Go library. What this project (and similar projects such as Rqlite) doesn't address is the hard problem -- sharding. Raft makes it quite trivial to write a master/slave system where e…
> It's fascinating how Raft has democratized distributed computing First I thought you probably ment commoditized but then I realized how democratized actually applies perfectly well to consensus algorithms because by definition they apply decisions via voting by all participants :) But anyways, I agree very much with you that Raft has transformed the industry and we can't be thankful enough for it. But like you said…
FWIW, the CockroachDB people developed a modified version they call MultiRaft [1] that deals with the scaling challenges they have around shards.
It's fascinating how Raft has democratized distributed computing -- writing a consistent, distributed state machine now can be done in a few lines of code, assuming you have a Raft implementation lying around, like Hashicorp's excellent Go library. What this project (and similar projects such as Rqlite) doesn't address is the hard problem -- sharding. Raft makes it quite trivial to write a master/slave system where e…
Check out http://www.actordb.com/ for a massively sharded sqlite system.
Earlier quoted context omitted.
Check out http://www.actordb.com/ for a massively sharded sqlite system.
ActorDB is very interesting. It uses two-phase commit, similar to CockroachDB, TiDB and Spanner, but as I understand it, relies on locking to update the participating shards. I'm not sure how well ActorDB performs in a large cluster with lots of distributed transactions, but I've never tried it.