Distributed SQLite for Go applications

Post by strken » Fri, Apr 06, 2018, 9:54 AM UTC

contingencies wrote: post #22

strken wrote: post #9

Earlier quoted context omitted.

It looks like a dependency of lxd, the successor to lxc, which is being developed by Canonical (of Ubuntu fame). You can see how they're using it in https://github.com/lxc/lxd/tree/master/lxd/cluster . Dqlite and rqlite are not in the same space as postgres and mysql, they're more comparable to tools like Zookeeper, etcd, or Consul[0] - low throughput CP data stores that are generally used for coordinating distribute…

Just what the world needs, yet another would-you-like-my-version-of-clustering-with-that virtual environments tool...

Post by hardwaresofton » Fri, Apr 06, 2018, 9:57 AM UTC

free-ekanayaka wrote: post #15

hardwaresofton wrote: post #3

Damn it, I was going to work on exactly this -- my goals were slightly different but basically the same paxos/raft/gossip + sqlite and see how far I could take it. Note for those who are interested in cool-stuff-with-SQLite, there's a similar project called RQLite (mentioned in the README): https://github.com/rqlite/rqlite The main differences are laid out in the README, but IMO it really all boils down to the fact t…

Hello, author of dqlite here. Things are still in flux, but yes, I plan to publish benchmarks before making a 1.0 release, as well as improving documentations and introduce some more abstractions to make it easier to use. The reason it might be slower is some cases is that RQlite replicates statements and dqlite replicates WAL frames (where "frames" roughly means "disk pages"), which are typically bigger. I didn't ru…

Post by thinkersilver » Fri, Apr 06, 2018, 10:08 AM UTC

hardwaresofton wrote: post #3

Damn it, I was going to work on exactly this -- my goals were slightly different but basically the same paxos/raft/gossip + sqlite and see how far I could take it. Note for those who are interested in cool-stuff-with-SQLite, there's a similar project called RQLite (mentioned in the README): https://github.com/rqlite/rqlite The main differences are laid out in the README, but IMO it really all boils down to the fact t…

Post by jxub » Fri, Apr 06, 2018, 10:32 AM UTC

atombender wrote: post #16

It's fascinating how Raft has democratized distributed computing -- writing a consistent, distributed state machine now can be done in a few lines of code, assuming you have a Raft implementation lying around, like Hashicorp's excellent Go library. What this project (and similar projects such as Rqlite) doesn't address is the hard problem -- sharding. Raft makes it quite trivial to write a master/slave system where e…

Post by free-ekanayaka » Fri, Apr 06, 2018, 10:44 AM UTC

hardwaresofton wrote: post #33

free-ekanayaka wrote: post #15

Earlier quoted context omitted.

Hello, author of dqlite here. Things are still in flux, but yes, I plan to publish benchmarks before making a 1.0 release, as well as improving documentations and introduce some more abstractions to make it easier to use. The reason it might be slower is some cases is that RQlite replicates statements and dqlite replicates WAL frames (where "frames" roughly means "disk pages"), which are typically bigger. I didn't ru…

Hey thanks for putting this out there, it's a pretty cool project. I see what you meant by it being slower, but that's only replication speed right? maybe that should be pointed out in the documentation when you find time to update it. Does it seem like the replication patch is going to land in Sqlite? I sure would like it to...