Distributed SQLite for Go applications
github.com
Distributed SQLite for Go applications
1–10 of 69 posts
Re: Distributed SQLite for Go applications
#2After looking through the repo, I'm still not sure exactly why you would want / need this. What exactly is the use-case here?
Re: Distributed SQLite for Go applications
#3Damn it, I was going to work on exactly this -- my goals were slightly different but basically the same paxos/raft/gossip + sqlite and see how far I could take it.
Note for those who are interested in cool-stuff-with-SQLite, there's a similar project called RQLite (mentioned in the README):
https://github.com/rqlite/rqlite
The main differences are laid out in the README, but IMO it really all boils down to the fact that since it's single-writer quorum'd there's only one node actually doing writes, and they've just chosen to replicate right-before (right after?) the WAL commit (I think this is what they're calling frames, basically one chunk of the WAL content), not at the point of receiving a query. This is more like having a read secondary more than anything (I also assume writes are redirected to master).
It says in the readme that it should be expected to be slower than RQlite, I'd love to see some numbers.
I do really like this though -- can't wait to pour through the code and see if there's anything I can learn.
Re: Distributed SQLite for Go applications
#4How does it compare to RocksDB?
edit: I'm sorry, I was thinking of BedrockDB.
Re: Distributed SQLite for Go applications
#5Projects like this(sparsely documented, tiny) always seem to be just internal tools that the company just threw out into the world just because they can. Not that I'm ungrateful, but without any sort of explanation as to why I'd use this over a clustered MySQL or Postgres it's a little hard to make heads or tails about whether I should care.
Re: Distributed SQLite for Go applications
#6Projects like this(sparsely documented, tiny) always seem to be just internal tools that the company just threw out into the world just because they can. Not that I'm ungrateful, but without any sort of explanation as to why I'd use this over a clustered MySQL or Postgres it's a little hard to make heads or tails about whether I should care.
You would probably integrate this into a self contained application binary.
Re: Distributed SQLite for Go applications
#7Damn it, I was going to work on exactly this -- my goals were slightly different but basically the same paxos/raft/gossip + sqlite and see how far I could take it. Note for those who are interested in cool-stuff-with-SQLite, there's a similar project called RQLite (mentioned in the README): https://github.com/rqlite/rqlite The main differences are laid out in the README, but IMO it really all boils down to the fact t…
Just because somethings already been done, doesn’t mean you can do it too, or even better.
Re: Distributed SQLite for Go applications
#8Damn it, I was going to work on exactly this -- my goals were slightly different but basically the same paxos/raft/gossip + sqlite and see how far I could take it. Note for those who are interested in cool-stuff-with-SQLite, there's a similar project called RQLite (mentioned in the README): https://github.com/rqlite/rqlite The main differences are laid out in the README, but IMO it really all boils down to the fact t…
Just because somethings already been done, doesn’t mean you can do it too, or even better.
Yeah, but it's probably not smart to re-invent the wheel too many times. Especially when the projects that came before don't look badly written or unreasonable.
I think I do have something to offer -- mostly trying gossip and making use of aggressive automatic sharding -- but I'm also more interested in what I wanted to build on top of distributed SQLite as well. No need to get down in the weeds if someone's already done the hard work for me :)
Re: Distributed SQLite for Go applications
#9Projects like this(sparsely documented, tiny) always seem to be just internal tools that the company just threw out into the world just because they can. Not that I'm ungrateful, but without any sort of explanation as to why I'd use this over a clustered MySQL or Postgres it's a little hard to make heads or tails about whether I should care.
It looks like a dependency of lxd, the successor to lxc, which is being developed by Canonical (of Ubuntu fame). You can see how they're using it in https://github.com/lxc/lxd/tree/master/lxd/cluster.
Dqlite and rqlite are not in the same space as postgres and mysql, they're more comparable to tools like Zookeeper, etcd, or Consul[0] - low throughput CP data stores that are generally used for coordinating distributed systems. In this case, it saves users from having to manage an external data store to run lxd clustered.
[0] in fact, dqlite uses the same raft implementation as Consul, which is cool
Re: Distributed SQLite for Go applications
#10How does it compare to RocksDB? edit: I'm sorry, I was thinking of BedrockDB.
RocksDB is a storage engine (a non-distriuted ordered Key-Value store with transactions). SQLite is a non-distributed SQL database that uses a storage engine (see SQLite4). The linked tool here makes SQLite distributed.