I hope the claim to being fully async I/O is just a buzz term, as it's no longer supported in SQLite.
Dqlite – High-Availability SQLite
31–40 of 120 posts
Re: Dqlite – High-Availability SQLite
#32I hope the claim to being fully async I/O is just a buzz term, as it's no longer supported in SQLite. https://www.sqlite.org/asyncvfs.html
The very top of that page suggests WAL with PRAGMA synchronous = 0 accomplishes roughly the same objective.
Re: Dqlite – High-Availability SQLite
#33I hope the claim to being fully async I/O is just a buzz term, as it's no longer supported in SQLite. https://www.sqlite.org/asyncvfs.html
The very top of that page suggests WAL with PRAGMA synchronous = 0 accomplishes roughly the same objective.
Based on the following header file, it would seem that they are using this unsupported module.
Re: Dqlite – High-Availability SQLite
#34I'm not brimming with ideas of where this would be a good fit tbh. Is anyone using it or considering it for anything?
Depending on some details, it would be perfect for storing state for a libvirt cluster management tool I'm working on. Concerns are I can read the data on disk using the regular sqlite3 cli tool, and lack of rust bindings.
8 years ago or so I was involved in building a cloud platform. The very first version of the design for keeping the VM and storage allocation metadata synced across the cluster involved syncing sqlite databases (which we moved off once we realised we'd pretty much have to invent something a lot like raft to make it work). If this had existed then, we'd just have picked it off the shelf.
Re: Dqlite – High-Availability SQLite
#35Earlier quoted context omitted.
The very top of that page suggests WAL with PRAGMA synchronous = 0 accomplishes roughly the same objective.
Based on the following header file, it would seem that they are using this unsupported module. https://github.com/canonical/dqlite/blob/master/src/vfs.h
They would have to, wouldn't they? The Sqlite VFS subsystem is the obvious place to intercept the usually local DB and WAL reads/writes and make them distributed functions that use Raft.
Re: Dqlite – High-Availability SQLite
#36Using Raft for the WAL sounds a bit like Calvin. How does Dqlite compare to eg FaunaDB in terms of distributed ACID guarantees?
Re: Dqlite – High-Availability SQLite
#37Earlier quoted context omitted.
The very top of that page suggests WAL with PRAGMA synchronous = 0 accomplishes roughly the same objective.
Based on the following header file, it would seem that they are using this unsupported module. https://github.com/canonical/dqlite/blob/master/src/vfs.h
Yes, you have to use a forked SQLite in order to make use of dqlite.
I believe they (the LXD team) are working on upstreaming the WAL changes but due to SQLite's very strong compatibility guarantees they want to be very certain the API and protocol are correct before carving it in stone. Not to mention they are the only major users of the feature, so more widespread use would also be nice before merging it upstream.
Re: Dqlite – High-Availability SQLite
#38> fully async disk I/O
I thought Linux didn't support real async disk IO. Is that not the case?
If Linux has no real async disk IO, how does Dqlite achieve fully async disk IO?
Re: Dqlite – High-Availability SQLite
#39The one annoying thing about SQLite is that there is no easy way to change the table structure. Adding/Removing/Renaming columns is super complicated and afaik there is no good command line tool that does it for you.
That is the primary reason why I do not consider it for new projects. It's just to slow to iterate on.
Re: Dqlite – High-Availability SQLite
#40The one annoying thing about SQLite is that there is no easy way to change the table structure. Adding/Removing/Renaming columns is super complicated and afaik there is no good command line tool that does it for you. That is the primary reason why I do not consider it for new projects. It's just to slow to iterate on.
> Adding/Removing/Renaming columns is super complicated and afaik there is no good command line tool that does it for you.
sqlite supports ADD COLUMN and RENAME COLUMN DDLs.
Dropping columns is not supported, nor is adding some of the more complex column, that does require going through full table rewriting.