A database for 2022
tailscale.com
A database for 2022
1–10 of 336 posts
Re: A database for 2022
#2They're using litestream [1] to replicate a SQLite database, which streams additions to the SQLite WAL file to object storage and can replay it back. This is fairly hands-off from SQLite's perspective. There's also the SQLite Session Extension [2] that is a built-in way to support generating and applying "patches".
I'm curious how these tools will mature, it seems like a good match for microservices.
Re: A database for 2022
#3They're using litestream [1] to replicate a SQLite database, which streams additions to the SQLite WAL file to object storage and can replay it back. This is fairly hands-off from SQLite's perspective. There's also the SQLite Session Extension [2] that is a built-in way to support generating and applying "patches". I'm curious how these tools will mature, it seems like a good match for microservices. [1]: https://git…
The litestream project was created by https://github.com/benbjohnson who wrote https://github.com/boltdb/bolt (a key value store) which has been instrumental (from my point of view) in the Go community as one of the original choices for an embedded database as it was not only fast, but had transactions with stable snapshots.
It was used by https://github.com/blevesearch/bleve, https://github.com/etcd-io/etcd, and number of other projects.
These days, https://github.com/dgraph-io/badger is often favored because of it's improved write throughput.
Re: A database for 2022
#4They're using litestream [1] to replicate a SQLite database, which streams additions to the SQLite WAL file to object storage and can replay it back. This is fairly hands-off from SQLite's perspective. There's also the SQLite Session Extension [2] that is a built-in way to support generating and applying "patches". I'm curious how these tools will mature, it seems like a good match for microservices. [1]: https://git…
The litestream project was created by https://github.com/benbjohnson who wrote https://github.com/boltdb/bolt (a key value store) which has been instrumental (from my point of view) in the Go community as one of the original choices for an embedded database as it was not only fast, but had transactions with stable snapshots. It was used by https://github.com/blevesearch/bleve , https://github.com/etcd-io/etcd , and n…
It should be noted that BoltDB has been retired as it suffered from a number of congenital defects. It was implicated in the major Roblox outage that happened a couple months ago, and Consul doesn't use it anymore.
https://news.ycombinator.com/item?id=30015913
(This comment is not to suggest that the Litestream project is of poor quality or anything like that.)
Re: A database for 2022
#5They're using litestream [1] to replicate a SQLite database, which streams additions to the SQLite WAL file to object storage and can replay it back. This is fairly hands-off from SQLite's perspective. There's also the SQLite Session Extension [2] that is a built-in way to support generating and applying "patches". I'm curious how these tools will mature, it seems like a good match for microservices. [1]: https://git…
The litestream project was created by https://github.com/benbjohnson who wrote https://github.com/boltdb/bolt (a key value store) which has been instrumental (from my point of view) in the Go community as one of the original choices for an embedded database as it was not only fast, but had transactions with stable snapshots. It was used by https://github.com/blevesearch/bleve , https://github.com/etcd-io/etcd , and n…
Another one of Ben's projects is on the front page right now:
Postgres wire compatible SQLite proxy - https://news.ycombinator.com/item?id=30875837
Re: A database for 2022
#6> This gives you near real time backups (*or with a couple deft modifications, lets your app block at critical sections until the backup is done*)
anyone has an example of how to do this?
Re: A database for 2022
#7I'm so desperate for a SQL database where I can just put it on a bunch of commodity hardware via Docker, connect them up and never worry about this again. Ideally it'd monitor my queries and create indexes for me. pleeeeeeaaaaaaaaaaaassssseeeeeeeeee
/dream
FoundationDB is similar but you have to do so much yourself. It's more or less what I'm describing for a key value store though. Not sure why it's not more popular.
Re: A database for 2022
#8Earlier quoted context omitted.
The litestream project was created by https://github.com/benbjohnson who wrote https://github.com/boltdb/bolt (a key value store) which has been instrumental (from my point of view) in the Go community as one of the original choices for an embedded database as it was not only fast, but had transactions with stable snapshots. It was used by https://github.com/blevesearch/bleve , https://github.com/etcd-io/etcd , and n…
It should be noted that BoltDB has been retired as it suffered from a number of congenital defects. It was implicated in the major Roblox outage that happened a couple months ago, and Consul doesn't use it anymore. https://news.ycombinator.com/item?id=30015913 (This comment is not to suggest that the Litestream project is of poor quality or anything like that.)
BoltDB author here. Just to clarify, the project was retired because of maintenance burden. CoreOS wanted to make some changes but I didn't have the bandwidth to test and maintain it all. Since this was before Go had good version management, we decided that they could fork the project as "bbolt" and users could move over as needed. They did a good job maintaining it so I eventually archived the original project.
Re: A database for 2022
#9They're using litestream [1] to replicate a SQLite database, which streams additions to the SQLite WAL file to object storage and can replay it back. This is fairly hands-off from SQLite's perspective. There's also the SQLite Session Extension [2] that is a built-in way to support generating and applying "patches". I'm curious how these tools will mature, it seems like a good match for microservices. [1]: https://git…
The litestream project was created by https://github.com/benbjohnson who wrote https://github.com/boltdb/bolt (a key value store) which has been instrumental (from my point of view) in the Go community as one of the original choices for an embedded database as it was not only fast, but had transactions with stable snapshots. It was used by https://github.com/blevesearch/bleve , https://github.com/etcd-io/etcd , and n…
Bolt author here. Badger is a good database but it's mostly a tradeoff of write versus read throughput, specifically, range queries. Badger is an LSM so it doesn't perform as well (last time I checked) with iterating over ordered key/value pairs. LSMs have bloom filters to speed up point queries so those aren't as much of an issue.
Re: A database for 2022
#10... this is just replication of a database? As in the full database per node/replica?
Hasn't AWS's SQL db-as-a-services had this for years now?