Earlier quoted context omitted.
You misinterpreted, I think. The point isn’t “snapshot isolation” in the MVCC sense (working with multiple snapshots-in-progress); it’s the ability to, in est, work with the database the way git works with commits: opening a transaction “on top of” a base commit, then “committing” that transaction to create a new commit object, with its own explicit ref, where you can later “check out” an arbitrary ref. Except, unlik…
It seems like coupling a database checkpoint process with file system’s snapshot process should be theoretically possible: 1) Database informed snapshot needed 2) Database finalizes any in progress writes and starts logging new writes to another file 3) Take file system snapshot 4) Inform database snapshot is fine Between #3 and the file system snapshot, you should have a perfect and quick representation of the datab…
A system that did its snapshots “inline” to itself, by e.g. managing a pool of pages with a free-list the way LMDB does — but where Txs ultimately add a new version to the root bucket as a snapshot, rather than replacing the root bucket page with themselves — would get a lot closer to allowing one to have at least tens-of-millions of snapshots online. At that point, to achieve a billion snapshots online, you “only” need to shard your timeline across a cluster of 100 nodes.
This is precisely one of the experiments I’m trying. :)