Meta-observation: LMDB is a highly-optimized storage engine. However, it is
not a full-featured DBMS (e.g., no support for query optimization). This talk focuses on the potential impact of NVM on different layers of a full-featured DBMS -- not only the storage engine.
> The fact that NVRAM is directly addressable (and thus can bypass the page cache) will eventually play out as irrelevant. It will always be a fact that slower mass-storage will exist, more cheaply than fast in-core storage, persistent or not.
Well, it depends on the latency of the NVM device. If the NVM device is directly addressable and has DRAM-like access latencies, I suspect that we must re-examine the assumptions underlying storage virtualization. Although I agree that the OS/DBMS would still need to maintain a page cache for data residing on slow block-addressable SSD/HDDs, I suspect that this will not be the case for data residing on fast byte-addressable NVM.
> Anything that requires logging or any form of compaction or garbage collection is wasted work. It's completely unnecessary, and that has nothing to do with NVM. Treating NVM DB design as if it's an entirely new and different animal is frankly ignorant. The right design works in all scenarios - as LMDB does.
If you think about it, logging is equivalent to versioning. To ensure atomicity and durability, a DBMS must maintain BEFORE and AFTER images of tuples. Unless it is a pure append-only system, the DBMS cannot refrain from reclaiming the storage space associated with OLDER tuple images. Note that these images can be stored either in the log or in multi-versioned heap storage. Maintaining a second B+tree to keep track of free pages, as is done is LMDB, is simply an implementation detail.