61–70 of 74 posts
Earlier quoted context omitted.
The top clouds (AWS/Azure/Google) have custom firmware to solve this problem. Second-tier clouds probably don't so customers can reflash firmware.
If your second sentence is true -- and I hope it isn't! -- that would be a gaping security hole.
Earlier quoted context omitted.
Writing to append-only files is a terrible idea if you want to query quickly. (yes it's fashionable, but it's still terrible for random read performance)
Care to elaborate? How is reading from an append-only file backed by a memory indexed DB slower compared to either 1) a mutated file, or 2) either append-only or mutated raw NVMe disk storage? I mean, what's the trick NVMe can do to be meaningfully faster?
But seriously, I've been thinking about an append-only files + memory indexed DB for the past couple of weeks - any prior art or links or papers or anything, lay it on me.
Earlier quoted context omitted.
Care to elaborate? How is reading from an append-only file backed by a memory indexed DB slower compared to either 1) a mutated file, or 2) either append-only or mutated raw NVMe disk storage? I mean, what's the trick NVMe can do to be meaningfully faster?
Your views are intriguing and I wish to subscribe your news letter. But seriously, I've been thinking about an append-only files + memory indexed DB for the past couple of weeks - any prior art or links or papers or anything, lay it on me.
Write speed wasn't my primary motivation though. I wanted a data storage solution that is hard to fuck up. Hard to beat append only in this regard. Plus everything is stored in merkle trees like in Git, so there is the added benefit of data integrity checks. Yes, bit rot is real, and I love to have a mechanism in place to detect and fix those.
I don't remember exactly why I have any of them saved, but these are some experimental data stores that seems to be fitting what you're looking for somewhat: - https://github.com/DataManagementLab/ScaleStore - "A Fast and Cost-Efficient Storage Engine using DRAM, NVMe, and RDMA" - https://github.com/unum-cloud/udisk ( https://github.com/unum-cloud/ustore ) - "The fastest ACID-transactional persisted Key-Value store d…
See https://www.snia.org/educational-library/key-value-standardi... for some description of the special command set to get an nvme drive to natively work as a key-value store. Also https://www.snia.org/sites/default/files/ESF/Key-Value-Stora...
Samsung's uNVMe evaluation guide (from 2019) device support section just states:
Guide Version: uNVMe2.0 SDK Evaluation Guide ver 1.2
Supported Product(s): NVMe SSD (Block/KV)
Interface(s): NVMe 1.2
I can't find detailed spec sheets detailing which NVMe command sets are supported even for their enterprise drives.
https://github.com/aerospike/aerospike-server/blob/master/cf...
There are other occurrences in the codebase, but that is the most prominent one.
> NVMe SSDs based on flash are cheap and offer high throughput. Combining several of these devices into a single server enables 10 million I/O operations per second or more. Our experiments show that existing out-of-memory database systems and storage engines achieve only a fraction of this performance. In this work, we demonstrate that it is possible to close the performance gap between hardware and software through an I/O optimized storage engine design. In a heavy out-of-memory setting, where the dataset is 10 times larger than main memory, our system can achieve more than 1 million TPC-C transactions per second.
Earlier quoted context omitted.
If your second sentence is true -- and I hope it isn't! -- that would be a gaping security hole.
To be fair, some of the bare metal providers reflash firmware when the machine is reprovisioned. In theory firmware "implants" could survive reflashing but I don't know if such a thing has ever been seen in the wild.
Naive question: are there really expected gains to address natively an NVMe disk wrt using a regular key-value database on a filesystem ?
Latency ought to be much better, since you're skipping several abstraction layers in the kernel. But that's about it. And the latency is still worse than in-memory solutions. Between that and the non-trivial effort needed to make this work in any sort of cloud setup (be it self-hosted k8s or AWS), it's a hard sell. If I really need latency above all, AWS gives me instances with 24TB RAM, and if I don't… why not just…
Note that some cloud VM types expose entire NVMe drives as-is directly the guest operating system without hypervisors or other abstractions in the way. The Azure Lv3/Lsv3/Lav3/Lasv3 series all provide this capability, for example. Ref: https://learn.microsoft.com/en-us/azure/virtual-machines/las...
Is there not any danger of tenants rewriting the firmware on these drives, and surprising (or compromising) future tenants? AIUI this is the central reason why even "baremetal" cloud instances still have a minimal hypervisor between the tenant and the hardware.