51–60 of 74 posts
Is there any performance gain over writing append-only data to a file? I mean, using a merkle tree or something like that to make sense of the underlying data.
(yes it's fashionable, but it's still terrible for random read performance)
I like how you reference the performance benefits of NVMe direct addressing, but then immediately lament that you can't access these benefits across a SEVEN LAYER STACK OF ABSTRACTIONS.
You can either lament the dearth of userland direct-addressable performant software, OR lament the dearth of convenient network APIs that thrash your cache lines and dramatically increase your access latency.
You don't get to do both simultaneously.
Embedded is a feature for performance-aware software, not a bug.
Earlier quoted context omitted.
Is there not any danger of tenants rewriting the firmware on these drives, and surprising (or compromising) future tenants? AIUI this is the central reason why even "baremetal" cloud instances still have a minimal hypervisor between the tenant and the hardware.
I’m not sure what makes you think an “minimal hypervisor” exists — Oracle Cloud Infrastructure doesn’t have a hypervisor of any sort between you and its .metal instance types. Don’t think Amazon EC2 does either.
Who could afford to develop and maintain such a niche thing, in today’s economy, without either a universal basic income or a “non-free” license to guarantee revenue?
Note that some cloud VM types expose entire NVMe drives as-is directly the guest operating system without hypervisors or other abstractions in the way. The Azure Lv3/Lsv3/Lav3/Lasv3 series all provide this capability, for example. Ref: https://learn.microsoft.com/en-us/azure/virtual-machines/las...
Is there not any danger of tenants rewriting the firmware on these drives, and surprising (or compromising) future tenants? AIUI this is the central reason why even "baremetal" cloud instances still have a minimal hypervisor between the tenant and the hardware.
I don't remember exactly why I have any of them saved, but these are some experimental data stores that seems to be fitting what you're looking for somewhat: - https://github.com/DataManagementLab/ScaleStore - "A Fast and Cost-Efficient Storage Engine using DRAM, NVMe, and RDMA" - https://github.com/unum-cloud/udisk ( https://github.com/unum-cloud/ustore ) - "The fastest ACID-transactional persisted Key-Value store d…
Is there any performance gain over writing append-only data to a file? I mean, using a merkle tree or something like that to make sense of the underlying data.
Writing to append-only files is a terrible idea if you want to query quickly. (yes it's fashionable, but it's still terrible for random read performance)
I mean, what's the trick NVMe can do to be meaningfully faster?
I don't remember exactly why I have any of them saved, but these are some experimental data stores that seems to be fitting what you're looking for somewhat: - https://github.com/DataManagementLab/ScaleStore - "A Fast and Cost-Efficient Storage Engine using DRAM, NVMe, and RDMA" - https://github.com/unum-cloud/udisk ( https://github.com/unum-cloud/ustore ) - "The fastest ACID-transactional persisted Key-Value store d…
See https://www.snia.org/educational-library/key-value-standardi... for some description of the special command set to get an nvme drive to natively work as a key-value store. Also https://www.snia.org/sites/default/files/ESF/Key-Value-Stora...
Earlier quoted context omitted.
Is there not any danger of tenants rewriting the firmware on these drives, and surprising (or compromising) future tenants? AIUI this is the central reason why even "baremetal" cloud instances still have a minimal hypervisor between the tenant and the hardware.
The top clouds (AWS/Azure/Google) have custom firmware to solve this problem. Second-tier clouds probably don't so customers can reflash firmware.