There’s Kvrocks. It uses the Redis protocol and it’s built on RocksDB https://github.com/apache/kvrocks
Ask HN: Why are there no open source NVMe-native key value stores in 2023?
31–40 of 74 posts
Re: Ask HN: Why are there no open source NVMe-native key value stores in 2023?
#32There's actually an NVMe command set which allows you to use the FTL directly as a K/V store. (This is limited to 16-byte keys [1] however, so it is not that useful and probably not implemented anywhere, my guess is Samsung looked at this for some hyperscaler, whipped up a prototype in their customer-specific firmware and the benefits were lesser than expected so it's dead now) [1] These slides claim up to 32 bytes,…
Presumably there is some way to use the hash of the actual key as the key, and then store both key and value as data? 16 bytes is long enough that collisions will be super rare, and while you obviously need to write code to support that case, it should have no performance impact.
[deleted]
Re: Ask HN: Why are there no open source NVMe-native key value stores in 2023?
#33I don't remember exactly why I have any of them saved, but these are some experimental data stores that seems to be fitting what you're looking for somewhat: - https://github.com/DataManagementLab/ScaleStore - "A Fast and Cost-Efficient Storage Engine using DRAM, NVMe, and RDMA" - https://github.com/unum-cloud/udisk ( https://github.com/unum-cloud/ustore ) - "The fastest ACID-transactional persisted Key-Value store d…
See https://www.snia.org/educational-library/key-value-standardi... for some description of the special command set to get an nvme drive to natively work as a key-value store.
Also https://www.snia.org/sites/default/files/ESF/Key-Value-Stora...
Re: Ask HN: Why are there no open source NVMe-native key value stores in 2023?
#34There's actually an NVMe command set which allows you to use the FTL directly as a K/V store. (This is limited to 16-byte keys [1] however, so it is not that useful and probably not implemented anywhere, my guess is Samsung looked at this for some hyperscaler, whipped up a prototype in their customer-specific firmware and the benefits were lesser than expected so it's dead now) [1] These slides claim up to 32 bytes,…
Presumably there is some way to use the hash of the actual key as the key, and then store both key and value as data? 16 bytes is long enough that collisions will be super rare, and while you obviously need to write code to support that case, it should have no performance impact.
A 32-byte key would allow using NVMe KV directly for content-addressed storage; many of those systems use 256-bit / 32-byte cryptographic hashes as keys. Notable exception would be git with 20-byte keys.
Re: Ask HN: Why are there no open source NVMe-native key value stores in 2023?
#35There’s Kvrocks. It uses the Redis protocol and it’s built on RocksDB https://github.com/apache/kvrocks
Does RocksDB speak NVMe directly?
> High-performance storage engines. There are a number of storage engines and key-value stores optimized for flash. RocksDB [36] is based on an LSM-Tree that is optimized for low write amplification (at the cost of higher read amplification). RocksDB was designed for flash storage, but at the time of SATA SSDs, and therefore cannot saturate large NVMe arrays.
From this slightly tangent mention, I am guessing not.
https://web.archive.org/web/20230624195551/https://www.vldb....
Re: Ask HN: Why are there no open source NVMe-native key value stores in 2023?
#36I don't remember exactly why I have any of them saved, but these are some experimental data stores that seems to be fitting what you're looking for somewhat: - https://github.com/DataManagementLab/ScaleStore - "A Fast and Cost-Efficient Storage Engine using DRAM, NVMe, and RDMA" - https://github.com/unum-cloud/udisk ( https://github.com/unum-cloud/ustore ) - "The fastest ACID-transactional persisted Key-Value store d…
See https://www.snia.org/educational-library/key-value-standardi... for some description of the special command set to get an nvme drive to natively work as a key-value store. Also https://www.snia.org/sites/default/files/ESF/Key-Value-Stora...
How is that implemented? Btree, hashtable?
Re: Ask HN: Why are there no open source NVMe-native key value stores in 2023?
#37It becomes complex when you want to support multiple NVMes
Even more complex when you want to have any kind of redundancy, as you'd essentially need to build-in some kind of RAID-like into your database.
Also few terabytes in RAID10 NVMes + PostgreSQL and something covers about 99% of companies needs for speed.
So you're left with 1% needing that kind of speeds
Re: Ask HN: Why are there no open source NVMe-native key value stores in 2023?
#38Earlier quoted context omitted.
Presumably there is some way to use the hash of the actual key as the key, and then store both key and value as data? 16 bytes is long enough that collisions will be super rare, and while you obviously need to write code to support that case, it should have no performance impact.
A 32-byte key would allow using NVMe KV directly for content-addressed storage; many of those systems use 256-bit / 32-byte cryptographic hashes as keys. Notable exception would be git with 20-byte keys.
You can still do this... Just use the first 16 bytes as the key, and the 2nd 16 bytes as the start of the data.
Re: Ask HN: Why are there no open source NVMe-native key value stores in 2023?
#39[deleted]
Re: Ask HN: Why are there no open source NVMe-native key value stores in 2023?
#40Why do you mean by non-embedded?
You might also be interested in xNVMe and the RocksDB/Ceph KV drivers:
https://github.com/OpenMPDK/xNVMe