Live data from Hacker News

HSE: Heterogeneous-memory storage engine designed for SSDs

github.com

1–10 of 47 posts

Re: HSE: Heterogeneous-memory storage engine designed for SSDs

#4
Looks pretty cool when you make it to the GitHub (https://github.com/hse-project). Order of magnitude performance gains! I imagine most of that come from skipping the Filesystem layer and just hitting the raw Block layer directly.

I am curious about the durability and how well tested all of that is though. On the one hand, filesystems put a lot of work towards ensuring that bytes written to disk and synced are most likely durable, but OTOH Micron is a native SSD vendor so they've probably thought of that.

I'm also curious whether RAIDing multiple SSDs together at the block layer and then running HSE on top of that will be faster or whether running multiple HSE instances (not the right word, it's a library, but you get what I mean) with one per drive and then executing redundantly across instances would be faster. Argument for the former is that each instance would have to redo the management work, argument for the latter is that there's probably synchronization overhead within the library so running more in parallel should allow for concurrency and parallelism gains.

Re: HSE: Heterogeneous-memory storage engine designed for SSDs

#8
"World's first" Open-Source storage engine for SSDs? I believe Aerospike has advertised itself as that for years, and certainly most MongoDB instances are backed by SSD these days. Heck, conceptually etcd is a key-value storage engine built for SSDs.

> HSE optimizes performance and endurance by orchestrating data placement across DRAM and multiple classes of SSDs or other solid-state storage.

Orchestrating data placement? Isn't that what all storage engines do?

What am I missing here? Is this a block level rather than file-system level driver?

Post reply on HN