Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HSE: Heterogeneous-memory storage engine designed for SSDs

github.com

Post reply on HN
Print view View original

21–30 of 47 posts

Re: HSE: Heterogeneous-memory storage engine designed for SSDs

#21

Post by ignoramous »

aloknnikhil wrote: post #12

> https://github.com/hse-project/hse Their benchmarks show significant gains compared to RocksDB. > https://github.com/spdk/rocksdb But what I'd really like to see is a comparison against RocksDB using SPDK > https://dqtibwqq6s6ux.cloudfront.net/download/papers/Hitachi... Based on these results, SPDK performs significantly better than the kernel requiring only 1-2 cores to saturate IOPS on an NVMe SSD (compared to th…

SPDK has had its share of detractors here on news.yc [0], especially with io_uring around the block [1]. It'd be interesting to see the improvements to these io-centric applications once they move to io_uring [2], which, in a way, like RocksDB, is sponsored by Facebook [3].

[0] https://news.ycombinator.com/item?id=10511960

[1] https://news.ycombinator.com/item?id=22266503

[2] https://news.ycombinator.com/item?id=19843464

[3] https://lkml.org/lkml/2014/1/24/252

Re: HSE: Heterogeneous-memory storage engine designed for SSDs

#22

Post by AtlasBarfed »

aloknnikhil wrote: post #12

> https://github.com/hse-project/hse Their benchmarks show significant gains compared to RocksDB. > https://github.com/spdk/rocksdb But what I'd really like to see is a comparison against RocksDB using SPDK > https://dqtibwqq6s6ux.cloudfront.net/download/papers/Hitachi... Based on these results, SPDK performs significantly better than the kernel requiring only 1-2 cores to saturate IOPS on an NVMe SSD (compared to th…

Is there an api compatibility for this thing so it can be switched easily out for things like Rocksandra?

Re: HSE: Heterogeneous-memory storage engine designed for SSDs

#23

Post by organicfigs »

Someone needs to write a book about breaking into writing software like RocksDB, HSE, etc. Years ago I found myself wanting to learn more however going from 0 to 1 felt impossible. Graduated from a T3 school in CS so understanding the concepts wasn't the issue, I just didn't know how to build a good foundation in low latency persistence. Years later I ended up contributing to low latency java which was really interesting, but what an opportunity missed.

Re: HSE: Heterogeneous-memory storage engine designed for SSDs

#24

Post by Rafuino »

ignoramous wrote: post #21
aloknnikhil wrote: post #12

> https://github.com/hse-project/hse Their benchmarks show significant gains compared to RocksDB. > https://github.com/spdk/rocksdb But what I'd really like to see is a comparison against RocksDB using SPDK > https://dqtibwqq6s6ux.cloudfront.net/download/papers/Hitachi... Based on these results, SPDK performs significantly better than the kernel requiring only 1-2 cores to saturate IOPS on an NVMe SSD (compared to th…

SPDK has had its share of detractors here on news.yc [0], especially with io_uring around the block [1]. It'd be interesting to see the improvements to these io-centric applications once they move to io_uring [2], which, in a way, like RocksDB, is sponsored by Facebook [3]. [0] https://news.ycombinator.com/item?id=10511960 [1] https://news.ycombinator.com/item?id=22266503 [2] https://news.ycombinator.com/item?id=1984…

SPDK has made some progress since [0] was posted in 2015 (see https://spdk.io/ if you're curious). Still, I am excited about io_uring and what it brings to the table.

Re: HSE: Heterogeneous-memory storage engine designed for SSDs

#25

Post by benlwalker »

ignoramous wrote: post #21
aloknnikhil wrote: post #12

> https://github.com/hse-project/hse Their benchmarks show significant gains compared to RocksDB. > https://github.com/spdk/rocksdb But what I'd really like to see is a comparison against RocksDB using SPDK > https://dqtibwqq6s6ux.cloudfront.net/download/papers/Hitachi... Based on these results, SPDK performs significantly better than the kernel requiring only 1-2 cores to saturate IOPS on an NVMe SSD (compared to th…

SPDK has had its share of detractors here on news.yc [0], especially with io_uring around the block [1]. It'd be interesting to see the improvements to these io-centric applications once they move to io_uring [2], which, in a way, like RocksDB, is sponsored by Facebook [3]. [0] https://news.ycombinator.com/item?id=10511960 [1] https://news.ycombinator.com/item?id=22266503 [2] https://news.ycombinator.com/item?id=1984…

io_uring is a fantastic development for the kernel, and I really can't praise it enough.

However, there's still lots of reasons to use SPDK. Performance is still significantly better[0], and you can directly access all the of the NVMe features on the device without going through any abstraction layers.

[0] https://spdk.io/news/2019/05/06/nvme/

Re: HSE: Heterogeneous-memory storage engine designed for SSDs

#26

Post by benlwalker »

aloknnikhil wrote: post #12

> https://github.com/hse-project/hse Their benchmarks show significant gains compared to RocksDB. > https://github.com/spdk/rocksdb But what I'd really like to see is a comparison against RocksDB using SPDK > https://dqtibwqq6s6ux.cloudfront.net/download/papers/Hitachi... Based on these results, SPDK performs significantly better than the kernel requiring only 1-2 cores to saturate IOPS on an NVMe SSD (compared to th…

SPDK's RocksDB integration really hasn't gotten a lot of love. There's really two main challenges we hit and then never revisited.

First, the IO threads in RocksDB are a thread pool that assume they perform blocking operations. That doesn't jive with SPDK's async model (nor io_uring's). We're having to message pass to an async thread and block on semaphores in the IO threads.

Second, RocksDB was heavily reliant on the kernel page cache to make it fast.

Both of these things could have changed since we last worked on the integration. I haven't kept up with RocksDB development recently.

source: am SPDK maintainer

Re: HSE: Heterogeneous-memory storage engine designed for SSDs

#27

Post by nn3 »

organicfigs wrote: post #23

Someone needs to write a book about breaking into writing software like RocksDB, HSE, etc. Years ago I found myself wanting to learn more however going from 0 to 1 felt impossible. Graduated from a T3 school in CS so understanding the concepts wasn't the issue, I just didn't know how to build a good foundation in low latency persistence. Years later I ended up contributing to low latency java which was really interes…

"How can I learn an instrument without listening to music" "How can I learn to write stories without reading books?"

Have you considered just reading the code? It's all available. Best way to learn is to look what the masters are doing.

And even complex software is typically just a collection of simple things together.

IMHO one of the biggest failing of many CS courses is that they never get above toy software. Would be much better to at least once dive into some production software and try to figure something out in a real code base.

Re: HSE: Heterogeneous-memory storage engine designed for SSDs

#28

Post by sahil-kang »

organicfigs wrote: post #23

Someone needs to write a book about breaking into writing software like RocksDB, HSE, etc. Years ago I found myself wanting to learn more however going from 0 to 1 felt impossible. Graduated from a T3 school in CS so understanding the concepts wasn't the issue, I just didn't know how to build a good foundation in low latency persistence. Years later I ended up contributing to low latency java which was really interes…

I just finished reading the OSTEP book[1] and it has a nice chapter on SSDs[2]. The entire last portion of the book is about filesystems/disks so you might find it interesting.

[1] http://pages.cs.wisc.edu/~remzi/OSTEP/

[2] http://pages.cs.wisc.edu/~remzi/OSTEP/file-ssd.pdf

Re: HSE: Heterogeneous-memory storage engine designed for SSDs

#29

Post by zerd »

benlwalker wrote: post #25
ignoramous wrote: post #21

Earlier quoted context omitted.

SPDK has had its share of detractors here on news.yc [0], especially with io_uring around the block [1]. It'd be interesting to see the improvements to these io-centric applications once they move to io_uring [2], which, in a way, like RocksDB, is sponsored by Facebook [3]. [0] https://news.ycombinator.com/item?id=10511960 [1] https://news.ycombinator.com/item?id=22266503 [2] https://news.ycombinator.com/item?id=1984…

io_uring is a fantastic development for the kernel, and I really can't praise it enough. However, there's still lots of reasons to use SPDK. Performance is still significantly better[0], and you can directly access all the of the NVMe features on the device without going through any abstraction layers. [0] https://spdk.io/news/2019/05/06/nvme/

Woah, just realizing that 10.39M 4k IOPS is 42GB/s. Doing 40GB/s of sequential IO was difficult not that many years ago, let alone random IO. That's faster than my memory bandwidth. https://www.microway.com/knowledge-center-articles/performan...

Re: HSE: Heterogeneous-memory storage engine designed for SSDs

#30

Post by tzone »

jandrewrogers wrote: post #15

PR copy aside, the claimed performance differences relative to RocksDB and WiredTiger are typical of many storage engines, the performance doesn't stand out. I don't think either RocksDB or WT has made a serious claim to prioritizing performance in their designs in any case. Also, I have to wonder how narrowly "open-source storage engine for SSDs" is being defined here such that it excludes so many earlier storage en…

All storage engines prioritize performance? Back in the day, way before WiredTiger got acquired by MongoDB they also had very similar graphs showing perf differences with RocksDB and InnoDB: https://github.com/wiredtiger/wiredtiger/wiki/Read-scalabili... https://github.com/wiredtiger/wiredtiger/wiki/iiBench-result... https://github.com/wiredtiger/wiredtiger/wiki/YCSB-Mapkeeper...

And of course RocksDB has had similar graphs to showing perf against other systems.

Every system manages to find a benchmark that fits their narrative :)

Reality is both RocksDB and WiredTiger are high performance storage engines, and they are both optimized for SSDs too. These type of benchmarks rarely tell real story.

Post reply on HN