Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

80M I/O Per Second with a Standard 2U System

spdk.io

Post reply on HN
Print view View original

11–20 of 41 posts

Re: 80M I/O Per Second with a Standard 2U System

#11

Post by bullen »

What is the point of that speed if the rest of your system is a bottle neck and the drives will fail in a year if you use that speed continously?

I think we need to go back and look at cycles per bit (DWPD is confusing because you need to know the number of years and sometimes it's not mentioned on the product page).

Complexity can increase some performance but there is always a cost to complexity. Optane seemed promising because they increased the speed (that I personally will never need) and preserved the resiliance.

Time will tell, but I bought 30x new 50nm SLC 64GB X25-E drives from 2011 on ebay for $100 each that I will mount on 12x SATA motherboards (only got cable/power for 8+1 per board) instead of buying one 480GB Optane drive for $1500!

Cheaper, slower and maybe more robust? Atleast it is simpler!

Atleast SATA I can use in my laptops/desktops/consoles etc. Also nobody is looking at CPU usage of NVMe vs. SATA!

Re: 80M I/O Per Second with a Standard 2U System

#12

Post by consp »

bullen wrote: post #11

What is the point of that speed if the rest of your system is a bottle neck and the drives will fail in a year if you use that speed continously? I think we need to go back and look at cycles per bit (DWPD is confusing because you need to know the number of years and sometimes it's not mentioned on the product page). Complexity can increase some performance but there is always a cost to complexity. Optane seemed prom…

I did something similar on a smaller scale: "Caching" ssd drives which are not the best but have a huge amount write bytes left are widely available on ebay and I bought 8 200GB SAS3 drives for dirt cheap (under 200$ total, drives from 2017). Not the fastest drives but good enough for my purpose (overkill even).

Re: 80M I/O Per Second with a Standard 2U System

#13

Post by bullen »

consp wrote: post #12
bullen wrote: post #11

What is the point of that speed if the rest of your system is a bottle neck and the drives will fail in a year if you use that speed continously? I think we need to go back and look at cycles per bit (DWPD is confusing because you need to know the number of years and sometimes it's not mentioned on the product page). Complexity can increase some performance but there is always a cost to complexity. Optane seemed prom…

I did something similar on a smaller scale: "Caching" ssd drives which are not the best but have a huge amount write bytes left are widely available on ebay and I bought 8 200GB SAS3 drives for dirt cheap (under 200$ total, drives from 2017). Not the fastest drives but good enough for my purpose (overkill even).

What drives did you get?

Re: 80M I/O Per Second with a Standard 2U System

#14

Post by gbrown_ »

wmf wrote: post #5
pacetherace wrote: post #4

80M 512 byte I/O per second. That’s 10M 4KB I/O per second. That’s not groundbreaking.

They also did 25M 4K IOPS on the same system. Anyway, it's basically an ad for Intel® Xeon® and Optane®.

> it's basically an ad for Intel® Xeon® and Optane®.

Bingo.

Re: 80M I/O Per Second with a Standard 2U System

#15

Post by user5994461 »

bullen wrote: post #11

What is the point of that speed if the rest of your system is a bottle neck and the drives will fail in a year if you use that speed continously? I think we need to go back and look at cycles per bit (DWPD is confusing because you need to know the number of years and sometimes it's not mentioned on the product page). Complexity can increase some performance but there is always a cost to complexity. Optane seemed prom…

For reference, you can buy a 2 TB SSD for $200 today. Same capacity as the 30 old drives, for much less space and cables.

Re: 80M I/O Per Second with a Standard 2U System

#16

Post by Empf »

bullen wrote: post #11

What is the point of that speed if the rest of your system is a bottle neck and the drives will fail in a year if you use that speed continously? I think we need to go back and look at cycles per bit (DWPD is confusing because you need to know the number of years and sometimes it's not mentioned on the product page). Complexity can increase some performance but there is always a cost to complexity. Optane seemed prom…

I'm lost on your comment.

What specific problem die you see?

It's much cheaper and easier to pay and use NVM than ram. You get best latency for databases across your data.

The prices are already super cheap.

I also not find it easier to buy a lot of small devices instead of one. Your NVM is doing the same thing you do but directly on the controller

Check out the latency difference between sata and NVM.

And yes CPU is idling often enough. Good thing if they have a little bit more io overhead but when you look at PS5 and Xbox, Sony solved this issue and Microsoft is bringing their new storage API to PC.

Re: 80M I/O Per Second with a Standard 2U System

#17

Post by bob1029 »

bullen wrote: post #11

What is the point of that speed if the rest of your system is a bottle neck and the drives will fail in a year if you use that speed continously? I think we need to go back and look at cycles per bit (DWPD is confusing because you need to know the number of years and sometimes it's not mentioned on the product page). Complexity can increase some performance but there is always a cost to complexity. Optane seemed prom…

>I think we need to go back and look at cycles per bit

I've started playing around with new storage architectures for databases and with flash storage in mind. One thing I encountered that seems incredibly powerful, when combined with a log-based store, is micro batching of operations.

When you aggregate a few hundred/thousand transactions together and then append them as modified subtrees to the end of a log, you start to be able to say crazy shit like "Transactions per I/O operation or block"

When working with objects in the 512b range, I can easily hit 2 million inserts per second on a single Samsung 960pro using this technique.

Re: 80M I/O Per Second with a Standard 2U System

#20

Post by Severian »

bullen wrote: post #11

What is the point of that speed if the rest of your system is a bottle neck and the drives will fail in a year if you use that speed continously? I think we need to go back and look at cycles per bit (DWPD is confusing because you need to know the number of years and sometimes it's not mentioned on the product page). Complexity can increase some performance but there is always a cost to complexity. Optane seemed prom…

This rack could be used for storage, not compute. You could add a Mellanox 400Gb NIC to the machine (assuming you have the required 32x PCI lanes for it), and then route that to your compute resource. This is a typical scenario.
Post reply on HN