Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

80M I/O Per Second with a Standard 2U System

spdk.io

Post reply on HN
Print view View original

31–40 of 41 posts

Re: 80M I/O Per Second with a Standard 2U System

#31

Post by jnwatson »

gravypod wrote: post #29
bashinator wrote: post #24

Earlier quoted context omitted.

I daydream about running our entire company k8s infra on a single 1U with 256 Epyc threads, a few terabytes of memory, and a bunch more of tiered storage. IMO this kind of miniaturization is amazing for container orchestration. Add a couple more, overprovision by 50%, let k8s handle the high availability, and you’ve got a datacenter in a small box.

I have nightmares of someone convincing my company to do something like that and then a having a hardware, network, or power failure.

Buy 2.

Re: 80M I/O Per Second with a Standard 2U System

#34

Post by SkipperCat »

bashinator wrote: post #24
SkipperCat wrote: post #22

Earlier quoted context omitted.

NVMe will allow your storage to be super fast, but it doesn't add any more compute resources. So unless your K8s cluster was all about IO, i'd say no...

I daydream about running our entire company k8s infra on a single 1U with 256 Epyc threads, a few terabytes of memory, and a bunch more of tiered storage. IMO this kind of miniaturization is amazing for container orchestration. Add a couple more, overprovision by 50%, let k8s handle the high availability, and you’ve got a datacenter in a small box.

We run single node K8s. I do it for some non-prod apps. K8s deployments are a lot more easier to organize than a bunch of docker compose deployments and its also a great way to learn K8s internals, etc.

Re: 80M I/O Per Second with a Standard 2U System

#36

Post by bashinator »

gravypod wrote: post #29
bashinator wrote: post #24

Earlier quoted context omitted.

I daydream about running our entire company k8s infra on a single 1U with 256 Epyc threads, a few terabytes of memory, and a bunch more of tiered storage. IMO this kind of miniaturization is amazing for container orchestration. Add a couple more, overprovision by 50%, let k8s handle the high availability, and you’ve got a datacenter in a small box.

I have nightmares of someone convincing my company to do something like that and then a having a hardware, network, or power failure.

Oh for sure. I'm happy to keep it as a daydream. Cloud providers are better at high-availability than I am.

Re: 80M I/O Per Second with a Standard 2U System

#37

Post by bullen »

tw04 wrote: post #28
bullen wrote: post #11

What is the point of that speed if the rest of your system is a bottle neck and the drives will fail in a year if you use that speed continously? I think we need to go back and look at cycles per bit (DWPD is confusing because you need to know the number of years and sometimes it's not mentioned on the product page). Complexity can increase some performance but there is always a cost to complexity. Optane seemed prom…

>What is the point of that speed if the rest of your system is a bottle neck and the drives will fail in a year if you use that speed continously? >I think we need to go back and look at cycles per bit (DWPD is confusing because you need to know the number of years and sometimes it's not mentioned on the product page). This work is for people developing dedicated storage arrays, not your laptop. Datacenter optane is…

60 DWPD was the original optane development result, but the final product that we can buy is 10 DWPD.

My issue with DWPD is that you can have the firmware complexity pretend the drive is good while losing space... I prefer to have really solid "100.000 writes per bit" 50nm SLC flash memory cells with less magic.

If everyone is measuring NVMe vs. SATA how come I can't find anything interesting when googling? It's only popular articles about the high level difference, "the plug is different"... I would like benchmarks with apples to apples CPU usage while having alot of concurrent data in flight simoultaneously!

Re: 80M I/O Per Second with a Standard 2U System

#38

Post by consp »

bullen wrote: post #13
consp wrote: post #12

Earlier quoted context omitted.

I did something similar on a smaller scale: "Caching" ssd drives which are not the best but have a huge amount write bytes left are widely available on ebay and I bought 8 200GB SAS3 drives for dirt cheap (under 200$ total, drives from 2017). Not the fastest drives but good enough for my purpose (overkill even).

What drives did you get?

Sun/oracle branded HGST SS300 drives (HUSMH4020ASS210), they have the specs in the ballpark of the SS800 drives in endurance (according to the scsi data pages from the disks) but read/write performance of the SS300.

Re: 80M I/O Per Second with a Standard 2U System

#39

Post by bullen »

consp wrote: post #38
bullen wrote: post #13

Earlier quoted context omitted.

What drives did you get?

Sun/oracle branded HGST SS300 drives (HUSMH4020ASS210), they have the specs in the ballpark of the SS800 drives in endurance (according to the scsi data pages from the disks) but read/write performance of the SS300.

Trying to google it it seems unclear to me what the product is... SSD or hybrid? Also what tech (SLC, MLC, TLC, QLC) and how many nm? This is the real reason why I went all in on the X25-E; there is no clear winner since 10 years... either it's super expensive and might be good or cheap and guaranteed crap!

Also the interfaces are all over the place, PCIe, SAS, SATA, NVMe (which is PCIe?), M.2 with double PCIe?! with all kinds of connectors...

We have a serious problem with storage of data when magnetic tape is still the real solution!

Re: 80M I/O Per Second with a Standard 2U System

#40

Post by consp »

bullen wrote: post #39
consp wrote: post #38

Earlier quoted context omitted.

Sun/oracle branded HGST SS300 drives (HUSMH4020ASS210), they have the specs in the ballpark of the SS800 drives in endurance (according to the scsi data pages from the disks) but read/write performance of the SS300.

Trying to google it it seems unclear to me what the product is... SSD or hybrid? Also what tech (SLC, MLC, TLC, QLC) and how many nm? This is the real reason why I went all in on the X25-E; there is no clear winner since 10 years... either it's super expensive and might be good or cheap and guaranteed crap! Also the interfaces are all over the place, PCIe, SAS, SATA, NVMe (which is PCIe?), M.2 with double PCIe?! with…

MLC SSD, Sorry for being unclear, googling it is difficult as they were never released to non DC providers (these are custom for oracle) so data sheets are hard to get (I only have partial data, rest is from the sas inquiry and mode pages). You can deduce some of the specs from the name as they are the same for all hgst drives:

HUS - hgst uktrastar standard

MH - MLC high endurance (4PB is claimed for the 200GB drives, verified this by looking at the lifetime counter and the bytes written)

4020 - max spec 400gb, device is 200gb

A - generation

S - 2.5" formfactor

S2 - 12GB/s (sas3), dual port

1 - probably means custom spec all my hgst former netapp/sun/oracle drives have this set to 1 both SSD and hdd

0 - crypto sanetize

Haven't seen them a lot on eBay last month but caching drives with low percentage writes do show up from time to time in batches.

Post reply on HN