NVMe, the fast future for SSDs • Hacker News Board

Re: NVMe, the fast future for SSDs

#31

Post by hmottestad » Sun, Apr 05, 2015, 7:48 PM UTC

Veratyr wrote: post #26

ziedaniel1 wrote: post #10

The numbers in the article are all wrong. SATA's ceiling is 600 MBps (megabytes per second), not 600 Gpbs (gigabits per second). SAS goes up to 12Gbps, not 12GBps, which is actually the same thing as 1.5GBps. At least the PCIe numbers look right.

The author seems to be mixing these around randomly without really knowing what they mean. Another example: "While a single SATA port is limited to 600Gbps, combining four makes for 2.4GBps of bandwidth." 600Gbps * 4 = 2400Gbps = 3GBps Maybe he thinks a byte is 10 bits or something? It's also really odd that he's using Bps at all. I've never seen MBps anywhere other than this article. Usually it's Mbps and MB/s.

2400Gbps = 3GBps

going from "b" to "B" is either 8 or 10 fold. As some of the other comments have noted. SATA uses 10 bits per byte.

so 2400 Gbps = 300 GBps (8 bit)

or 2400 Gbps = 240 GBps (10 bit)

Re: NVMe, the fast future for SSDs

#32

Post by wtallis » Sun, Apr 05, 2015, 7:50 PM UTC

Veratyr wrote: post #26

ziedaniel1 wrote: post #10

The numbers in the article are all wrong. SATA's ceiling is 600 MBps (megabytes per second), not 600 Gpbs (gigabits per second). SAS goes up to 12Gbps, not 12GBps, which is actually the same thing as 1.5GBps. At least the PCIe numbers look right.

The author seems to be mixing these around randomly without really knowing what they mean. Another example: "While a single SATA port is limited to 600Gbps, combining four makes for 2.4GBps of bandwidth." 600Gbps * 4 = 2400Gbps = 3GBps Maybe he thinks a byte is 10 bits or something? It's also really odd that he's using Bps at all. I've never seen MBps anywhere other than this article. Usually it's Mbps and MB/s.

Since SATA uses 8b/10b encoding, the data rate in bytes/s is actually 1/10th of the raw bit rate. The same applies to earlier PCIe versions. 6Gbps SATA can transfer 600MB/s (ignoring protocol overhead).

I don't really know why it's become standard to publish raw bit rates in bit/s and data rates in bytes/s with error correction taken into account but not protocol overhead, but those are the two kinds of numbers you almost always see quoted nowadays. Raw bit rates at least map pretty directly to clock speed, and I guess protocol overhead must be too variable and too complicated for most people to bother explaining.

Re: NVMe, the fast future for SSDs

#33

Post by rosser » Sun, Apr 05, 2015, 7:57 PM UTC

Veratyr wrote: post #26

ziedaniel1 wrote: post #10

The numbers in the article are all wrong. SATA's ceiling is 600 MBps (megabytes per second), not 600 Gpbs (gigabits per second). SAS goes up to 12Gbps, not 12GBps, which is actually the same thing as 1.5GBps. At least the PCIe numbers look right.

The author seems to be mixing these around randomly without really knowing what they mean. Another example: "While a single SATA port is limited to 600Gbps, combining four makes for 2.4GBps of bandwidth." 600Gbps * 4 = 2400Gbps = 3GBps Maybe he thinks a byte is 10 bits or something? It's also really odd that he's using Bps at all. I've never seen MBps anywhere other than this article. Usually it's Mbps and MB/s.

PCI-Express, SAS, SATA, and many other protocols use an 8b/10b encoding — encoding 8-bit bytes in 10-bit words.

https://en.wikipedia.org/wiki/8b/10b_encoding

Re: NVMe, the fast future for SSDs

#34

Post by justinsb » Sun, Apr 05, 2015, 9:05 PM UTC

skrause wrote: post #21

nqzero wrote: post #19

one of the fundamental problem with SSDs is the impedance mismatch introduced by emulating HDDs. NVMe doesn't appear to help with that at all we need an interface that allows us to bypass the FTL and access the underlying erase blocks

Is there actually any evidence that this would improve performance significantly? Removing the translation layer means that every OS and their file systems have to do good wear leveling because otherwise you'll destroy blocks quickly.

I think this is the idea behind FusionIO: doing the translation on the host CPU is faster, and allows you to expose additional commands (like atomic writes, or direct key-value interfaces).

Re: NVMe, the fast future for SSDs

#35

Post by ccleve » Sun, Apr 05, 2015, 9:41 PM UTC

So the important question is, how does the affect how we write applications?

Is the api different, or are we still reading/writing disk files?

Should we do memory mapping of the files or not?

Should we parallelize access to different sections of big files? Or write a ton of small files?

How does this affect database design? Current big data apps emphasize large append-only writes and large sequential reads (think LSM trees). Does this make sense any more?

What does disk caching mean in the context of these new drives?

Re: NVMe, the fast future for SSDs

#36

Post by MCRed » Sun, Apr 05, 2015, 10:13 PM UTC

Why not simply make an SSD Controller that has a thunderbolt port? Since Intel is building thunderbolt into its support chips now, this seems like a good way to get quick performance, plenty of bandwidth without having to come up with new standard, separate drivers, etc. Thunderbolt ports could be put on motherboards fairly easily, etc.

Is there something I'm missing?

Plus this would have the advantage of driving down prices for thunderbolt and increasing adoption.

Re: NVMe, the fast future for SSDs

#37

Post by mrmondo » Sun, Apr 05, 2015, 10:43 PM UTC

NVMe is one of the most important changes to storage over the past decade.

I'm currently replacing all our SANs with storage servers filled with NVMe SSDs (as the tier 1 storage, commodity SATA SSDs for second tier). I've posted the link to my first blog post about it in the comments on another NVMe post in the past: http://smcleod.net/building-a-high-performance-ssd-san/

I'm close to writing the next post around the actual build, my findings, benchmarks etc... Hopefully I'll have that done next week - but the system comes first.

I'm a little disappointed with this article as I think it could do with a) some technical review and b) some more detailed information.

Re: NVMe, the fast future for SSDs

#38

Post by pjc50 » Sun, Apr 05, 2015, 10:57 PM UTC

ccleve wrote: post #35

So the important question is, how does the affect how we write applications? Is the api different, or are we still reading/writing disk files? Should we do memory mapping of the files or not? Should we parallelize access to different sections of big files? Or write a ton of small files? How does this affect database design? Current big data apps emphasize large append-only writes and large sequential reads (think LSM…

API is the same. Memory map if you prefer that access style and it suits your OS/language preferences. The OS will almost certainly not let you map straight across into the device's PCI memory mapped window so you'll incur a copy to userspace penalty either way. Benchmark. There is probably no longer any advantage to a sequential write, but you still have a per-syscall and per-IOP overhead, so large writes will be faster than N small ones. Disk caching is still there and still decreases latency but isn't so critical.

Re: NVMe, the fast future for SSDs

#39

Post by pjc50 » Sun, Apr 05, 2015, 10:58 PM UTC

MCRed wrote: post #36

Why not simply make an SSD Controller that has a thunderbolt port? Since Intel is building thunderbolt into its support chips now, this seems like a good way to get quick performance, plenty of bandwidth without having to come up with new standard, separate drivers, etc. Thunderbolt ports could be put on motherboards fairly easily, etc. Is there something I'm missing? Plus this would have the advantage of driving dow…

Why Thunderbolt? Its license fees are the main barrier.

Re: NVMe, the fast future for SSDs

#40

Post by eurleif » Sun, Apr 05, 2015, 11:02 PM UTC

MCRed wrote: post #36

Why not simply make an SSD Controller that has a thunderbolt port? Since Intel is building thunderbolt into its support chips now, this seems like a good way to get quick performance, plenty of bandwidth without having to come up with new standard, separate drivers, etc. Thunderbolt ports could be put on motherboards fairly easily, etc. Is there something I'm missing? Plus this would have the advantage of driving dow…

Isn't Thunderbolt just externalized PCI Express?