Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing disks: Lessons from our odyssey selecting replacement SSDs

bbc.co.uk

Post reply on HN
Print view View original

41–50 of 78 posts

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#41

Post by wtallis »

masklinn wrote: post #40
nickcw wrote: post #26

This is the problem IMHO > We also looked up whether our HBA used TRIM in its current configuration. It turns out, in RAID mode, the HBA did not support TRIM. We did do some trim-enabled testing with a different machine, but these results are hard to compare fairly. In any case, we can't currently enable TRIM on our production systems. In our experience SSD write performance goes to sh*t if you don't regularly TRIM t…

> In our experience SSD write performance goes to sh t if you don't regularly TRIM them. Interesting, is that because of the load? It seemed "modern" SSDs have GCs good enough that trim isn't quite necessary anymore to ensure good performances in consumer loads. > RAID cards not passing TRIM is a big problem for us too... Are there NVMe RAID cards? I assume they'd necessarily pass the command along considering deallo…

Modern consumer SSDs still benefit from TRIM, but are mostly able to keep up with GC just fine without it when subjected to typical consumer IO workloads, which are full of idle time for the drive to catch up. But if you fill an SSD to the brim, it'll slow down, and the cheapest SSDs will slow down a lot.

There are no hardware RAID solutions for NVMe, though there are now several hardware platforms supporting software RAID for NVMe devices in their motherboard firmware so you can boot from a NVMe RAID array. As with any other RAID solution, translating trim/unmap/deallocate commands takes a bit of effort, and less mature NVMe RAID solutions don't necessarily bother.

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#42

Post by olavgg »

barrkel wrote: post #37
olavgg wrote: post #23

Earlier quoted context omitted.

I also have a Samsung 960 Evo. Its performance is what I consider a joke, fio and pg_test_fsync make it almost look as slow as spinning SAS drives. For example on a 4kb sync write with 16 threads test, the 960 Evo cannot do more than 1000 iops. In comparison the Intel P4800X (Optane) does friggin 500 000 iops on the same test. That is a 500X difference. https://forums.servethehome.com/index.php?threads/did-some-w...

It is nowhere near as slow as spinning drives, that's ludicrous. Mega IOPS are simply not required in a desktop. I'm not trying to run multiple VMs with multiple databases on this thing. In fact it's rarely writing at all.

Have you checked your SMART data for how much your read/write ratio is? I think you will find those results surprising, even when you think that you don't write that often.

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#43

Post by wtallis »

olavgg wrote: post #42
barrkel wrote: post #37

Earlier quoted context omitted.

It is nowhere near as slow as spinning drives, that's ludicrous. Mega IOPS are simply not required in a desktop. I'm not trying to run multiple VMs with multiple databases on this thing. In fact it's rarely writing at all.

Have you checked your SMART data for how much your read/write ratio is? I think you will find those results surprising, even when you think that you don't write that often.

It's not just a matter of how many bytes are being written; desktop workloads rarely need to do synchronous writes.

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#44

Post by HarryHirsch »

Flash memory has three operations, read, write and erase, the last two destructively. If you pretend they are harddisks with two operations of read and write you go through all sorts of contortions. Sometimes you fall flat on the face, as seen here.

Why don't operating systems treat SSDs more flash memory, and why doesn't the file system cooperate with the underlying hardware instead of pretending it's a disk? For home use that may even work, but in a demanding environment the extra complexity will invariably fail.

This is a genuine question, I'm an amateur here.

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#45

Post by wtallis »

HarryHirsch wrote: post #44

Flash memory has three operations, read, write and erase, the last two destructively. If you pretend they are harddisks with two operations of read and write you go through all sorts of contortions. Sometimes you fall flat on the face, as seen here. Why don't operating systems treat SSDs more flash memory, and why doesn't the file system cooperate with the underlying hardware instead of pretending it's a disk? For ho…

There is some work on Open-Channel SSDs, that move most of the flash translation layer (FTL) to the host system. There are two major problems with this approach:

1. Each OS that wants to use the drive needs a compatible implementation of the FTL. Consumer systems always have at least two operating systems in play (UEFI counts for these purposes). Enterprise systems are where you will actually find non-boot data-only drives.

2. Flash memory changes. The FTL needs very different parameters depending on whether you're using Toshiba flash or Samsung flash, and even depending on whether you're using last year's Toshiba flash or the stuff they're manufacturing today.

These aren't insurmountable problems, but they're enough to keep such products confined to a small niche. Instead, we're seeing a trend of SSDs accepting optional hints that allow them to perform the kinds of optimizations you'd expect from a fully host-managed SSD. The ATA TRIM command was just the tip of this iceberg.

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#46

Post by _urga »

wtallis wrote: post #45
HarryHirsch wrote: post #44

Flash memory has three operations, read, write and erase, the last two destructively. If you pretend they are harddisks with two operations of read and write you go through all sorts of contortions. Sometimes you fall flat on the face, as seen here. Why don't operating systems treat SSDs more flash memory, and why doesn't the file system cooperate with the underlying hardware instead of pretending it's a disk? For ho…

There is some work on Open-Channel SSDs, that move most of the flash translation layer (FTL) to the host system. There are two major problems with this approach: 1. Each OS that wants to use the drive needs a compatible implementation of the FTL. Consumer systems always have at least two operating systems in play (UEFI counts for these purposes). Enterprise systems are where you will actually find non-boot data-only…

Could you provide more details on these hints? Are they ioctl calls? Assuming one is using the disk as a raw block device, without a filesystem.

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#47

Post by olavgg »

wtallis wrote: post #43
olavgg wrote: post #42

Earlier quoted context omitted.

Have you checked your SMART data for how much your read/write ratio is? I think you will find those results surprising, even when you think that you don't write that often.

It's not just a matter of how many bytes are being written; desktop workloads rarely need to do synchronous writes.

There are a lot of desktop applications that does synchronous all the time. Chrome, Firefox and Spotify are a few examples as they use SQLite which does fsync() system call. But yeah, you will be fine even with a spinning hard drive. My point is, what use case is there for paying a premium for the Samsung NVMe's when you would be just fine with a cheaper model.

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#48

Post by gaius »

fjsolwmv wrote: post #35
gaius wrote: post #19

Earlier quoted context omitted.

The constant favourable coverage of Google and Apple, always talking about hipster-friendly Flickr when boring old Photobucket was doing 20x the volume... they apply their rules very selectively....

Photobucket was for link sharing to other sites. , no? While flicker was a destination for publishing albums and browsing, with much higher quality photos.

That's imgur you're thinking of

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#49

Post by DSMan195276 »

HarryHirsch wrote: post #44

Flash memory has three operations, read, write and erase, the last two destructively. If you pretend they are harddisks with two operations of read and write you go through all sorts of contortions. Sometimes you fall flat on the face, as seen here. Why don't operating systems treat SSDs more flash memory, and why doesn't the file system cooperate with the underlying hardware instead of pretending it's a disk? For ho…

> Why don't operating systems treat SSDs as flash memory, and why doesn't the file system cooperate with the underlying hardware instead of pretending it's a disk? For home use that may even work, but in a demanding environment the extra complexity will invariably fail.

The simple reason is because the SSDs themselves expose a regular HD interface and then does a lot of the flash-memory related stuff itself. For example, if you don't include TRIM support (Which early SSDs did not have) there is no 'erase' command the OS can send to an SSD.

With that in mind, SSDs also have memory controllers on them that map the blocks the OS sees to actual SSD blocks (scattered across the memory chips). So when the OS writes to block 1 it may write to block 15 internally on the SSD, and then block 2 might write to block 4002. Combine this with caching and other various details on the SSD side, and it leaves little predictable behavior for the OS to exploit.

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#50

Post by olavgg »

wtallis wrote: post #30
olavgg wrote: post #23

Earlier quoted context omitted.

I also have a Samsung 960 Evo. Its performance is what I consider a joke, fio and pg_test_fsync make it almost look as slow as spinning SAS drives. For example on a 4kb sync write with 16 threads test, the 960 Evo cannot do more than 1000 iops. In comparison the Intel P4800X (Optane) does friggin 500 000 iops on the same test. That is a 500X difference. https://forums.servethehome.com/index.php?threads/did-some-w...

The 960 EVO is a consumer grade SSD with firmware tuned for bursts of I/O (through eg. the use of SLC write caching) at the expense of sustained write throughput. It doesn't have power loss protection capacitors, so it can't perform safe write caching when you're issuing synchronous writes. 4kB is much smaller than the underlying page size of its NAND flash, so performance is going to suck without write combining. Yo…

So what is the use case for this drive? The 960 Evo/Pro are supposed to be premium models, but a better investment would be a cheaper SSD drive with more storage. And if you rarely write that much, more ram will increase the read speed significantly.
Post reply on HN