What every programmer should know about SSDs

Post by CoolGuySteve » Mon, Jun 21, 2021, 2:00 AM UTC

pkaye wrote: post #31

CoolGuySteve wrote: post #7

The claim about parallelism isn't true. Most benchmarks and my own experience show that sequential reads are still significantly faster than random reads on most NVME drives. However, random read performance is only somewhere between a 3rd to half as fast as sequential compared to a magnetic disk where it's often 1/10th as fast.

What kind of queue depth do you test the read performance? The sequential can be made fast at low queue depth by the SSD controller doing prefetch reads internally. I've worked on such algorithms myself.

Post by gruez » Mon, Jun 21, 2021, 2:09 AM UTC

jedberg wrote: post #82

gruez wrote: post #81

Earlier quoted context omitted.

The kernel can't optimize that because sqlite is specifically requesting it to force a write.

Yes but you can configure the kernel to ignore that, and by default it does. For example, way back in the day, to get more life out of my laptop during college, I configured the kernel to only write to disk once an hour or when the buffer filled up. That effectively meant I was only writing to disk once per hour when I shut down to change classes. The modern linux kernel doesn't actually write to disk when fsync is c…

       fsync() transfers ("flushes") all modified in-core data of (i.e.,
       modified buffer cache pages for) the file referred to by the file
       descriptor fd to the disk device (or other permanent storage
       device) so that all changed information can be retrieved even if
       the system crashes or is rebooted.  This includes writing through
       or flushing a disk cache if present.  The call blocks until the
       device reports that the transfer has completed.

Post by ww520 » Mon, Jun 21, 2021, 2:32 AM UTC

bob1029 wrote: post #26

Things I have learned about SSDs: If you want to go fast & save NAND lifetime, use append-only log structures. If you want to go even faster & save even more NAND lifetime, batch your writes in software (i.e. some ring buffer with natural back-pressure mechanism) and then serialize them with a single writer into an append-only log structure. Many newer devices have something like this at the hardware level, but your…

Post by quotemstr » Mon, Jun 21, 2021, 3:27 AM UTC

lazide wrote: post #84

quotemstr wrote: post #68

Earlier quoted context omitted.

With O_DIRECT though you're opting out of the filesystem's caching (well, VFS's), forced flushes, and most FS level optimizations, so I'd expect it to perform on par with direct partition access. Do you have numbers showing an advantage of going directly to the block device? Personally, I'd consider the management advantages of a filesystem compelling absent specific performance numbers showing the benefit of direct…

You do when it does that/respects it which isn’t always. The point is that you have more layers. If you’re trying to be as direct as possible, more layers is unhelpful. Since you get most of the same advantages management wise with lvm while using the block interface (including snapshots, resizing, and all the other management goodies), you’re not exactly getting much extra functionality either.

Post by riobard » Mon, Jun 21, 2021, 3:57 AM UTC

rdc12 wrote: post #105

riobard wrote: post #100

One thing I'm still puzzled about SSD over-provisioning, which is also mentioned by the tutorial ( https://codecapsule.com/2014/02/12/coding-for-ssds-part-4-ad... ) recommended by the article: > A drive can be over-provisioned simply by formatting it to a logical partition capacity smaller than the maximum physical capacity. The remaining space, invisible to the user, will still be visible and used by the SSD control…

The SSD maintains a translation table for all the virtual addresses exposed by the drive, that maps to the underlying flash physical addresses. Any physical address not in that table, is unallocated and the drive can use freely.