What everyone should know is that flash drives can lose their data when left unpowered for as little as three months.
Do you have a current source for that?
I've turned on plenty of cell phones that hadn't been charged or powered on for a couple of years and everything worked normally. Same with thumb drives I've picked up after years.
I mean, anything can fail after three months. Your statement doesn't really add anything without stating the failure rates. For all I know the failure rate could be less than that of physical hard drives.
if that is true disks should come with a very visible note stating this... seriously, 3 months would be nothing. i doubt it is true because 3 months is a time frame which should be surpassed quite often making this more known.
Depending on manufacturer, and storage conditions, it can be up to about ten years. But the “three months” number is real: https://web.archive.org/web/20210502042514/http://www.dell.c...
That's a document from nine and a half years ago, and it states:
> It depends on the how much the flash has been used (P/E cycle used), type of flash, and storage
temperature. In MLC and SLC, this can be as low as 3 months and best case can be more than 10
years. The retention is highly dependent on temperature and workload.
Are there any modern sources provide more accurate stats? "3 months to 10 years" is so vague as to be useless.
A little off topic, but I bought a new Macbook Pro with the M1 chip with 8GB of RAM, and I'm worried about the swap usage of this machine wearing out the SSD too quickly. Is this an actual concern, as my swap has been in the multiple GB range with my use?
Why did you get the 8 gig version? If you are using all this swap then your purchased the wrong MacBook.
Honestly, don't run much, so didn't think it would be that bad stepping down from my 16GB machine.
A little off topic, but I bought a new Macbook Pro with the M1 chip with 8GB of RAM, and I'm worried about the swap usage of this machine wearing out the SSD too quickly. Is this an actual concern, as my swap has been in the multiple GB range with my use?
From what I’ve been able to gather, the excessive paging may actually have to do with non-native apps running on the M1. Avoid those.
Most of my programs are JetBrains IDE's and browsers. Don't know if they're optimized for M1.
There's nothing whatsoever I should need to know about SSDs as a Javascript programmer and if there is then the programmers on the lower levels haven't done their jobs right and are wasting my time
What's the flash translation layer made of? Is the flash technology used for that more durable than the rest of the SSD itself? (like say MLC vs. QLC?)
The FTL is like a virtual memory manager. It is firmware/hardware to manage things like the logical to physical mapping table, garbage collection, error correction, bad block management. Yes there will be a lot of FTL data structures stored on the flash. It can be made durable by redundant copies, writing in SLC mode or having recovery algorithms. I used to develop SSD firmware in the past if you have further questio…
Hey that's very interesting! How much of the FTL logic is done with regular MCU code vs custom hardware? Is there any open source SSD firmware out there that one could look at to start experimenting in this field, or at least something pointing in that direction, be it open or affordable software, firmware, FPGA gateway or even IC IP? I believe there is value in integrating that part of the stack with the higher level software, but it seems quite difficult to experiment unless one is in the right circles / close to the right companies. Thanks!
Why would you want to bypass the filesystem by talking to the block device directly? Doesn't O_DIRECT on a preallocated regular file accomplish the same thing with less management complexity and special OS permissions? Granted, the file extents might be fragmented a bit, but that can be fixed.
If you understand your workload and the hardware well enough to understand how doing direct I/O on a file will help - then you’re going to generally do better against a direct block device because there are fewer intermediate layers doing the wrong optimizations or otherwise messing you up. From a pure performance perspective anyway. Extents are one part of the issue, flushes to disk (and how/when they happen), cachi…
With O_DIRECT though you're opting out of the filesystem's caching (well, VFS's), forced flushes, and most FS level optimizations, so I'd expect it to perform on par with direct partition access.
Do you have numbers showing an advantage of going directly to the block device? Personally, I'd consider the management advantages of a filesystem compelling absent specific performance numbers showing the benefit of direct partition access.
Achieving cutting-edge storage performance tends to require bypassing the filesystem anyways. Traditionally, that meant using SPDK. Nowadays, opening /dev/nvme* with O_DIRECT and operating on it with io_uring will get you most of the way there. In either case, the advice given in the article and by the OP is filesystem agnostic.
Will an end user downloading a video editing app (or similar) have a NVME drive, know how to give your app direct access to a NVME drive, and will your app not corrupt the rest of the files on the drive?
Extreme performance requires extreme tradeoffs. As with anything else, you have to evaluate your use cases and determine for yourself whether the tradeoffs are worth it. For a mass-market application that has to play nice with other applications and work with a wide variety of commodity hardware, it's probably not worthwhile. For a state-of-the-art high performance data store that expects low latencies and high throughput (à la ScyllaDB), it may very well be.