Why I'm usually unnerved when modern SSDs die on us • Hacker News Board

Re: Why I'm usually unnerved when modern SSDs die on us

#61

Post by DoctorOetker » Tue, Dec 11, 2018, 5:06 PM UTC

cyphar wrote: post #43

niftich wrote: post #21

Not that spinning HDDs are really any different, but SSDs are a perfect example of an entire computer that you attach to yours, and speak with through one of the (many) storage-oriented protocols. The device itself is a black box, and complex transformations take place between the physical persistence of the data and the logical structures that are exchanged on the wire. There are many layers of indirection, and many…

> skates of magnetic readers flying on a cushion of air above a rapidly rotating disc, with the gap separating a few dozen nanometers, often smaller than the process size in the controller's silicon. Complete aside, the fly-height of a magnetic head is actually fractions of a nanometer (i.e. hundreds of picometers). EDIT: I got this from a talk by Bryan Cantrill[1]. The fly-height is allegedly 0.8 nanometers (800 pic…

I didnt know they where that close, isn't that similar distances as an scanning tunneling microscope tip to an object? or even closer?

Could you provide a reference to a scientific article stating this?

Re: Why I'm usually unnerved when modern SSDs die on us

#62

Post by marcosdumay » Tue, Dec 11, 2018, 5:08 PM UTC

ars wrote: post #9

Waterluvian wrote: post #5

"When a HD died early, you could also imagine undetected manufacturing flaws that finally gave way. With SSDs, at least in theory that shouldn't happen" Why shouldn't it? Isn't it just hardware too? "With spinning HDs, drives might die abruptly but you could at least construct narratives about what could have happened to do that" Why can't you do the same with SSDs? It feels like the author's main complaint is the fr…

> Why can't you do the same with SSDs? Because they don't die incrementally. Wtih a hard disk you'll get bad sectors, growing slowly over time. Or a head crash and then it's all dead. What could cause an entire SSD to die at once? I would totally understand bad sectors, but the whole thing at once? Where it doesn't even try to read existing data?

> What could cause an entire SSD to die at once?

SSD has a problem in that almost every failure case looks more like a head-scratch than a bad sector.

Even the most local problem that is excessive writing on a sector can take the entire chip down.

Re: Why I'm usually unnerved when modern SSDs die on us

#63

Post by lisper » Tue, Dec 11, 2018, 5:11 PM UTC

This is not a technological problem, it's a cultural one. These problems are easily fixed ("easily" by the standards of technical problems that regularly get fixed in other regimes). The reason they don't get fixed is that the customer reaction to failures like this is to rant at the mysterious storage gods that are making their lives miserable.

Needless to say, there are no mysterious storage gods. These are artifacts made by humans, and somewhere out there, there is an engineer who either understands why these failures are happening, or knows how to engineer these devices in such a way that when these failures happen, the cause can be determined, and then a design iteration can be done to reduce the failure rate and make the failure modes more robust. The reason this doesn't happen is that customers aren't demanding it. If major purchasers started demanding, essentially, an SLA from their SSD manufacturers, with actual financial consequences for violating it, you would be amazed how fast all of these problems would get fixed. But instead we vent our frustrations in blog posts and HN comments :-(

Re: Why I'm usually unnerved when modern SSDs die on us

#64

Post by lordnacho » Tue, Dec 11, 2018, 5:14 PM UTC

This seems like a the very human problem of trying to grapple with probability.

We have all sorts of knowledge about about it but when something happens we're still looking for an explanation for each instance.

If you think of it like nuclear decay you'll still be able to say things about the ensemble, but not each individual member.

Re: Why I'm usually unnerved when modern SSDs die on us

#65

Post by joosters » Tue, Dec 11, 2018, 5:16 PM UTC

selectodude wrote: post #38

Wowfunhappy wrote: post #18

Earlier quoted context omitted.

Is there a good reason why we use separate SSD controllers instead of letting the primary cpu handle it? The obvious reason is backwards compatibility, but as more of computing moves to SSDs, is this still relevant? ZFS has shown that removing layers of abstraction with regard to storage can be beneficial.

Apple uses their T2 security enclave as the SSD controller and it's caused quite a stir.

I wonder if you can read any extra state out of the T2 as a result, e.g. more information on wear leveling, temporary read failures and so on, more than the standard SMART counters?

Re: Why I'm usually unnerved when modern SSDs die on us

#66

Post by ricardobeat » Tue, Dec 11, 2018, 5:16 PM UTC

sebazzz wrote: post #50

nneonneo wrote: post #29

A major problem with SSDs seems to be “firmware death” - where the flash chips are physically fine (or mostly fine), but the firmware (or firmware memory) has gotten corrupted due to some programming error, electrical glitch, or cosmic ray. I’ve had scores of older SSDs die after things like power outages and sudden shutdown events. This is super frustrating because the data is physically OK but the controller just i…

I have seen some weird issues with SSDs. I had an OCZ Vertex 2 die on me multiple times, but one thing that stood out most is that after a power cycle or complete system shutdown (note: reboots were just fine) everything I had done last time - install software, update Windows, create files - was gone. The state was reverted to before the time it booted. It was like my computer contained some kind of Reborn chip, exce…

Modern SSDs can have very large DRAM write buffers (8-256MB), this is pretty plausible if it was failing to flush it.

Re: Why I'm usually unnerved when modern SSDs die on us

#67

Post by soundwave106 » Tue, Dec 11, 2018, 5:20 PM UTC

mkirklions wrote: post #44

nneonneo wrote: post #29

A major problem with SSDs seems to be “firmware death” - where the flash chips are physically fine (or mostly fine), but the firmware (or firmware memory) has gotten corrupted due to some programming error, electrical glitch, or cosmic ray. I’ve had scores of older SSDs die after things like power outages and sudden shutdown events. This is super frustrating because the data is physically OK but the controller just i…

Does this explain why I often can access my 'dead' SSDs with my Ubuntu install rather than Windows? I havent run into this issue for ~4 years, so I'm not sure if this is a solved problem or I got lucky a few years back.

Several years ago I was able to revive a "dead" SSD simply by updating the firmware to the latest version.

I haven't had a dead SSD problem in a while though, so I don't know how common firmware issues are now.

Re: Why I'm usually unnerved when modern SSDs die on us

#68

Post by happycube » Tue, Dec 11, 2018, 5:20 PM UTC

gruez wrote: post #15

pythonpatrol wrote: post #7

Earlier quoted context omitted.

Crucial? I can tell you most of them got spectek's chips inside, no wonder why they fail.

what's wrong with spectek chips?

They're Micron's B-stock that didn't pass quality tests. i.e. they'll probably die sooner.

Re: Why I'm usually unnerved when modern SSDs die on us

#69

Post by Malic » Tue, Dec 11, 2018, 5:21 PM UTC

selectodude wrote: post #38

Wowfunhappy wrote: post #18

Earlier quoted context omitted.

Is there a good reason why we use separate SSD controllers instead of letting the primary cpu handle it? The obvious reason is backwards compatibility, but as more of computing moves to SSDs, is this still relevant? ZFS has shown that removing layers of abstraction with regard to storage can be beneficial.

Apple uses their T2 security enclave as the SSD controller and it's caused quite a stir.

Where can I read more about this?

Re: Why I'm usually unnerved when modern SSDs die on us

#70

Post by [deleted] » Tue, Dec 11, 2018, 5:22 PM UTC

joosters wrote: post #65

selectodude wrote: post #38

Earlier quoted context omitted.

Apple uses their T2 security enclave as the SSD controller and it's caused quite a stir.

I wonder if you can read any extra state out of the T2 as a result, e.g. more information on wear leveling, temporary read failures and so on, more than the standard SMART counters?

[deleted]