Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I'm usually unnerved when modern SSDs die on us

utcc.utoronto.ca

Post reply on HN
Print view View original

141–150 of 258 posts

Re: Why I'm usually unnerved when modern SSDs die on us

#141

Post by ezoe »

pkaye wrote: post #91

I worked on SSD firmware for quite a long time and here is my perspective. Early flash used to fairly reliable with almost minimal error correction. However with increasing density, smaller processes and multi level cells, it has gone progressively less reliable and slower. Here are some of the things that we need to worry about: https://www.flashmemorysummit.com/English/Collaterals/Procee... To compensate for all th…

But why is that the pay is not so high?

Manufacturing the hardware requires a lot of capitol to begin with. Why these SSD vendors can't afford the high salary to lure the top talents?

Re: Why I'm usually unnerved when modern SSDs die on us

#142

Post by ams6110 »

pkaye wrote: post #91

I worked on SSD firmware for quite a long time and here is my perspective. Early flash used to fairly reliable with almost minimal error correction. However with increasing density, smaller processes and multi level cells, it has gone progressively less reliable and slower. Here are some of the things that we need to worry about: https://www.flashmemorysummit.com/English/Collaterals/Procee... To compensate for all th…

Let's be honest, the chance of a multi-million dollar payoff for an employee is low regardless. Young entry-level devs are optimistic and also vulnerable to believing a line of bullshit on how much their options might be worth some day. I do agree it is a "higher" chance in web/mobile technology, sort of like how your chance of winning the lottery is "higher" if you buy 10 tickets instead of 1.

Re: Why I'm usually unnerved when modern SSDs die on us

#143

Post by ams6110 »

ezoe wrote: post #141
pkaye wrote: post #91

I worked on SSD firmware for quite a long time and here is my perspective. Early flash used to fairly reliable with almost minimal error correction. However with increasing density, smaller processes and multi level cells, it has gone progressively less reliable and slower. Here are some of the things that we need to worry about: https://www.flashmemorysummit.com/English/Collaterals/Procee... To compensate for all th…

But why is that the pay is not so high? Manufacturing the hardware requires a lot of capitol to begin with. Why these SSD vendors can't afford the high salary to lure the top talents?

Hardware is a ruthlessly competitive business and profit margins are very thin.

Re: Why I'm usually unnerved when modern SSDs die on us

#144

Post by ezoe »

godzillabrennus wrote: post #99
loeg wrote: post #93

Earlier quoted context omitted.

Users and administrators almost certainly prefer a 20 minute IO latency over data corruption. Host operating systems should probably flag an IO as failed long before 20 minutes and then you know 1) nothing made it to disk and 2) have some chance to avoid introducing additional corruption, if e.g., the OS is smart enough to kick out the drive when this happens. > Another problem is that the job while rewarding is not…

As someone that identified a bug in a Drobo firmware once and was offered a job on the spot I think the problem with attracting talent is two fold. The first problem is really two parts, not only is it rare to find people who have passion for storage related technologies but very few will gain exposure to these technologies to develop that passion. Kids don't routinely grow up with a SAN in the house. They do tend to…

>very few will gain exposure to these technologies to develop that passion.

This, exactly.

SSD firmware is opaque, hard to learn from outside. On the other hand, trending web-based framework has all the source code opened, with great documents, and ready to use tools. No wonder young people of today find passion on other things rather than SSD.

Re: Why I'm usually unnerved when modern SSDs die on us

#145

Post by LeftTurnSignal »

bitL wrote: post #74

Did you try to bake it in the oven ("reflow")? Sometimes you can add a few more hours to its life, enough for backing it up.

I've had moderate success with sticking failing HDs in the freezer overnight in hopes of getting it to spin one last time.

Never tried baking a drive (ssd or hd), but I have with a red ring'd xbox360 mobo.

Sometimes the "low tech" solutions still work.

Re: Why I'm usually unnerved when modern SSDs die on us

#146

Post by klaasvakie »

loeg wrote: post #93
pkaye wrote: post #91

I worked on SSD firmware for quite a long time and here is my perspective. Early flash used to fairly reliable with almost minimal error correction. However with increasing density, smaller processes and multi level cells, it has gone progressively less reliable and slower. Here are some of the things that we need to worry about: https://www.flashmemorysummit.com/English/Collaterals/Procee... To compensate for all th…

Users and administrators almost certainly prefer a 20 minute IO latency over data corruption. Host operating systems should probably flag an IO as failed long before 20 minutes and then you know 1) nothing made it to disk and 2) have some chance to avoid introducing additional corruption, if e.g., the OS is smart enough to kick out the drive when this happens. > Another problem is that the job while rewarding is not…

> Users and administrators almost certainly prefer a 20 minute IO latency over data corruption.

If the drive part of a RAID setup I would actually prefer it just reports itself failed and doesn't slow down access to the array by scanning itself for 20 minutes.

As far as I know, that is one of the main differences when buying enterprise or nas drives compared to consumer drives. With nas drives, the firmware gives up very quickly since the drive is assumed to be part of an array with redundancy. Consumer drives will retry reads for a very long time before reporting i/o error.

Re: Why I'm usually unnerved when modern SSDs die on us

#148

Post by nightfly »

undersuit wrote: post #123
yzmtf2008 wrote: post #118

Earlier quoted context omitted.

Modern SSDs have incredibly large caches. For example, HP EX920, which is a TLC (triple layer cell) SSD, in its 1TB model contains a whooping 200GB of SLC (single layer cell) cache. It's entirely possible that your changes are simply in cache only, and hasn't been committed to the actual storage.

I don't know if I'm getting exactly what you're saying, but a sudden power off isn't going to wipe your SSD's SLC cache like it would do the DRAM cache on more expensive drives.

But if the controller gets into a weird state and "forgets" to flush the cache and then doesn't even look at the SLC cache on the next boot...

Re: Why I'm usually unnerved when modern SSDs die on us

#149

Post by Filligree »

tzs wrote: post #75
lisper wrote: post #63

This is not a technological problem, it's a cultural one. These problems are easily fixed ("easily" by the standards of technical problems that regularly get fixed in other regimes). The reason they don't get fixed is that the customer reaction to failures like this is to rant at the mysterious storage gods that are making their lives miserable. Needless to say, there are no mysterious storage gods. These are artifac…

> Needless to say, there are no mysterious storage gods What about Consus, the god who protected grain storage in the ancient Roman religion? [1]. Or Eopsin, the Korean goddess of storage? [2] [1] https://en.wikipedia.org/wiki/Consus [2] https://en.wikipedia.org/wiki/Eopsin

I'll give you Consus, but what's mysterious about Eopsin?

Re: Why I'm usually unnerved when modern SSDs die on us

#150

Post by mandelbrotwurst »

apetresc wrote: post #140
loeg wrote: post #104

Earlier quoted context omitted.

I'm mostly curious because I work in a storage-adjacent field (NAS) for a BigCorp and the pay is pretty good, if not quite FAANG level. It's not a startup by any means, but I will easily become a multi-millionaire in a handful of years. I was curious about the other side of the fence.

Huh? If you're going to "easily become a multi-millionaire in a handful of years", then your pay is more than pretty good, and certainly not worse than FAANG level.

Maybe his hands are larger than ours ;)
Post reply on HN