SSDs: A gift and a curse • Hacker News Board

Re: SSDs: A gift and a curse

#91

Post by 72deluxe » Wed, Jun 03, 2015, 8:45 AM UTC

noobie wrote: post #43

mrb wrote: post #10

Of course SSD firmware is buggy. You know why? Because any half-decent electrical and computer engineering team can slap a NAND flash controller and some flash chips on a PCB, take the controller's manufacturer's reference firmware implementation, tweak the dozens of knobs provided by the reference implementation (ignore FLUSH commands, change the amount of reserved sectors, disable this, enable that, etc), change th…

Excuse me but most of what you typed went right over my head. Where can one start learning about SSD/HDD firmware and most of what you said?

Firmware is the software on the hardware, basically. If you take a hard disk and look underneath it, there is a PCB with chips on it. The firmware lives in there and is responsible for doing the job of reading/writing data to the disk in a specific format.

If you take the platters out of one hard disk and put it into another hard disk with different firmware, the other hard disk PCB won't be able to read it or write to it. I found this out by having a dead Hitachi drive (yes, the IBM "Deathstar" disks after they were sold to Hitachi) so I bought another identical drive and swapped PCBs but on startup the read/write head would not settle as there was a mismatch between the PCB firmware number and the firmware identifier written on the start of the disk. So, although I thought they were identical disks they were not.

Re: SSDs: A gift and a curse

#92

Post by JoeAltmaier » Wed, Jun 03, 2015, 1:23 PM UTC

Laforet wrote: post #65

JoeAltmaier wrote: post #34

Earlier quoted context omitted.

To expand: in my 20-year experience, no small-computer RAID 5/6 controller I have had experience with has ever saved any data, ever. All failures are hard and non-recoverable. Which begs the question: why use them at all? I mirror only.

RAIDs are in place to for your guaranteed uptime in the SLA. With the current storage density RAID5/6 is probably more fragile than single disks because UREs are very likely during a rebuild. Nonetheless having a degraded array is probably better than having an offline system and it will buy you some time to migrate. Mirrors are ideal, but it is hard to justify the upfront cost.

For small home machines, the storage cost is minimal. All my home machines are mirrored, for around $100 each. Its saved me several times.

Re: SSDs: A gift and a curse

#93

Post by deelowe » Wed, Jun 03, 2015, 3:45 PM UTC

linker3000 wrote: post #88

deelowe wrote: post #66

Isn't running raid1/5/6 on ssds silly b/c they'll all die at the same time? And hardware raid on top of that? Why? SSDs have a fairly consistent failure curve (exusing firmware bugs and other random events) for a given model, so they'll wear evenly in a raid setup. This means they'll all die at the same time as writes/reads are distributed fairly evenly across the disks. Given the size of today's drives, you may not…

In my experience (Flash Platforms Group, HSGT), a significant number of flash device failures are caused by mechanical issues, such as: on-die faults, wire bonding problems and solder joint/package stresses which may occasinally be down to a production issue, but are more often attributed to rough handling during installation or thermal stress. This is less of an issue with SFF SSDs, but especially true of PCIe produ…

That's what I had considered as well. Specifically, that the wear out failures are on the right hand of the curve, meaning that other issues would tend to dominate. What you said makes a lot of sense: failures due to thermal/electrical stress, manufacturing issues, and handling.

Also, there's the controllers, which tend to be a significant source for issues.

So yeah, maybe theoretically, the wear out would be a concern for raid. However, practically this is rarely an issue as other failures randomize the distribution enough to where this never causes a problem in-situ.

Re: SSDs: A gift and a curse

#94

Post by marssaxman » Wed, Jun 03, 2015, 4:02 PM UTC

gcr wrote: post #67

marssaxman wrote: post #45

Earlier quoted context omitted.

I'm eagerly awaiting the part of the cycle where the GPU gets folded back into the processor. I'm really not interested in writing code against mystery-meat architectures with no documented ABI...

We're already seeing that with integrated, "soldered-on" graphics cards, and with frameworks like OpenCL, and higher-level abstractions like Theano, Torch7, etc.

All we need now is documentation for the low-level abstractions so we can scrap all the vendor code and compile our own GPU machine language.

Re: SSDs: A gift and a curse

#95

Post by greggyb » Wed, Jun 03, 2015, 6:23 PM UTC

marssaxman wrote: post #45

Turing_Machine wrote: post #22

Earlier quoted context omitted.

I wouldn't say that it's never going to happen based on current trends. This kind of thing tends to be quasi-cyclic. At one time, desktop computers had separate FPUs. Over time, those got integrated onto the main CPU, but then the trend switched back, with heavy calculations moving off-CPU to the GPU. The mainframe->desktop->cloud story is similar. I say "quasi"-cyclic because (e.g.) cloud isn't really the same as ma…

I'm eagerly awaiting the part of the cycle where the GPU gets folded back into the processor. I'm really not interested in writing code against mystery-meat architectures with no documented ABI...

I am not sure based on reading your post whether or not I am providing new information to you:

Keep an eye on the AMD APUs moving forward and their HSA progress.

Re: SSDs: A gift and a curse

#96

Post by yellowapple » Wed, Jun 03, 2015, 6:38 PM UTC

Laforet wrote: post #65

JoeAltmaier wrote: post #34

Earlier quoted context omitted.

To expand: in my 20-year experience, no small-computer RAID 5/6 controller I have had experience with has ever saved any data, ever. All failures are hard and non-recoverable. Which begs the question: why use them at all? I mirror only.

RAIDs are in place to for your guaranteed uptime in the SLA. With the current storage density RAID5/6 is probably more fragile than single disks because UREs are very likely during a rebuild. Nonetheless having a degraded array is probably better than having an offline system and it will buy you some time to migrate. Mirrors are ideal, but it is hard to justify the upfront cost.

If you're trying to maintain guaranteed uptime, you're probably better off with machine-level redundancy rather than disk-level redundancy.

> Mirrors are ideal, but it is hard to justify the upfront cost.

The upfront cost of redundant backups can also be high, but that doesn't mean one should forego them (unless their data isn't worth it, in which case, why bother collecting it?).

Going cheap on things is fine in the home computing realm, but once you're in the business realm, going cheap almost inevitably becomes more expensive in the long run.

Re: SSDs: A gift and a curse

#97

Post by marssaxman » Wed, Jun 03, 2015, 8:07 PM UTC

greggyb wrote: post #95

marssaxman wrote: post #45

Earlier quoted context omitted.

I'm eagerly awaiting the part of the cycle where the GPU gets folded back into the processor. I'm really not interested in writing code against mystery-meat architectures with no documented ABI...

I am not sure based on reading your post whether or not I am providing new information to you: Keep an eye on the AMD APUs moving forward and their HSA progress.

Thanks, that sounds like an interesting development.

Re: SSDs: A gift and a curse

#98

Post by cerberusss » Thu, Jun 04, 2015, 10:18 AM UTC

JoeAltmaier wrote: post #92

Laforet wrote: post #65

Earlier quoted context omitted.

RAIDs are in place to for your guaranteed uptime in the SLA. With the current storage density RAID5/6 is probably more fragile than single disks because UREs are very likely during a rebuild. Nonetheless having a degraded array is probably better than having an offline system and it will buy you some time to migrate. Mirrors are ideal, but it is hard to justify the upfront cost.

For small home machines, the storage cost is minimal. All my home machines are mirrored, for around $100 each. Its saved me several times.

I've never understood this. At home, you don't need uptime, thus you're much better off with a real offline backup instead of a mirror.

Re: SSDs: A gift and a curse

#99

Post by yellowapple » Thu, Jun 04, 2015, 6:06 PM UTC

cerberusss wrote: post #98

JoeAltmaier wrote: post #92

Earlier quoted context omitted.

For small home machines, the storage cost is minimal. All my home machines are mirrored, for around $100 each. Its saved me several times.

I've never understood this. At home, you don't need uptime, thus you're much better off with a real offline backup instead of a mirror.

You don't need uptime, but it's nice to have. Repairing a degraded mirror takes a lot less time than restoring from a backup in the vast majority of cases.