Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What SMART Stats Tell Us About Hard Drives

backblaze.com

Post reply on HN
Print view View original

21–30 of 71 posts

Re: What SMART Stats Tell Us About Hard Drives

#21

Post by takeda »

joenathan wrote: post #4

This is good info to know, helps me as a sysadmin to be confident in making decisions for my customers and their data. I regularly use a tool called Crystaldiskinfo to check the SMART stats of drives. Will pay more attention to the raw values in the future.

It's interesting that most people rely on the raw values, since the standard does not require them to be meaningful and depending on the vendor it could be anything.

I suspect this is because value, worst, threshold columns are kind of confusing to understand.

Re: What SMART Stats Tell Us About Hard Drives

#22

Post by honkhonkpants »

matt4077 wrote: post #20
honkhonkpants wrote: post #16

Earlier quoted context omitted.

Yeah but no. The problem is that absence of smart counters does in no way indicate good health of the device.

If the presence of smart counters indicates imminent failure, their absence by definition indicates health. Not perfectly, obviously, but this is about probabilities. Say you want some level of data security (i. e. 99.9% over one year). The formula for the risk of data loss is (r^n) where r is the failure rate of drives and n is the number of drives (assuming independence and using just mirroring). But – if you have…

This is a discrete problem. You cannot have 2.9 replicas instead of 3. The fact that 24% of these drives failed without smart indicators means no test exists which can lower your replication from 3 to 2.

Re: What SMART Stats Tell Us About Hard Drives

#23

Post by Retr0spectrum »

nwmcsween wrote: post #7

This data should be put into smartmontools or a separate tool to give a simple good/bad rating as the actual values are somewhat meaningless w/o data.

I feel like this data would be fairly specific to different manufacturers and models. I agree though, looking at raw smart results can be a bit meaningless without context.

Re: What SMART Stats Tell Us About Hard Drives

#24

Post by sn »

esaym wrote: post #14

Ten years ago when I was I trying to learn how to "program", I wrote this bash script (to be added into /etc/cron.daily) that dumps a few smart stats that are normally 0 or slow changing, diffs it with the copy from the previous run, and if anything is different (and cron in configured right) it will email you the diff. Every linux machine I touch gets this file dropped onto it. I've replaced many hard drives because…

I'm not sure why you wouldn't use smartd, which has sane defaults, can immediately alert based on arbitrary smart properties changing, and also handles scheduling smart tests. An arbitrary command can be run instead of sending email.

For example, reallocated sectors are not alerted on by default, so we added '-R 5!' to our smartd config. The full config we have is:

DEVICESCAN -a -s (L/../../6/01) -l selftest -l error -m -M daily -M test -R 5!

Re: What SMART Stats Tell Us About Hard Drives

#25

Post by nitrogen »

honkhonkpants wrote: post #22
matt4077 wrote: post #20

Earlier quoted context omitted.

If the presence of smart counters indicates imminent failure, their absence by definition indicates health. Not perfectly, obviously, but this is about probabilities. Say you want some level of data security (i. e. 99.9% over one year). The formula for the risk of data loss is (r^n) where r is the failure rate of drives and n is the number of drives (assuming independence and using just mirroring). But – if you have…

This is a discrete problem. You cannot have 2.9 replicas instead of 3. The fact that 24% of these drives failed without smart indicators means no test exists which can lower your replication from 3 to 2.

Can having ~2.9 replicas be approximated by using an error-correcting code across a large enough number of drives? Isn't that how RAID 5/6/Z work?

Re: What SMART Stats Tell Us About Hard Drives

#26

Post by sn »

Retric wrote: post #8
mjb wrote: post #2

SMART is a fantastic exercise in sensitivity and specificity. As backblaze is showing with this data, SMART stats have poor sensitivity, but what's much worse for those who run big fleets of drives is their poor specificity. Lots of healthy drives are reported unhealthy by SMART. If I'm running a gold-plated database server, that doesn't matter. A couple of extra planned drive replacements is a small price to pay for…

Thresholds are often useful with these kinds of stats. Aka a drive moving one sector might mean nothing, but moving 30 in a week could be great predictor. Further they only have 70k drives across a range of product lines so what predicts drive X failing very well might say little about drive Y. PS: Rememebr all RAM gets bit flit errors over time. Which is one of the reasons rebooting is often so useful, but also mean…

ECC ram will either correct the error or can raise a non maskable interrupt if it can't be corrected.

Re: What SMART Stats Tell Us About Hard Drives

#27

Post by matt4077 »

honkhonkpants wrote: post #22
matt4077 wrote: post #20

Earlier quoted context omitted.

If the presence of smart counters indicates imminent failure, their absence by definition indicates health. Not perfectly, obviously, but this is about probabilities. Say you want some level of data security (i. e. 99.9% over one year). The formula for the risk of data loss is (r^n) where r is the failure rate of drives and n is the number of drives (assuming independence and using just mirroring). But – if you have…

This is a discrete problem. You cannot have 2.9 replicas instead of 3. The fact that 24% of these drives failed without smart indicators means no test exists which can lower your replication from 3 to 2.

Well, by that measure it doesn't even matter if a drive fails with 10% or 5% probability beyond the replacement costs. Because a test with 50% sensitivity effectively halves the failure rate you have to use.

Since it's all probabilities, triple redundancy does not guarantee complete absence of data loss. On the other side of the spectrum, single replication might offer a better cost/safety ratio for some applications.

A failure that is known in advance is equal to no failure in these term and even when discrete, it will make the difference once in a while.

Re: What SMART Stats Tell Us About Hard Drives

#28

Post by honkhonkpants »

nitrogen wrote: post #25
honkhonkpants wrote: post #22

Earlier quoted context omitted.

This is a discrete problem. You cannot have 2.9 replicas instead of 3. The fact that 24% of these drives failed without smart indicators means no test exists which can lower your replication from 3 to 2.

Can having ~2.9 replicas be approximated by using an error-correcting code across a large enough number of drives? Isn't that how RAID 5/6/Z work?

Sort of but again no. He problem is if you have really wide coded stripes your I/O costs to reconstruct after a failure will be astronomical, and the probability of one failure increases with the number of participating devices. Besides you could not reshape the stripe in response to anticipated failures without reading and writing the whole thing, in which case you'd be better off in terms of I/O costs just evacuating the device in question.

Re: What SMART Stats Tell Us About Hard Drives

#30

Post by teh_klev »

arielhn wrote: post #17

Small nitpicking here, but the moment it popup a modal dialog asking me to enter my email for some kind of subscription I simply close the tab. I do this since three months ago for any unknown website I visit. Such nuisance for what might be a good read.

Shame...you could've just dismissed the popup and not missed out on an interesting article, it's same energy expended but with a nett gain instead of your loss. A small price to pay for BackBlaze willing to share interesting stuff like this and hardly the most egregious examples of this type of thing. Also these types of complaints have been done to death here over the years and are really, really tedious. Please complain to BackBlaze instead of trying to take this thread off-topic.
Post reply on HN