Operational drives with one or more of our five SMART stats greater than zero – 4.2%
This doesn't gel with the prior table, which shows that 4.8% of operation drives have non-zero SMART188 alone.
What SMART Stats Tell Us About Hard Drives
41–50 of 71 posts
Re: What SMART Stats Tell Us About Hard Drives
#42Earlier quoted context omitted.
(Warning pedantic script review) Add a "set -e" to catch errors. Say if the disk can't be read or file can't be written. Why reuse the same temp file? Make a new one with mktemp and auto clean it via an exit trap. As it's written this isn't concurrently safe. Exiting -1 on error? Don't use negatives. Wrap it all in a main() function and use locals instead of global vars.
You don't even need the 'smartStats' temporary file at all, since you can do that with just one grep: smartctl -a /dev/sda | grep '\(Reallocated_Sector_Ct\|Current_Pending_Sector\|Offline_Uncorrectable\|UDMA_CRC_Error_Count\)' > /root/stats
awk is probably better in this case, since you can save only the raw values (and other columns that you deem important):
awk '/Reallocated_Sector_Ct|Current_Pending_Sector|Offline_Uncorrectable|UDMA_CRC_Error_Count/ {print $2,$10}'Re: What SMART Stats Tell Us About Hard Drives
#43Ten years ago when I was I trying to learn how to "program", I wrote this bash script (to be added into /etc/cron.daily) that dumps a few smart stats that are normally 0 or slow changing, diffs it with the copy from the previous run, and if anything is different (and cron in configured right) it will email you the diff. Every linux machine I touch gets this file dropped onto it. I've replaced many hard drives because…
This is basically what smartmontools does:
/var/log/syslog:Oct 6 08:14:10 hostname smartd[573]: Device: /dev/sda [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 61 to 62
/var/log/syslog:Oct 6 09:44:10 hostname smartd[573]: Device: /dev/sda [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 62 to 61
/var/log/syslog:Oct 6 10:14:10 hostname smartd[573]: Device: /dev/sda [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 61 to 60
/var/log/syslog:Oct 6 18:44:10 hostname smartd[573]: Device: /dev/sda [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 60 to 59Re: What SMART Stats Tell Us About Hard Drives
#44SMART is a fantastic exercise in sensitivity and specificity. As backblaze is showing with this data, SMART stats have poor sensitivity, but what's much worse for those who run big fleets of drives is their poor specificity. Lots of healthy drives are reported unhealthy by SMART. If I'm running a gold-plated database server, that doesn't matter. A couple of extra planned drive replacements is a small price to pay for…
SMART stats are only one source of data though that can be predictors of drive failures. Hard drives can experience latency spikes and erratic performance compared to peers before SMART records problems, for example. This is part of why predictive monitoring can be so difficult - the data you didn't know you needed is probably among the ones you didn't ingest into your metrics, and you can't dump literally everything in /proc every other second to your metrics system without sacrificing some CPU or network bandwidth either.
Re: What SMART Stats Tell Us About Hard Drives
#45interestingly not tracking 184 'End-TO-END ERROR', considered by couple of smart tools to be a failure of the drive.
Re: What SMART Stats Tell Us About Hard Drives
#46Ten years ago when I was I trying to learn how to "program", I wrote this bash script (to be added into /etc/cron.daily) that dumps a few smart stats that are normally 0 or slow changing, diffs it with the copy from the previous run, and if anything is different (and cron in configured right) it will email you the diff. Every linux machine I touch gets this file dropped onto it. I've replaced many hard drives because…
Now you can just `apt install smart-notifier`
Re: What SMART Stats Tell Us About Hard Drives
#47Someone should set up an AI startup for harddrive failure prediction.
Re: What SMART Stats Tell Us About Hard Drives
#48This is good info to know, helps me as a sysadmin to be confident in making decisions for my customers and their data. I regularly use a tool called Crystaldiskinfo to check the SMART stats of drives. Will pay more attention to the raw values in the future.
It's interesting that most people rely on the raw values, since the standard does not require them to be meaningful and depending on the vendor it could be anything. I suspect this is because value, worst, threshold columns are kind of confusing to understand.
There aren't too many vendors for spinning disks, and if you have a lot of disks it doesn't take too long to see that the sector count metrics correspond to sectors. In my experience, bad sector count is a good predictor of future trouble, and running disks until they threw read errors (before we were running smart monitoring), they all had lots of bad sectors. That said, there's a threshold, getting to 100 slowly is probably ok, a thousand is probably not.
SSDs though, they just disappear from the bus when they fail; so I haven't been able to look at a dead one and see what looks like a useful predictor. I have seen some ssds reallocating a big block, which kills performance while its going on...
Re: What SMART Stats Tell Us About Hard Drives
#49Ten years ago when I was I trying to learn how to "program", I wrote this bash script (to be added into /etc/cron.daily) that dumps a few smart stats that are normally 0 or slow changing, diffs it with the copy from the previous run, and if anything is different (and cron in configured right) it will email you the diff. Every linux machine I touch gets this file dropped onto it. I've replaced many hard drives because…
How about: a comment on (1) what this bash script does, (2) and why, and (3) the reason for dumping a disk as a result.
Re: What SMART Stats Tell Us About Hard Drives
#50Ten years ago when I was I trying to learn how to "program", I wrote this bash script (to be added into /etc/cron.daily) that dumps a few smart stats that are normally 0 or slow changing, diffs it with the copy from the previous run, and if anything is different (and cron in configured right) it will email you the diff. Every linux machine I touch gets this file dropped onto it. I've replaced many hard drives because…
How about: a comment on (1) what this bash script does, (2) and why, and (3) the reason for dumping a disk as a result.
1) How about read the script. 2) How about read the subject 3) How about understanding it.