21–30 of 123 posts
Will we hit a point where the size of the drive is simply too big to get the data off in any decent amount of time?
It's an SMR drive. So better for archiving than regular IO
Copy on write filesystems can probably be optimized for SMR by using TRIM commands to punch holes and rewrite the content sequentially in a new zone. Afaik both zfs and btrfs have plans to do this. That way they can be useful for more than archival.
Will we hit a point where the size of the drive is simply too big to get the data off in any decent amount of time?
This is one reason why RAID 0+1 is a best practice, and RAID 5 & 6 are no longer recommended. It takes too long to rebuild the array, leading to a multi-failed disk situation.
Will we hit a point where the size of the drive is simply too big to get the data off in any decent amount of time?
This is one reason why RAID 0+1 is a best practice, and RAID 5 & 6 are no longer recommended. It takes too long to rebuild the array, leading to a multi-failed disk situation.
Of course one should source RAID disks form 3 different vendors, to ensure that they are from different batches, and are not going to fail at approximately the same time.
Earlier quoted context omitted.
This is one reason why RAID 0+1 is a best practice, and RAID 5 & 6 are no longer recommended. It takes too long to rebuild the array, leading to a multi-failed disk situation.
Raid 01 has its own risk. If the wrong two disks fail your entire array is toast
https://en.wikipedia.org/wiki/Nested_RAID_levels#RAID_50_(RA...
I am going for the 15tb instead of that 14 so I have extra space for backups. Says no body. We are clearly close to the end of spinning rust, absent some new breakthrough.
Spinning drives are definitely not going away anytime soon unless there is a much more significant drop in the cost of SSDs.
I was about to buy 3x 12TB Toshiba for Deep Learning datasets; now I need to reconsider... Does anyone know what are the current reliability stats for >10TB drives? My old 6x 4TB HGST in NAS are running without a single problem for the past 3 years...
Here are the ones for Q3 2018: https://www.backblaze.com/blog/2018-hard-drive-failure-rates...
Scroll down a bit and you'll see the annualized failure rates (AFR). The 10TB and 12TB ones seem to be pretty excellent.
I was about to buy 3x 12TB Toshiba for Deep Learning datasets; now I need to reconsider... Does anyone know what are the current reliability stats for >10TB drives? My old 6x 4TB HGST in NAS are running without a single problem for the past 3 years...
I was about to buy 3x 12TB Toshiba for Deep Learning datasets; now I need to reconsider... Does anyone know what are the current reliability stats for >10TB drives? My old 6x 4TB HGST in NAS are running without a single problem for the past 3 years...
The best resource for that would probably be Backblaze's quarterly hard drive stats. Here are the ones for Q3 2018: https://www.backblaze.com/blog/2018-hard-drive-failure-rates... Scroll down a bit and you'll see the annualized failure rates (AFR). The 10TB and 12TB ones seem to be pretty excellent.
Earlier quoted context omitted.
This is one reason why RAID 0+1 is a best practice, and RAID 5 & 6 are no longer recommended. It takes too long to rebuild the array, leading to a multi-failed disk situation.
Raid 01 has its own risk. If the wrong two disks fail your entire array is toast
However both raid 5 and 6 have 2 huge problems:
Data inflight at write time (power/hardware failures are more likely to corrupt the array, especially silently, which is the worst outcome).
Parity calculations require you to spin up the whole raid5/6 array during a rebuild, massively increasing the chance of a multi drive failure and a lost array. If one close-to-EOL drive dies, putting its sister drives through what is essentially an all day full tilt stress test is a terrible, terrible idea, and this idea keeps getting worse (takes longer) as drive sizes grow.
raid 0+1 sidesteps these issues mostly at a modest increase in drive count, its a no brainier for most setups.