Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HGST gets closer to shipping 10TB HDD

zdnet.com

Post reply on HN
Print view View original

51–60 of 66 posts

Re: HGST gets closer to shipping 10TB HDD

#51

Post by moe »

jacquesm wrote: post #46
moe wrote: post #8

Earlier quoted context omitted.

the risk of an additional failure in say a RAID5 or RAID6 configuration is just too high. Two disk failures in the same array within 15 hours is very rare.

Frequent enough that I've seen it twice on different arrays. The reason for this is that a drive failure can be caused by an external factor (usually multiple drives fed by the same broken power supply, but not yet so broken that it goes off-line).

I didn't say they don't happen.

They're just so rare that the difference between ~6h rebuilds with 4T spindles versus 15h rebuilds with 10T spindles simply doesn't matter. Any homeopathic advantage of slightly faster rebuilds is dwarfed by the benefits of higher density (less hardware per capacity).

Re: HGST gets closer to shipping 10TB HDD

#52

Post by mtanski »

ryan-c wrote: post #26
ars wrote: post #2

Wow, hard drives are getting complicated! You have to write 256MB to change a single bit? And you have various zones, and you have to keep track of where data is written, because it's not written in order, and can be on any zone? You would need some sort of battery backed scratch/cache area to pull this off, so that you don't have to write very often, otherwise I can't imagine performance will be very good.

Having maybe 16 to 64 GB of flash storage (for writeback cache) would probably make a significant difference here. I think over the next five to ten years tiered storage will probably go mainstream.

Tiered storage is already here. If you're using block storage in the cloud those guys are already do all sorts of optimizations (for their costs) that include tiering. And on your own desktop / own server side you can do this today on Linux with bcache or dm-cache. Works great for databases.

Re: HGST gets closer to shipping 10TB HDD

#53

Post by mtanski »

We spent a bit of time talking about about these "shingled drives" at the LSF/MM summit earlier in the week in Boston. Storage is becoming a more fascinating topic with so many technologies (SSDs, shingled disk, non-volatile memory) and so new open source solutions (Ceph, Luster).

Re: HGST gets closer to shipping 10TB HDD

#54

Post by dmm »

rasz_pl wrote: post #43
ars wrote: post #33

Earlier quoted context omitted.

http://www.hgst.com/tech/techlib.nsf/techdocs/CCA92CECECFCC7... The way it works is by realizing there are only a few bad sectors on the failed drive, so rebuild that area first, the used the "bad" drive to help rebuild the rest of the array at full speed.

Sounds clever. Too bad SSD (due to being shit and simply dying all at once) will never support that.

Spinning rust can fail all at once too. When ssds die of write endurance they loudly complain long before they actually lose data.

Re: HGST gets closer to shipping 10TB HDD

#55

Post by mdnormy »

cmurf wrote: post #35
walrus wrote: post #30

Earlier quoted context omitted.

Does this cause problems in practice? I've never used Ceph in production, but it doesn't seem unreasonable to expect the sysadmin to keep clock skew in check.

Ceph isn't ready for production. GlusterFS is, and it also behaves more like a file system, while Ceph is more object storage. They serve different use cases but GlusterFS is more general purpose I think. No MDS required, scales fairly linearly. They're using it at Facebook with XFS and Btrfs underneath depending on workload. At home you can build little ARM gluster clusters, so it kinda even scales down! http://even…

> Ceph isn't ready for production

DreamHost would beg to differ. They've been using it for their DreamObject[1] for few years. Granted that they have have better visibility earlier because Sage Weil, but I've seen Ceph deployment for production in the wild since mid-2014.

[1] https://www.dreamhost.com/cloud/storage/

Re: HGST gets closer to shipping 10TB HDD

#57

Post by duaneb »

walrus wrote: post #30
insaneirish wrote: post #27

Earlier quoted context omitted.

So, yea, Ceph: https://twitter.com/tbmcmullen/status/566856092185923584

Does this cause problems in practice? I've never used Ceph in production, but it doesn't seem unreasonable to expect the sysadmin to keep clock skew in check.

Clock skew is notoriously difficult to actually fix, especially with a network partition.

Re: HGST gets closer to shipping 10TB HDD

#58

Post by ghshephard »

epistasis wrote: post #21
ghshephard wrote: post #17

Earlier quoted context omitted.

A Drive still needs 15 hours to rebuild though, so still the same amount of work. You just have more redundancy with Ceph, and don't have the ticking time bomb of losing an entire array if you don't complete the rebuild before another drive in the array goes. Also, does Ceph distribute it's objects so that two drives don't contain the same set of objects? I.E. it's probabilistically impossible that a number of drives…

Yes, there's no more ticking time bomb so there's no urgency to replace the drive, nor any urgency to max out the drive for 15 hours straight during the rebuild. It can be eased into the cluster over the course of days or a week. And yes, in the clusters I've built I've always calculated what the chances are that X simultaneous drive failures will take out any data, and it's always been astronomically low.

What's the sweet spot for Ceph cluster redundancy - Original + 3 Copies? I realize the answer is, "It depends on how much redundancy you want" - but I'm guessing that there is a general best practice that Ceph administrators have adopted?

Re: HGST gets closer to shipping 10TB HDD

#59

Post by mtanski »

MPSimmons wrote: post #6
fiatmoney wrote: post #4

The 8 TB HDDs available now take a full 15 hours or so for a linear scan at max speed. God help you if you ever need to rebuild a RAID array made of them; the risk of an additional failure in say a RAID5 or RAID6 configuration is just too high. We're nearing the point at which the throughput relative to the capacity and the risk of failure or corruption makes further capacity less and less useful, unless you're doing…

Shingled drives are probably more likely used in a RAID-10 scenario where you're not recalculating parity, but just rebuilding linearly. These drives (like the Western Digital Purple) are used predominantly for archival data. You should never have to randomly write to, for instance, surveillance video, or similar streamed media. This isn't for your SQL database server.

This is pedantic, but unless the filesystem is build specifically for WORM devices you always end up with some non-sequential data. In the use cases you're describing they'll be some filesystem metadata (freemaps, directory entries, inode tables) that will need to be rewritten.

Re: HGST gets closer to shipping 10TB HDD

#60

Post by cmurf »

cmurf wrote: post #35
walrus wrote: post #30

Earlier quoted context omitted.

Does this cause problems in practice? I've never used Ceph in production, but it doesn't seem unreasonable to expect the sysadmin to keep clock skew in check.

Ceph isn't ready for production. GlusterFS is, and it also behaves more like a file system, while Ceph is more object storage. They serve different use cases but GlusterFS is more general purpose I think. No MDS required, scales fairly linearly. They're using it at Facebook with XFS and Btrfs underneath depending on workload. At home you can build little ARM gluster clusters, so it kinda even scales down! http://even…

The supplied link suggests it's not production ready, but it's the wrong link. This is the correct link. John Spray, a senior software engineer at Red Hat, "Some people are already using it in production; we're terrified of this. It's really not ready yet." http://www.zdnet.com/article/linux-storage-futures/

EDIT: And I'm not suggestion it isn't badass, just that the idea Ceph as a first suggestion for general purpose RAID replacement is probably not a good idea just yet, and that there's an alternative.

Post reply on HN