Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HGST gets closer to shipping 10TB HDD

zdnet.com

Post reply on HN
Print view View original

61–66 of 66 posts

Re: HGST gets closer to shipping 10TB HDD

#61

Post by ars »

dmm wrote: post #54
rasz_pl wrote: post #43

Earlier quoted context omitted.

Sounds clever. Too bad SSD (due to being shit and simply dying all at once) will never support that.

Spinning rust can fail all at once too. When ssds die of write endurance they loudly complain long before they actually lose data.

It can, sure, but I think most failures are of the slower sort.

And SSD's only complain if you listen. Does Windows come with a smart checking tool by default? Don't you have to manually install one and check?

I suspect most people with an SSD have no idea where it's holding.

But it doesn't change the "defective by design" nature. It's one thing if you can't do anything about it - but the bricking design is not good. It should go read only on max writes, not brick.

Re: HGST gets closer to shipping 10TB HDD

#62

Post by mtanski »

cmurf wrote: post #60
cmurf wrote: post #35

Earlier quoted context omitted.

Ceph isn't ready for production. GlusterFS is, and it also behaves more like a file system, while Ceph is more object storage. They serve different use cases but GlusterFS is more general purpose I think. No MDS required, scales fairly linearly. They're using it at Facebook with XFS and Btrfs underneath depending on workload. At home you can build little ARM gluster clusters, so it kinda even scales down! http://even…

The supplied link suggests it's not production ready, but it's the wrong link. This is the correct link. John Spray, a senior software engineer at Red Hat, "Some people are already using it in production; we're terrified of this. It's really not ready yet." http://www.zdnet.com/article/linux-storage-futures/ EDIT: And I'm not suggestion it isn't badass, just that the idea Ceph as a first suggestion for general purpos…

cmurf. What the article and John is talking about is CephFS (which we do use). As you can see that is in the heading of the quote that you took.

What John is not talking about is the Object Store (S3 compatible) or RBD (rados block device, EBS like block storage). There's countless people using both Object Store and RBD in production today. Both of these are the most deployed solutions for OpenStack (for storage or object storage). For example the Bloomberg folks use in production.

We use CephFS, and contributed to it earlier, and helped find a lot of bugs in the kernel and the MDS. Last time we uncounted a bug was over a year ago. We currently only use one MDS with a hot spare (backup node actively replaying the log).

What's really missing today from CephFS is the fsck and restore utilities and I believe people are working on them as we speak.

Re: HGST gets closer to shipping 10TB HDD

#63

Post by pkaye »

ars wrote: post #61
dmm wrote: post #54

Earlier quoted context omitted.

Spinning rust can fail all at once too. When ssds die of write endurance they loudly complain long before they actually lose data.

It can, sure, but I think most failures are of the slower sort. And SSD's only complain if you listen. Does Windows come with a smart checking tool by default? Don't you have to manually install one and check? I suspect most people with an SSD have no idea where it's holding. But it doesn't change the "defective by design" nature. It's one thing if you can't do anything about it - but the bricking design is not good.…

"And SSD's only complain if you listen." I'm not sure how this is different from HDDs? For both of them you have to listen and install a SMART checking tool. And HDD failure rates are not great either ranging from 2-12% according to recent Backblaze data.

Re: HGST gets closer to shipping 10TB HDD

#64

Post by epistasis »

ghshephard wrote: post #58
epistasis wrote: post #21

Earlier quoted context omitted.

Yes, there's no more ticking time bomb so there's no urgency to replace the drive, nor any urgency to max out the drive for 15 hours straight during the rebuild. It can be eased into the cluster over the course of days or a week. And yes, in the clusters I've built I've always calculated what the chances are that X simultaneous drive failures will take out any data, and it's always been astronomically low.

What's the sweet spot for Ceph cluster redundancy - Original + 3 Copies? I realize the answer is, "It depends on how much redundancy you want" - but I'm guessing that there is a general best practice that Ceph administrators have adopted?

Ceph is a Consistent and Partition tolerant system in the CAP formulation, at the expense of Availability. Fortunately for my use cases I've not needed high availability, which would require two separate clusters and replication between them.

I've mostly seen people use triplicate pools. For semi-warm storage I've been testing erasure coded pools, with a triplicate cache tier on top, and had good experiences on my test cluster.

Re: HGST gets closer to shipping 10TB HDD

#65

Post by ars »

pkaye wrote: post #63
ars wrote: post #61

Earlier quoted context omitted.

It can, sure, but I think most failures are of the slower sort. And SSD's only complain if you listen. Does Windows come with a smart checking tool by default? Don't you have to manually install one and check? I suspect most people with an SSD have no idea where it's holding. But it doesn't change the "defective by design" nature. It's one thing if you can't do anything about it - but the bricking design is not good.…

"And SSD's only complain if you listen." I'm not sure how this is different from HDDs? For both of them you have to listen and install a SMART checking tool. And HDD failure rates are not great either ranging from 2-12% according to recent Backblaze data.

> I'm not sure how this is different from HDDs?

HDDs don't have a set end of life. Instead you use them till the die naturally. SSD you need to be notified when the end of life is.

There is nothing inherently wrong with either way, it's the action AFTER the death (of either) that is the problem: HDDs do they best they can even after a failure. SSDs just die because they are programed to, not because they have to.

Saying: But you are warned does not excuse that.

Re: HGST gets closer to shipping 10TB HDD

#66

Post by letstryagain »

jacquesm wrote: post #34
letstryagain wrote: post #28

Earlier quoted context omitted.

We stopped using RAID5/6 several years ago because of rebuild time and associated performance drop. Now with these massive disks we don't even use RAID10 any more! Every disk is just paired with one mirror and that's it. (RAID1) The plan is to move to JBOD soon and just use software to store each file on 3 random disks across 3 separate servers.

That creates a substantial window during which you could lose the second drive.

RAID 1 rebuild time is the same as RAID 10 but if the entire thing fails you only lose one drive's info, not the entire RAID 10 array.
Post reply on HN