Glacier redux
storagemojo.com
Glacier redux
1–10 of 55 posts
Re: Glacier redux
#2Erm, isn't it getting a bit meta to have a post to respond to the comments on HN for a previous post of yours? Surely, that is exactly what the HN comments are for?
Re: Glacier redux
#3Erm, isn't it getting a bit meta to have a post to respond to the comments on HN for a previous post of yours? Surely, that is exactly what the HN comments are for?
Robin's blog post is not only to respond to HN comments (as you may noticed, he addresses an observation made by another commenter on his own blog).
And he not only responds to the comments, but adds plenty of other information and reasons why he suspects optical media is used by Glacier.
So no, I don't see anything "meta" about this post.
Re: Glacier redux
#4> Old hard drives that are no longer economical for more intensive service, supported by disk-handling robotics.
I am with the author in that I do not think they are using older disks. From personal experience (working with petabytes of disk based storage), I would put money on the fact that AWS is not using stock off the shelf older disks and then powering them on/off as a method of storage. The last thing you want to do with an older disk (5+ years) is ever power it off. The chances of it coming back to life decrease rapidly with each cycle. Picture seized bearings, etc. Your best hope it to keep them continually moving, but never powered down for extended periods of time.
Also, the GB/power/physical space required ratio does not make sense after X number of years. For example, you could replace 3 shelves of 1TB disks, with one shelf of 3TB disks. Yielding less power consumption in less physical space! When you are at scale, data centre space and power are big factors. For these reasons, I do not think they are using old stock disks.
NOTE: Maybe they are using some custom make disks, but from what I know about standard disks, even enterprise, they do not like being powered on/off when they get old.
Re: Glacier redux
#5The deletion charge is interesting. Does deletion guarantee a scrubbing of the data as soon as possible? If so, I could see that as justification for the fee since some sort of significant work is involved in retrieving data, implied by the hours of wait time. If not though, the author has a good point in that Amazon needs to make at least .03/GB in order to be profitable.
Re: Glacier redux
#6> Old hard drives that are no longer economical for more intensive service, supported by disk-handling robotics. I am with the author in that I do not think they are using older disks. From personal experience (working with petabytes of disk based storage), I would put money on the fact that AWS is not using stock off the shelf older disks and then powering them on/off as a method of storage. The last thing you want…
Emphasizing the above, the study a few years ago of disk drives at big HPC centers found that they don't follow a bathtub failure model. Rather, there are very few infant failures, and wear is noticeable starting in the range of 1-2 years in service.
Re: Glacier redux
#7If you have a lot of heavily-read data distributed across machines, you're probably constrained by available spindles rather than available storage space. So co-locating data that will almost never be read with heavily-read data is effectively free. I would guess that Amazon stores Glacier data alongside S3 data, and the lower price reflects the fact that the limiting factor in their storage system is IO rather than capacity.
Re: Glacier redux
#8Full disclosure: I work at UpCloud, a cloud hosting provider and I think business in this industry day and night.
One thing that gets overlooked in almost all comparisons is the pricing models. I actually use Glacier through Arq (brilliant backup for Mac), but the catch is in the requests. I recently uploaded 200GB of photos to glacier and the upload process cost me about $10. The monthly storage is about $2.
The thing is that you shouldn't compare the sole storage price, but the total of cost of storing your data in glacier, which was also overlooked in the original article.
I'm sure AWS has understood this through their S3 storage lifecycle and thus developed an appropriate pricing model for glacier that arouses interest in the product like no other.
Re: Glacier redux
#9If you have a lot of heavily-read data distributed across machines, you're probably constrained by available spindles rather than available storage space. So co-locating data that will almost never be read with heavily-read data is effectively free. I would guess that Amazon stores Glacier data alongside S3 data, and the lower price reflects the fact that the limiting factor in their storage system is IO rather than…
That doesn't offer any clear explanation of why they would charge extra for early deletion of that data, though.
Re: Glacier redux
#10 But NONE of the Hacker News commenters addressed Sony and
Panasonic’s continued investment in high-density optical
disc technology. [...] There has to be a business reason
for the continued investment, i.e. customers prepared to
buy a lot of product in the future and buying a lot right
now.
There must be other customers somewhere if two companies are continuing to develop this stuff.