Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Glacier redux

storagemojo.com

Post reply on HN
Print view View original

31–40 of 55 posts

Re: Glacier redux

#31

Post by raverbashing »

reitzensteinm wrote: post #19

The author handwaves away the power savings of powering down disks by talking about the capital cost of providing power. "Unless the prices of copper, PDUs and diesel-generators have started following Moore’s Law, this is probably more true today than in 2007." But this is fallaciously assuming that Glacier servers would need to be like EC2 or S3 servers that are switched on for x random hours per day. This isn't the…

Makes me wonder something

Database machines are usually never shut down. And when they are, it's usually a manual process (unless we're talking about a power failure or something)

"Diesel generators, for instance - what's the use?"

Backup power (not only for Glacier).

Reducing power consumption in peak hours is a good strategy, however, I'm not sure how worried Amazon is about this (and the difference in price between regular hours and peak hours)

Re: Glacier redux

#32

Post by dmourati »

batbomb wrote: post #29
dmourati wrote: post #24

Earlier quoted context omitted.

Not tape. http://www.zdnet.com/amazon-launches-glacier-cloud-storage-h...

No, it doesn't definitively say the backing storage is not tape based. Only that: "Essentially you can see this as a replacement for tape," and: "inexpensive commodity hardware components" Nowhere did they explicitly deny that it may be tape-backed. In addition, From the article: Instead, Glacier runs on "inexpensive commodity hardware components", he said, noting that the service is designed to be hardware-agnostic.…

"Replacement for tape" is tape? OK.

Re: Glacier redux

#33

Post by raverbashing »

michaelt wrote: post #10

But NONE of the Hacker News commenters addressed Sony and Panasonic’s continued investment in high-density optical disc technology. [...] There has to be a business reason for the continued investment, i.e. customers prepared to buy a lot of product in the future and buying a lot right now. If Amazon was the only customer for high-density optical storage, they'd be crazy to invest in developing it, because Amazon wil…

Well, it's sold at Amazon, it's off the shelf, hardly something with an "exclusive customer"

Backing up to the cloud is great, but a lot of people can't do that

Re: Glacier redux

#34

Post by dkoch »

batbomb wrote: post #29
dmourati wrote: post #24

Earlier quoted context omitted.

Not tape. http://www.zdnet.com/amazon-launches-glacier-cloud-storage-h...

No, it doesn't definitively say the backing storage is not tape based. Only that: "Essentially you can see this as a replacement for tape," and: "inexpensive commodity hardware components" Nowhere did they explicitly deny that it may be tape-backed. In addition, From the article: Instead, Glacier runs on "inexpensive commodity hardware components", he said, noting that the service is designed to be hardware-agnostic.…

"Asked what IT equipment Glacier uses, Amazon told ZDNet it does not run on tape."

Re: Glacier redux

#35

Post by reitzensteinm »

raverbashing wrote: post #31
reitzensteinm wrote: post #19

The author handwaves away the power savings of powering down disks by talking about the capital cost of providing power. "Unless the prices of copper, PDUs and diesel-generators have started following Moore’s Law, this is probably more true today than in 2007." But this is fallaciously assuming that Glacier servers would need to be like EC2 or S3 servers that are switched on for x random hours per day. This isn't the…

Makes me wonder something Database machines are usually never shut down. And when they are, it's usually a manual process (unless we're talking about a power failure or something) "Diesel generators, for instance - what's the use?" Backup power (not only for Glacier). Reducing power consumption in peak hours is a good strategy, however, I'm not sure how worried Amazon is about this (and the difference in price betwee…

It's not about the price they're paying for power (peak hours from the power company's perspective), it's about the peak usage of their data center.

As the article (correctly) says, the capital costs are dependent on your peak usage. If you have 1000 servers using 400kW at peak, you need sufficient air conditioning to extract 400kW worth of heat, and backup generation capable of producing 400kW. It doesn't matter if you only use 100kW 16 hours a day - the capital costs are the same.

I'm suggesting that Glacier could live entirely in non peak periods, meaning that the capital costs are unchanged and the demand curve is flattened.

Re: Glacier redux

#36

Post by justinsb »

What I'd like to see is a storage product that aggregates the unused storage space on EC2 instances.

* By default instance storage isn't attached, so there's probably a lot of completely available capacity.

* Even if attached, it's rare that the full capacity would be used, so thin provisioning would leave some space available.

* Some host machines won't be fully allocated.

I imagine taking this pool of capacity and using erasure coding and replication to build reliable storage. As your volumes come and go, you need to make sure it remains available, which is why I imagine erasure coding across a large number of customers. By integrating with the guest -> host assignment function you can ensure that you never lose data, if need be delaying scheduling until you've copied data elsewhere.

You'd have to throttle reads & writes to ensure that the guests weren't unduly impacted (easier with SSD's predictable IOPS), and the splitting and erasure coding would make for slow reads & writes as well. But this makes the economics a lot more attractive (free!).

One thing that suggests Glacier could be doing this: if I was AWS, and I was doing this, I would not be in any rush to tell EC2 customers that I was "stealing" their unused capacity!

Re: Glacier redux

#37

Post by olegkikin »

Don't forget that S3 costs additional $0.12/GB for transfer out.

So either cheap VPSs or dedicated servers are still much cheaper if you don't want to deal with Glacier.

Re: Glacier redux

#39

Post by raverbashing »

reitzensteinm wrote: post #35
raverbashing wrote: post #31

Earlier quoted context omitted.

Makes me wonder something Database machines are usually never shut down. And when they are, it's usually a manual process (unless we're talking about a power failure or something) "Diesel generators, for instance - what's the use?" Backup power (not only for Glacier). Reducing power consumption in peak hours is a good strategy, however, I'm not sure how worried Amazon is about this (and the difference in price betwee…

It's not about the price they're paying for power (peak hours from the power company's perspective), it's about the peak usage of their data center. As the article (correctly) says, the capital costs are dependent on your peak usage. If you have 1000 servers using 400kW at peak, you need sufficient air conditioning to extract 400kW worth of heat, and backup generation capable of producing 400kW. It doesn't matter if…

The real question then is: is there really a peak hour for datacenter usage/consumption? What's the consumption difference between peak and regular hours (and low-demand hours)

How much power does Amazon use during mornings compared to Netflix watching peak time?

Re: Glacier redux

#40

Post by VintageCool »

dmourati wrote: post #32
batbomb wrote: post #29

Earlier quoted context omitted.

No, it doesn't definitively say the backing storage is not tape based. Only that: "Essentially you can see this as a replacement for tape," and: "inexpensive commodity hardware components" Nowhere did they explicitly deny that it may be tape-backed. In addition, From the article: Instead, Glacier runs on "inexpensive commodity hardware components", he said, noting that the service is designed to be hardware-agnostic.…

"Replacement for tape" is tape? OK.

"Essentially you can see this (our tape) as a replacement for [your] tape".
Post reply on HN