Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Glacier Costlier Than S3 for Small Files

therub.org

Post reply on HN
Print view View original

11–17 of 17 posts

Re: Glacier Costlier Than S3 for Small Files

#11

Post by Veratyr »

tghw wrote: post #8

Personally, I'm looking forward to Backblaze's cloud storage, which is being advertised at $0.005/GB/month. Cheaper than glacier, without all of the transaction fees. https://www.backblaze.com/b2/cloud-storage.html

And none of the 3 hour retrieval time. Might drag me off GCS.

Re: Glacier Costlier Than S3 for Small Files

#12

Post by tw04 »

tghw wrote: post #8

Personally, I'm looking forward to Backblaze's cloud storage, which is being advertised at $0.005/GB/month. Cheaper than glacier, without all of the transaction fees. https://www.backblaze.com/b2/cloud-storage.html

The whole custom API instead of S3 or Swift compatibility is really, REALLY annoying.

Re: Glacier Costlier Than S3 for Small Files

#13

Post by toomuchtodo »

tw04 wrote: post #12
tghw wrote: post #8

Personally, I'm looking forward to Backblaze's cloud storage, which is being advertised at $0.005/GB/month. Cheaper than glacier, without all of the transaction fees. https://www.backblaze.com/b2/cloud-storage.html

The whole custom API instead of S3 or Swift compatibility is really, REALLY annoying.

Someone from Backblaze previously mentioned on HN they're working on an S3-compatible API.

Re: Glacier Costlier Than S3 for Small Files

#14

Post by nine_k »

Glacier has a very clear use case, to my mind.

It is useful for keeping archives of massive data you're unlikely to ever need, but legally obliged to keep around, or just want to have available for a very improbable later examination. Think some huge transaction logs of two years back.

For a case like this, you don't need fast retrieval, and mostly you don't need retrieval at all. You plan ahead to only ever retrieve a small percent of these data. The rest will be silently discarded when retention period has expired.

If your use case is not like that, Glacier probably makes little sense for you.

This is totally not a backup which you likely keep in order to restore the entire state from as soon as possible.

Re: Glacier Costlier Than S3 for Small Files

#15

Post by drue »

raverbashing wrote: post #3

The real question is why are people glaciering small files? Tar everything and send it up. And then do an incremental backup Amazon charges per operation, reduce the number of operations

If you write a AWS-native service that uses S3 as a back end (a very reasonable thing to do), it may create many small files in S3. It would be unreasonable to incur the costs to retrieve them and tar them up, and it also does not make cents to "click the box" to have them archived to glacier.

Re: Glacier Costlier Than S3 for Small Files

#16

Post by drue »

TazeTSchnitzel wrote: post #6

> For consistency and precision, the following units are used throughout this article. > KB: 1,024 bytes, expressed as 2^10 Why not just use KiB? Unlike KB, it's unambiguously binary.

I primarily wanted to note that I used 2^X notation in the math, in case it was confusing for folks. I used KB/MB/GB to be consistent with the language AWS uses on their S3 pricing page (https://aws.amazon.com/s3/pricing/).

Re: Glacier Costlier Than S3 for Small Files

#17

Post by drue »

nine_k wrote: post #14

Glacier has a very clear use case, to my mind. It is useful for keeping archives of massive data you're unlikely to ever need, but legally obliged to keep around, or just want to have available for a very improbable later examination. Think some huge transaction logs of two years back. For a case like this, you don't need fast retrieval, and mostly you don't need retrieval at all. You plan ahead to only ever retrieve…

Right, but you missed the salient point: it also does not make sense for files smaller than about 200KB.
Post reply on HN