Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloud Storage for $2 per TB per month

blog.sia.tech

Post reply on HN
Print view View original

11–20 of 353 posts

Re: Cloud Storage for $2 per TB per month

#11

Post by reaperducer »

toomuchtodo wrote: post #6
standardUser wrote: post #4

On a related topic, I've had a ton of problems finding a cloud storage system that will reliably handle files around 100-200gb. Does anyone have a recommendation for a service that can handle that file size with ease?

Any object storage system (S3, Backblaze B2, Azure Blob, GCP) should be able to handle those file sizes with proper chunking into smaller parts (limit details below per object store). S3: https://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.... (Max object size: 5TB, Max single multipart size: 5GB) Backblaze B2: https://www.backblaze.com/b2/docs/large_files.html (Max file size: 10TB, Max single object size: 5G…

I'd drop Microsoft from that list because the request for for something "reliable."

It's reliable enough, if you can get it to Microsoft's cloud. But for the last six months I've struggled putting very large files into Azure, using five different connections from five different providers in three locations. Small files are no problem. But large ones take two, three, or four tries.

Re: Cloud Storage for $2 per TB per month

#12

Post by toomuchtodo »

reaperducer wrote: post #11
toomuchtodo wrote: post #6

Earlier quoted context omitted.

Any object storage system (S3, Backblaze B2, Azure Blob, GCP) should be able to handle those file sizes with proper chunking into smaller parts (limit details below per object store). S3: https://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.... (Max object size: 5TB, Max single multipart size: 5GB) Backblaze B2: https://www.backblaze.com/b2/docs/large_files.html (Max file size: 10TB, Max single object size: 5G…

I'd drop Microsoft from that list because the request for for something "reliable." It's reliable enough, if you can get it to Microsoft's cloud. But for the last six months I've struggled putting very large files into Azure, using five different connections from five different providers in three locations. Small files are no problem. But large ones take two, three, or four tries.

I don't disagree Azure is hot garbage, just listing for completeness. S3 and Backblaze B2 are my go-to object stores.

Re: Cloud Storage for $2 per TB per month

#14

Post by jakear »

> That means about 2 hours of labor per rig. We’ll call that $50

Does that seem low to anyone else? I don’t really have any background in the area, but 25/hr cost to the company would be less than 20/hr pay for the skilled labor. Other countries are different of course, but in US I could make that much flipping burgers in the right area.

Re: Cloud Storage for $2 per TB per month

#15

Post by kmod »

I worked on the design of Dropbox's exabyte-scale storage system, and from that experience I can say that these numbers are all extremely optimistic, even with their "you can do it cheaper if you only target 95% uptime" caveat. Networking is much more expensive, labor is much more expensive, space is much more expensive, depreciation is faster than they say, etc etc. I don't think the authors have ever done any actual hardware provisioning before.

I didn't read all their math but I expect their final result to be off by a factor of 2-5x. Hard drives are a surprisingly low percentage of the cost of a storage system.

Re: Cloud Storage for $2 per TB per month

#17

Post by kazen44 »

riffic wrote: post #13

Durability? Availability?

Availability is the big one. Getting high availability makes systems incredibly more complex and expensive.

It's the same reason increasing redundancy and uptime to the next factor of nines is almost logarithmatically more expensive.

Re: Cloud Storage for $2 per TB per month

#18

Post by walrus01 »

I work in telecom/datacenter infrastructure and this is fanciful. The whole way they take the wattage load of one machine and then hand wave away all of the rest of the costs of either building and running a datacenter, or paying ongoing monthly colocation costs... Is just scary. I truly don't mean to offend anyone but this looks like a bunch of enthusiastic dilettantes.

Generators?

UPS?

Cooling costs?

Square footage costs for the real estate itself?

Security and staffing?

At the scale they intend to accomplish they will need at minimum several hundred kilowatts of datacenter space. Even assuming somewhere with a very low kWh cost of electricity, that much space for bare metal things isn't cheap. Go price a lot of square footage and 300kW of equipment load in Quincy, WA or anywhere else comparable, the monthly recurring dollar figure will be quite high.

And all of that is before you even start to look into network costs to build a serious IP network and interconnect with transits and peers.

Re: Cloud Storage for $2 per TB per month

#19

Post by asdkhadsj »

border43 wrote: post #7

I've been using Sia for about three months to backup some personal files. Nothing crazy, but it seems to work well. I'm looking forward to seeing this project mature as well as have some more layers build on top of it moving forward. I really wish the client offered synchronization or access across multiple devices. For now you have to try third party layers on top of Sia to accomplish this.

> I'm looking forward to seeing this project mature as well as have some more layers build on top of it moving forward. I really wish the client offered synchronization or access across multiple devices. For now you have to try third party layers on top of Sia to accomplish this.

Yea I'd actually pick it up now and give it a try if it had this feature.

Re: Cloud Storage for $2 per TB per month

#20

Post by mtzaldo »

border43 wrote: post #7

I've been using Sia for about three months to backup some personal files. Nothing crazy, but it seems to work well. I'm looking forward to seeing this project mature as well as have some more layers build on top of it moving forward. I really wish the client offered synchronization or access across multiple devices. For now you have to try third party layers on top of Sia to accomplish this.

Is this an ad?
Post reply on HN