Cloud Storage for $2 per TB per month • Hacker News Board

Re: Cloud Storage for $2 per TB per month

#61

Post by simias » Fri, Mar 06, 2020, 9:25 PM UTC

>If we assume that the 30 hosts go offline independently

I wonder how reasonable this assumption really is. For regular CPU-bound crypto-mining we see that it tends to centralize geographically in zones where electricity, workforce and real-estate space to build a datacenter are cheap.

Assuming that Sia ends up following a similar distribution, it wouldn't be surprising if several of these hosts ended up sharing a single point of failure.

Beyond that, if only copying stuff around three times to provide tolerance is enough to lower the costs to $2/TB/Mo, why aren't centralized commercial offerings already offering something like that? Just pool three datacenters with 95+% uptime around the world and you should get the same numbers without the overhead of the decentralized solution, no? Surely the overhead of accounting for hosts going offline and redistributing the chunks alone must be very non-trivial. With a centralized, trusted solution it would be much simpler to deal with.

Or is the real catch that Sia has very high latency?

Re: Cloud Storage for $2 per TB per month

#62

Post by Taek » Fri, Mar 06, 2020, 9:25 PM UTC

eyegor wrote: post #42

jakear wrote: post #14

> That means about 2 hours of labor per rig. We’ll call that $50 Does that seem low to anyone else? I don’t really have any background in the area, but 25/hr cost to the company would be less than 20/hr pay for the skilled labor. Other countries are different of course, but in US I could make that much flipping burgers in the right area.

It's outrageously low. They're also fancifully assuming cpu tdp = electrical power, cooling = 0w, and another 0w to the motherboard/network cards. And each box has 1 non-redundant, schmuck-grade $80 psu, as well as a consumer grade mobo. This would never be anywhere near their uptime.

CPU TDP is accounted for, the CPU linked in the blog post draws 65w and that is used in the electricity calculations.

I did realize that I completely forgot about RAM, when I get back to a computer I'll have to make some updates, but it won't materially move the numbers, there's 33% margin of error between the number in the spreadsheet and $2 / TB / Mo.

The $80 PSU is what I could link to from Newegg, I do have experience in industrial electronics, and I know from firsthand experience that you can buy a 10+year PSU at 93% efficiency for well under $80 at 300 watts. At that level, you're going to be able to request all the required cabling as well, which means you're getting a much better price than the $7 per cable linked in the post.

95% uptime means 18 days of downtime per year. Consumer grade PSUs and mobos do much better than that.

Re: Cloud Storage for $2 per TB per month

#63

Post by Mave83 » Fri, Mar 06, 2020, 9:25 PM UTC

We from croit.io operate Ceph based storage including everything from datacenter, power, switches, labor, licenses, all to a price point of 3€/TB.

No consumer ware

Re: Cloud Storage for $2 per TB per month

#64

Post by sydney6 » Fri, Mar 06, 2020, 9:26 PM UTC

vikiomega9 wrote: post #41

kmod wrote: post #15

I worked on the design of Dropbox's exabyte-scale storage system, and from that experience I can say that these numbers are all extremely optimistic, even with their "you can do it cheaper if you only target 95% uptime" caveat. Networking is much more expensive, labor is much more expensive, space is much more expensive, depreciation is faster than they say, etc etc. I don't think the authors have ever done any actua…

> exabyte-scale storage system Somewhat of a random question, can you point me to some state of the art research?

Storage Systems of this scale are thesedays almost exclusively built around object based storage rather than "legacy" block or file backed solutions. I guess today, min.io is would be the way to go. (to "go".. little pun on the end:)

Re: Cloud Storage for $2 per TB per month

#65

Post by john_minsk » Fri, Mar 06, 2020, 9:27 PM UTC

breakyerself wrote: post #51

wmf wrote: post #44

Earlier quoted context omitted.

They're not talking about a real datacenter; they're talking about a deathtrap crypto mine with hard disks instead of GPUs/ASICs. Can't you run a million hard disks off a consumer cable modem? ;-) At the (slow) rate Sia is growing, I don't think there will ever be enough demand to justify this design anyway.

How much is a terabyte of dropbox storage?

0. You can only get 2 TB+

Re: Cloud Storage for $2 per TB per month

#66

Post by z3t4 » Fri, Mar 06, 2020, 9:27 PM UTC

kmod wrote: post #15

I worked on the design of Dropbox's exabyte-scale storage system, and from that experience I can say that these numbers are all extremely optimistic, even with their "you can do it cheaper if you only target 95% uptime" caveat. Networking is much more expensive, labor is much more expensive, space is much more expensive, depreciation is faster than they say, etc etc. I don't think the authors have ever done any actua…

Storage can be far cheaper when decentralized. Sending data over the Atlantic is super expensive compared to LAN networking. Almost all content providers peer with ISP's with onsite hardware. But why stop there, put the "racks" in ppl's basements. Data storage is very compact now a days, you can probably fit 100 TB is a shoe-box.

Re: Cloud Storage for $2 per TB per month

#67

Post by [deleted] » Fri, Mar 06, 2020, 9:30 PM UTC

chx wrote: post #58

kmod wrote: post #15

I worked on the design of Dropbox's exabyte-scale storage system, and from that experience I can say that these numbers are all extremely optimistic, even with their "you can do it cheaper if you only target 95% uptime" caveat. Networking is much more expensive, labor is much more expensive, space is much more expensive, depreciation is faster than they say, etc etc. I don't think the authors have ever done any actua…

> I didn't read all their math but I expect their final result to be off by a factor of 2-5x. Can't be more than 2.5 because Backblaze B2 already gives you $5/TB/Mo.

[deleted]

Re: Cloud Storage for $2 per TB per month

#68

Post by AaronFriel » Fri, Mar 06, 2020, 9:32 PM UTC

Really smart people make this mistake a lot, so I'm wondering what Sia is doing to decorrelate failure rates. If hedge fund quants can turn mortgage tranches into a machine for massive correlated economic losses, can blockchain quants turn storage tranches into a machine for massive correlated storage losses?

Or if one of the major hyperscalers or datacenter operators decides to start selling storage to Sia, it seems likely that their control plane across datacenters could result in correlated failures. A networking outage for their AS could result in multiple datacenters appearing offline concurrently, for example.

Re: Cloud Storage for $2 per TB per month

#69

Post by notyourday » Fri, Mar 06, 2020, 9:32 PM UTC

TheDong wrote: post #37

growt wrote: post #9

I feel like backblaze has already done most of this and has it in production [1]. Whereas this is just done back of the napkin calculation. [1] https://www.backblaze.com/b2/storage-pod.html

Backblaze has tried to make their datacenter as efficient as possible, and still only ends up hitting $5/tb/mo for their b2 service, as a point of reference.

Backblaze is very good but they are definitely not efficient in $$ utilization.

Efficient $$ utilization is bread racks, built out data centers abandoned by the likes of PepBoys that landlords will part for $3/sq foot per year and Google using servers without cases and velcros to keep hard drives attached.

Re: Cloud Storage for $2 per TB per month

#70

Post by arcticbull » Fri, Mar 06, 2020, 9:32 PM UTC

Taek wrote: post #36

kmod wrote: post #15

I worked on the design of Dropbox's exabyte-scale storage system, and from that experience I can say that these numbers are all extremely optimistic, even with their "you can do it cheaper if you only target 95% uptime" caveat. Networking is much more expensive, labor is much more expensive, space is much more expensive, depreciation is faster than they say, etc etc. I don't think the authors have ever done any actua…

Author here. A lot of these numbers are drawn from experience in the mining world, where people realized that when cost is the ultimate bottom line, a lot of corners can be cut. Sia systems don't need a ton of networking. I ran the networking buildout costs by some networking people, and again it comes down to cutting corners. If you only need 10 gbps per rack, if you don't mind having extra milliseconds added, etc,…

Here's the issue. We know that due to economy of scale and domain experience, AWS will always have the lowest cost (to Amazon) for storage -- whether that's totally-reliable storage, or sorta-reliable. If there was a demand for sorta-reliable, they'd build a sorta-reliable S3 and undercut you. Then, blockchain adds inefficiency. Therefore, it's basically impossible for any blockchain solution to have a lower total cost to provide storage.