Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloud Storage for $2 per TB per month

blog.sia.tech

Post reply on HN
Print view View original

231–240 of 353 posts

Re: Cloud Storage for $2 per TB per month

#231

Post by justinmeiners »

samstave wrote: post #215
Taek wrote: post #36

Earlier quoted context omitted.

Author here. A lot of these numbers are drawn from experience in the mining world, where people realized that when cost is the ultimate bottom line, a lot of corners can be cut. Sia systems don't need a ton of networking. I ran the networking buildout costs by some networking people, and again it comes down to cutting corners. If you only need 10 gbps per rack, if you don't mind having extra milliseconds added, etc,…

Why the heck would one want to store data in a non highly reliable anything run by a team who is building around cutting corners??? Wtf am i missing here? —- So, I assume you’re quite young. ~31 or so... I say this because youre not speaking with a level of professionalism one would ostensibly have when talking about storing (and providing access to) another’s data... And if your actual business model is focused on c…

You're taking "cutting corners" out of context. He is describing how individual nodes don't need to run at the standards of regular data servers and are hence cheaper, but in aggregate provide a reliable service.

> So, I assume you’re quite young.

So you do you have a technical criticism?

Re: Cloud Storage for $2 per TB per month

#232

Post by ianopolous »

ianopolous wrote: post #217
ianopolous wrote: post #204

Earlier quoted context omitted.

No need to be rude. EDIT: The offensive part was removed What was the probability of failure of your drives? My guess is you just didn't hit the threshold for your failure rate. The maths checks out (PhD here). Seriously, do the calculation.

To clarify, the assumptions I'm making for the calculation are: 1) a Fixed probability of a server failing 2) a fixed erasure coding scheme used for all files 3) uncorrelated server failures 4) an erasure fragment is stored on a random server

It boils down to the following:

You can calculate a probability L of losing a given file.

Because we've assumed totally uncorrelated failures that means this is the same for all files, and that the probability of losing NO files if you have T files is (1 - L)^T

As you can see, this approaches 0, meaning Pr(losing a file) approaches 1 as T increases.

Using the probability of file loss in Sia, which I would say is is too low, but lets ignore that. They get L = 10^-19.

This leads to T = ~10^19 before you expect to lose data. If you're erasure coding on the byte level, then that's 10 exa bytes.

I expect your probability of failure is much less than random nodes on a distributed global network of volunteers. so yes, ~petabyte is below the threshold, but there is a threshold.

Re: Cloud Storage for $2 per TB per month

#234

Post by iampims »

justinmeiners wrote: post #228
jakewalker wrote: post #94

Earlier quoted context omitted.

Sounds kinda like the people who thought that bundling a bunch of bad mortgage debt together in slices could get it rated AAA.

Not really. Just probability. If you have fully redundant services then ALL of them have to go down to have an outage. Suppose you 5 copies with 75% uptime each. The probability that all of them are down is 0.25^5 ~ 0.0009 Now of course that assumes they are uncorrelated, but since Sia nodes are distributed across the internet, that's likely as opposed to multiple servers at a few data centers like AWS. Turns out mor…

They all run the same software stack though, so despite being deployed on diverse hardware, so they can’t claim to only have independent failure modes.

Re: Cloud Storage for $2 per TB per month

#235

Post by ac29 »

Ajedi32 wrote: post #103
throwaway3157 wrote: post #93

Earlier quoted context omitted.

Who are the customers for such unreliable systems?

Sia is very reliable from the customer's perspective. Its only individual systems and storage pools that have lax uptime requirements. Thanks to some clever network-level redundancy mechanisms (10-of-30 redundancy), 95% uptime at the storage pool level translates to 99.9999% uptime from the customer's perspective. See the "Uptime Math" section in the OP for details.

When you says Sia is reliable, do you mean it could be reliable if hypothetical X, Y, and Z things happen?

Because according to the homepage (sia.tech) there are only 895 hosts storing a total of 206TB right now, which is a very, very small amount. Backblaze, a relatively small player as compared to the big cloud providers has 1.1 million TB of raw capacity as of last year (redundancy reduces the available capacity, but still) [0].

[0] https://www.backblaze.com/blog/hard-drive-stats-for-2019/

Re: Cloud Storage for $2 per TB per month

#237

Post by ComputerGuru »

kllrnohj wrote: post #200
ComputerGuru wrote: post #141

There is way too much hand-waving and assuming going on this article. It is a load of BS that does not take into account real-world inefficiencies. e.g. sometimes buying in bulk is more expensive than buying at retail, esp when you need consistent supply. Sure, you may need only an hour of sysadmin time a day, but what sysadmin will let you employ them an hour a day? The buildout did not list a CPU. The assumptions a…

> The buildout did not list a CPU. It did, or does now anyway, the RYZEN 3 1200 for $95. EDIT: Although the better option is the 3200G so that you can actually get a display output from the thing. Same price, so it doesn't really change anything, but it does cut the CPU core count down a bit if that matters at all. That said the buildout still doesn't work because you can't actually plug the "sata splitter" cable the…

Yeah, that cpu wasn’t listed at the time. It’s clear that this is a thought experiment and nowhere near worth the attention it is getting.

Re: Cloud Storage for $2 per TB per month

#238

Post by ac29 »

Taek wrote: post #39
jakear wrote: post #14

> That means about 2 hours of labor per rig. We’ll call that $50 Does that seem low to anyone else? I don’t really have any background in the area, but 25/hr cost to the company would be less than 20/hr pay for the skilled labor. Other countries are different of course, but in US I could make that much flipping burgers in the right area.

Assembling computers and replacing drives is generally speaking unskilled labor, right there next to factory workers.

Right, the people putting together servers for Dell or HP certainly are not getting paid $20 or $25/hr. If your standard of quality is "crypto shitbox", even $10/hr seems more than enough. This is assuming that many of these storage farms are going to be in places like Vietnam or Romania, not the US or Germany.

Re: Cloud Storage for $2 per TB per month

#239

Post by ineedasername »

Taek wrote: post #140
arcticbull wrote: post #70

Earlier quoted context omitted.

Here's the issue. We know that due to economy of scale and domain experience, AWS will always have the lowest cost (to Amazon) for storage -- whether that's totally-reliable storage, or sorta-reliable. If there was a demand for sorta-reliable, they'd build a sorta-reliable S3 and undercut you. Then, blockchain adds inefficiency. Therefore, it's basically impossible for any blockchain solution to have a lower total co…

Sia is a lot more than a low cost storage platform. It's a full reimagining of how the cloud should work with an emphasis on user ownership and control, open access for developers, and ultimately decentralization that allows users to be certain their applications will always work (no more "RSS Reader is shutting down"). This post is intended to address people who do not feel that decentralization can be cost effectiv…

What aspect of this infrastructure is user-owned?

Re: Cloud Storage for $2 per TB per month

#240

Post by ac29 »

ericflo wrote: post #226
krick wrote: post #167

I don't know anything about the subject, so no idea if these claims are realistic. But whatever, either they deliver or they don't. My (or their, actually) problem is I don't really get what they are offering right now. There is an impressive landing page with big numbers and pretty pictures which explains pretty much nothing. Project seems to be in production for at least 3 years, there are some apps, but I don't ac…

> but it cannot be true, right? It's just what they promise in the hypothetical future, not current price-tag? Here's the live pricing, right now: https://siastats.info/storage_pricing > Is it cheaper to implement it on blockchain than by conventional means? It's more so that anyone can join the network as a host. They don't have to have a financial or business relationship with anyone, they can just provide their st…

> It's more so that anyone can join the network as a host. They don't have to have a financial or business relationship with anyone, they can just provide their storage service and charge for it. No way to do that currently in the world without a blockchain.

Maybe I misunderstand your point, but I could certainly install MinIO (a S3 compatible object store) on a home NAS and charge people for it without using a blockchain. I see your point about not having a financial or business relationship with a blockchain network acting as an intermediary, but I can assure you that the IRS and various law enforcement and regulatory agencies would tell you that you absolutely do have a financial and business relationship with whoever is paying you via the crypto-network whether you'd like to or not.

Post reply on HN