Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloud Storage for $2 per TB per month

blog.sia.tech

Post reply on HN
Print view View original

101–110 of 353 posts

Re: Cloud Storage for $2 per TB per month

#101

Post by WrtCdEvrydy »

simias wrote: post #61

>If we assume that the 30 hosts go offline independently I wonder how reasonable this assumption really is. For regular CPU-bound crypto-mining we see that it tends to centralize geographically in zones where electricity, workforce and real-estate space to build a datacenter are cheap. Assuming that Sia ends up following a similar distribution, it wouldn't be surprising if several of these hosts ended up sharing a si…

I'm guessing there's not a lot of 95% datacenters that don't have heavy generators or UPS on site. You'd have to basically build a datacenter that has lower guarantees.

Re: Cloud Storage for $2 per TB per month

#102

Post by Hamuko »

jknoepfler wrote: post #86
arcticbull wrote: post #70

Earlier quoted context omitted.

Here's the issue. We know that due to economy of scale and domain experience, AWS will always have the lowest cost (to Amazon) for storage -- whether that's totally-reliable storage, or sorta-reliable. If there was a demand for sorta-reliable, they'd build a sorta-reliable S3 and undercut you. Then, blockchain adds inefficiency. Therefore, it's basically impossible for any blockchain solution to have a lower total co…

having worked at aws, all I'll say is I think you're wildly over-estimating their agility

Care to share any funny stories?

Re: Cloud Storage for $2 per TB per month

#103

Post by Ajedi32 »

throwaway3157 wrote: post #93
Ajedi32 wrote: post #82

Earlier quoted context omitted.

They're not talking about a datacenter. Datacenters need to be reliable. Sia storage pools don't, because security and reliability is achieved at the global network level, not at the level of individual systems or storage pools. 95% reliability means you can be down for two whole weeks out of every year and still be well within acceptable uptime requirements. Generators? Who needs those? Just wait for the power to co…

Who are the customers for such unreliable systems?

Sia is very reliable from the customer's perspective. Its only individual systems and storage pools that have lax uptime requirements. Thanks to some clever network-level redundancy mechanisms (10-of-30 redundancy), 95% uptime at the storage pool level translates to 99.9999% uptime from the customer's perspective. See the "Uptime Math" section in the OP for details.

Re: Cloud Storage for $2 per TB per month

#104

Post by theamk »

I don't think it is correct to say that the only options are "host failures are truly independent" or "world war three".

The hosts are not ever going to be fully independent. There will be hundreds, if not thousands, host co-located in the same location -- likely of the cheapest grade, without any extras like fire alarms or halon extinguishers or redundant power feeds. A single fire (flood, broken power station) has a chance of taking out thousands of hosts simultaneously.

And there is management system as well -- AWS has thousands of engineers working on security. Will there be one at this super-cheap farm? What are the chances there will be farms with default passwords and password-less VNC connections? And since machines are likely to be cloned, any compromise affects thousands of hosts.

... and all of those things are made worse by the fact that if you store hundreds of thousands of files, your failure probability raises significantly. If a data center burns down, at least few of your files may be unlucky enough to be lost.

Re: Cloud Storage for $2 per TB per month

#105

Post by jessriedel »

nemonemo wrote: post #89
throwaway9878 wrote: post #79

Earlier quoted context omitted.

Everybody with more money than you can always undercut you in anything you ever do so why bother ever trying to do anything

I think the point is that the cost should not be the main motivator. I would agree that there needs to be other differentiators in addition to cost, which could provide sufficient moat against other competitors, big or small.

It's possible that there could be non-obvious innovations in how to save money with a low reliability threshold, which Amazon might not be able to effortlessly copy.

Re: Cloud Storage for $2 per TB per month

#107

Post by notyourday »

walrus01 wrote: post #75
notyourday wrote: post #69

Earlier quoted context omitted.

Backblaze is very good but they are definitely not efficient in $$ utilization. Efficient $$ utilization is bread racks, built out data centers abandoned by the likes of PepBoys that landlords will part for $3/sq foot per year and Google using servers without cases and velcros to keep hard drives attached.

Abandoned pepboys stores don't usually have very good fiber connectivity. Backblaze and similar hosting/storage companies move enough traffic that they need to be topologically close to major IX points. If all you want is cheap commercial real estate with cheap dollars per square foot figures, there are plenty of economically depressed areas within the United States that you could put things. Those areas usually have…

> Abandoned pepboys stores don't usually have very good fiber connectivity. Backblaze and similar hosting/storage companies move enough traffic that they need to be topologically close to major IX points.

Not stores. Data centers.

It is typically cheaper to get long distance fiber links than metro fiber and midsize data centers do not consume that much power, power which is plentiful outside the major metros, especially in the old manufacturing areas.

The real reason why companies do not go there is because it is not sexy and non-sexy places do not get "ninja" employees that would be passing brain teasers on a whiteboard.

> Bulldozing empty land in Quincy and putting up a tilt-up concrete on slab dedicated purpose datacenter structure is much less costly than extensively retrofitting abandoned, 30, 40, 50 year old commercial real estate.

If you are doing it in a major metro then unless you get a big fat tax break your real estate taxes are going to kill you.

Re: Cloud Storage for $2 per TB per month

#109

Post by tracker1 »

zaroth wrote: post #96
Tepix wrote: post #27

Earlier quoted context omitted.

Using "8x 4 way SATA data splitters". What i don't get is why they don't use 14TB HDDs, they are only 15% more expensive per TB. On the other hand they'd need 2.33x less PCs at $550 each, plus their power use. So instead of every 7 PCs with 6TB HDDs they'd need 3 with 14TB HDDs. PS: They could also use a mainboard with 10 SATA ports instead of 8. They are only $15 more than the chosen board. Adding one or more PCIe 8…

> Using "8x 4 way SATA data splitters". There's no such thing as a passive SATA data splitter.

it's probably a SAS/Sata controller, and the sas interface split to 4x sata.

Re: Cloud Storage for $2 per TB per month

#110

Post by kmod »

arcticbull wrote: post #70
Taek wrote: post #36

Earlier quoted context omitted.

Author here. A lot of these numbers are drawn from experience in the mining world, where people realized that when cost is the ultimate bottom line, a lot of corners can be cut. Sia systems don't need a ton of networking. I ran the networking buildout costs by some networking people, and again it comes down to cutting corners. If you only need 10 gbps per rack, if you don't mind having extra milliseconds added, etc,…

Here's the issue. We know that due to economy of scale and domain experience, AWS will always have the lowest cost (to Amazon) for storage -- whether that's totally-reliable storage, or sorta-reliable. If there was a demand for sorta-reliable, they'd build a sorta-reliable S3 and undercut you. Then, blockchain adds inefficiency. Therefore, it's basically impossible for any blockchain solution to have a lower total co…

I agree that AWS has economies of scale that makes it hard to build a better/cheaper S3, but one way you can get around this problem is by building to lesser requirements. S3 has to work for all use cases, but if you know you need less of something costly (say, less IOPs) you can build a system that's cheaper than S3 even if the individual components are more expensive than S3 is paying.
Post reply on HN