Cloud Storage for $2 per TB per month • Hacker News Board

Re: Cloud Storage for $2 per TB per month

#211

Post by Dylan16807 » Sat, Mar 07, 2020, 1:14 AM UTC

arcticbull wrote: post #70

Taek wrote: post #36

Earlier quoted context omitted.

Author here. A lot of these numbers are drawn from experience in the mining world, where people realized that when cost is the ultimate bottom line, a lot of corners can be cut. Sia systems don't need a ton of networking. I ran the networking buildout costs by some networking people, and again it comes down to cutting corners. If you only need 10 gbps per rack, if you don't mind having extra milliseconds added, etc,…

Here's the issue. We know that due to economy of scale and domain experience, AWS will always have the lowest cost (to Amazon) for storage -- whether that's totally-reliable storage, or sorta-reliable. If there was a demand for sorta-reliable, they'd build a sorta-reliable S3 and undercut you. Then, blockchain adds inefficiency. Therefore, it's basically impossible for any blockchain solution to have a lower total co…

> If there was a demand for sorta-reliable, they'd build a sorta-reliable S3 and undercut you.

Amazon's goal is to get people to pay a premium to get access to their entire ecosystem of services. They don't optimize the price of each individual service. There's a lot of take it or leave it for their specific offerings. Look at how they used to have reduced redundancy as a cost saver, but don't anymore. If you want corner-cutting storage from Amazon, you get funneled into Glacier now.

So let's look at how glacier deep archive is $.99 per TB per month, with a waiting time. And Google's weird offering where it costs $1.23 per TB per month, with instant access, but you pay a bunch of money to access. That means they can't store it on tapes, but they expect to profit anyway. And they're probably not running that as a loss leader that depends on the archival data being accessed.

Re: Cloud Storage for $2 per TB per month

#212

Post by paulsutter » Sat, Mar 07, 2020, 1:14 AM UTC

ianopolous wrote: post #204

paulsutter wrote: post #195

Earlier quoted context omitted.

We stored hundreds of petabytes on cheap SATA drives with random fragment placement using reed solomon 6+3 coding (half the space of 3 replicas but same durability). Never lost a byte. Speed of recovery is crucial, because that’s your window of vulnerability to multiple failures. For example. try raid 5 on giant drives. The chances of losing a second drive during recovery is very likely.

No need to be rude. EDIT: The offensive part was removed What was the probability of failure of your drives? My guess is you just didn't hit the threshold for your failure rate. The maths checks out (PhD here). Seriously, do the calculation.

We lost drives all the time. In fact we moved so much data we needed checksums to avoid the 1e-13 undetected data errors.

We seriously did do the calculations (done by serious PhDs) and we seriously did not lose data.

I’m sure you are imagining a system that doesn’t work. But that doesn’t mean only a raidlike setup can work.

And by the way, could you explain how to calculate chance of data loss without taking recovery time into account?

Re: Cloud Storage for $2 per TB per month

#213

Post by kllrnohj » Sat, Mar 07, 2020, 1:15 AM UTC

execve12 wrote: post #193

> It also turns out that 32 HDDs only consume 200w, so the 750w PSU we picked is more than sufficient. Yep. Stopped reading right there. HDDs use ~15 watts when they boot up. I experienced this and I never allocate less than 20 watts / hard disk.

Staggered spin-up is a thing. That motherboard doesn't have it most likely, but they also can't plug 32 HDDs into it either. They'd need an HBA card of some kind, which is more likely to have staggered spin-up.

Re: Cloud Storage for $2 per TB per month

#214

Post by vincentlee » Sat, Mar 07, 2020, 1:16 AM UTC

throwaway9878 wrote: post #79

arcticbull wrote: post #70

Earlier quoted context omitted.

Here's the issue. We know that due to economy of scale and domain experience, AWS will always have the lowest cost (to Amazon) for storage -- whether that's totally-reliable storage, or sorta-reliable. If there was a demand for sorta-reliable, they'd build a sorta-reliable S3 and undercut you. Then, blockchain adds inefficiency. Therefore, it's basically impossible for any blockchain solution to have a lower total co…

Everybody with more money than you can always undercut you in anything you ever do so why bother ever trying to do anything

market your product better, steal better companies people, lie, play political games, leverage relationships, kill

so many options

Re: Cloud Storage for $2 per TB per month

#215

Post by samstave » Sat, Mar 07, 2020, 1:18 AM UTC

Taek wrote: post #36

kmod wrote: post #15

I worked on the design of Dropbox's exabyte-scale storage system, and from that experience I can say that these numbers are all extremely optimistic, even with their "you can do it cheaper if you only target 95% uptime" caveat. Networking is much more expensive, labor is much more expensive, space is much more expensive, depreciation is faster than they say, etc etc. I don't think the authors have ever done any actua…

Author here. A lot of these numbers are drawn from experience in the mining world, where people realized that when cost is the ultimate bottom line, a lot of corners can be cut. Sia systems don't need a ton of networking. I ran the networking buildout costs by some networking people, and again it comes down to cutting corners. If you only need 10 gbps per rack, if you don't mind having extra milliseconds added, etc,…

Why the heck would one want to store data in a non highly reliable anything run by a team who is building around cutting corners???

Wtf am i missing here?

—-

So, I assume you’re quite young. ~31 or so...

I say this because youre not speaking with a level of professionalism one would ostensibly have when talking about storing (and providing access to) another’s data...

And if your actual business model is focused on cutting corners to save costs in exchange for reliability - then the obvious is that you seek to cut corners and costs in all other aspects of your business and are telling people “STAY THE FUCK AWAY” from your business... unless your customers are of similar ilk, and if not - then their data isn't even important enough to persist, your systems or theirs...

Re: Cloud Storage for $2 per TB per month

#216

Post by wmf » Sat, Mar 07, 2020, 1:21 AM UTC

vikiomega9 wrote: post #41

kmod wrote: post #15

I worked on the design of Dropbox's exabyte-scale storage system, and from that experience I can say that these numbers are all extremely optimistic, even with their "you can do it cheaper if you only target 95% uptime" caveat. Networking is much more expensive, labor is much more expensive, space is much more expensive, depreciation is faster than they say, etc etc. I don't think the authors have ever done any actua…

> exabyte-scale storage system Somewhat of a random question, can you point me to some state of the art research?

https://blogs.dropbox.com/tech/2016/05/inside-the-magic-pock...

http://static.googleusercontent.com/media/research.google.co...

https://queue.acm.org/detail.cfm?id=1594206

http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Ke...

Re: Cloud Storage for $2 per TB per month

#217

Post by ianopolous » Sat, Mar 07, 2020, 1:21 AM UTC

ianopolous wrote: post #204

paulsutter wrote: post #195

Earlier quoted context omitted.

We stored hundreds of petabytes on cheap SATA drives with random fragment placement using reed solomon 6+3 coding (half the space of 3 replicas but same durability). Never lost a byte. Speed of recovery is crucial, because that’s your window of vulnerability to multiple failures. For example. try raid 5 on giant drives. The chances of losing a second drive during recovery is very likely.

No need to be rude. EDIT: The offensive part was removed What was the probability of failure of your drives? My guess is you just didn't hit the threshold for your failure rate. The maths checks out (PhD here). Seriously, do the calculation.

To clarify, the assumptions I'm making for the calculation are:

1) a Fixed probability of a server failing

2) a fixed erasure coding scheme used for all files

3) uncorrelated server failures

4) an erasure fragment is stored on a random server

Re: Cloud Storage for $2 per TB per month

#218

Post by ericflo » Sat, Mar 07, 2020, 1:21 AM UTC

system2 wrote: post #202

Ajedi32 wrote: post #103

Earlier quoted context omitted.

Sia is very reliable from the customer's perspective. Its only individual systems and storage pools that have lax uptime requirements. Thanks to some clever network-level redundancy mechanisms (10-of-30 redundancy), 95% uptime at the storage pool level translates to 99.9999% uptime from the customer's perspective. See the "Uptime Math" section in the OP for details.

It is not the answer to the question above. What is the type of customer uses this company?

Users of Sia are the customers. So your question reduces to: who uses Sia? It used to mostly be interesting for customers interested in reliable large-file data backup on the cheap, but now it has expanded to customers looking to do content distribution on the web, etc.

Re: Cloud Storage for $2 per TB per month

#219

Post by samstave » Sat, Mar 07, 2020, 1:24 AM UTC

system2 wrote: post #202

Ajedi32 wrote: post #103

Earlier quoted context omitted.

Sia is very reliable from the customer's perspective. Its only individual systems and storage pools that have lax uptime requirements. Thanks to some clever network-level redundancy mechanisms (10-of-30 redundancy), 95% uptime at the storage pool level translates to 99.9999% uptime from the customer's perspective. See the "Uptime Math" section in the OP for details.

It is not the answer to the question above. What is the type of customer uses this company?

Shady neckbeards.

Seriously, the type of people with the scruples which would piss you off both in their tactics and their profits.

Lest we forget where the fuck FB, Myspace, Friendster, HI5 all came from: unscrupulous spamsters etc.. many made lots of money, but they’re still douchey in their inceptions...

Re: Cloud Storage for $2 per TB per month

#220

Post by notyourday » Sat, Mar 07, 2020, 1:29 AM UTC

kllrnohj wrote: post #206

Taek wrote: post #36

Earlier quoted context omitted.

Author here. A lot of these numbers are drawn from experience in the mining world, where people realized that when cost is the ultimate bottom line, a lot of corners can be cut. Sia systems don't need a ton of networking. I ran the networking buildout costs by some networking people, and again it comes down to cutting corners. If you only need 10 gbps per rack, if you don't mind having extra milliseconds added, etc,…

The buildout in the article doesn't work. You can't plug in that 4-lane SAS SFF-8087 splitter cable into that motherboard. You're only getting 8 hard drives per motherboard with that setup, not 32. That puts the cost of 192 TB at more like $6240‬, not $4945. Could be less if you find a good deal on mini-SAS PCIE cards, but still going to be substantially higher than $4945.

You are going to get a 4U server that takes 60 disks and populate it with SAS cards driving them using SAS-2-SATA cables.