Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sia – Decentralized cloud storage network

blog.sia.tech

Post reply on HN
Print view View original

81–90 of 199 posts

Re: Sia – Decentralized cloud storage network

#81

Post by Taek »

neiman wrote: post #68

I tried Sia about 2 years ago. It worked "ok" for half a GB files, However, for 4-5 GB my upload got stuck while the help forum couldn't provide answers besides "wait for the next release". There were also unclear issues back then. Are the files encrypted on my computer before being uploaded? are they sharded between many hosts? or are they stored in one host? How long will they be stored. Most of these issues seem l…

Sia has come a long way in the past two years. Uploads, downloads, and repairs are both much more stable and also a lot faster. Sia can effortlessly sustain 100mbps upload and download if your connection is that fast.

Sia is happy with files that are hundreds of GBs in size today, and with a filesystem that that gets up to about 20 TB. Beyond that it starts to struggle, but we're continuously working to expand its capabilities.

Data has always been encrypted client-side, even in very early releases. Data today is sharded between hosts in a 10-of-30 Reed-Solomon scheme.

I think you will find that the experience is much smoother and more complete compared to the experience from 2 years ago. And, a critical feature is now available as of the latest release: seed based file recovery. After you upload your files, it's possible to create a snapshot of your data that you can recover later on a different machine using nothing more than your wallet seed. This makes Sia a practical and cost effective solution for secondary or tertiary backup.

Re: Sia – Decentralized cloud storage network

#82

Post by Taek »

rjeli wrote: post #75
wmf wrote: post #55

Earlier quoted context omitted.

Unless SGX works.

SGX has always seemed like the Netflix DRM problem to me.. eventually, pixels have to show up on a screen somewhere, right? Am I mistaken in this line of thinking? Homomorphic encryption solves this, but it’s throwing an awful lot of performance out the window.

I think the idea behind SGX based cloud computing would be that you send encrypted data and code to the enclave, which then decrypts the data, does the computation, encrypts the result, and then returns the encrypted result.

The enclave gets to see everything that's going on, but in theory nothing can view or corrupt what's going on inside of the enclave.

I'm personally pretty skeptical that SGX is a secure platform, however if you do decide that SGX works and is trustworthy, it would be possible to outsource your computation in a mostly cost effective way to anyone who has an SGX enclave.

Re: Sia – Decentralized cloud storage network

#84

Post by toomuchtodo »

buildzr wrote: post #69
pmlnr wrote: post #62

Earlier quoted context omitted.

2x3Tb disk at hetzner, line rental, electricity included is 30€pcm.

Depends largely on your bandwidth usage if that's beneficial, for me personally I'm talking home/small business backups where the additional reliability of B2 is great and the price actually works out better than putting that system in RAID1.

(EDIT: PSA) Remember, redundancy (RAID) is not a replacement for backups.

Re: Sia – Decentralized cloud storage network

#85

Post by PlaneSploit »

pmlnr wrote: post #59
buildbuildbuild wrote: post #42

Abuse concerns and “does it need a blockchain” cynicism aside I find all of these “crypto” cloud computing projects fascinating. I highly recommend playing around on their testnets. Storj and Filecoin are also competing in the storage space, and BitTorrent is vying to be a CDN alternative. SONM aims to be an EC2 competitor, and interestingly requires ID verification to participate in their testnet due to the clear ab…

Storj was a brilliant idea then suddenly someone put blockchain into it.

Blockchain is just for payments, the actual object store doesn't use it

Re: Sia – Decentralized cloud storage network

#86

Post by AcerbicZero »

toomuchtodo wrote: post #41
idlewords wrote: post #40

Earlier quoted context omitted.

Except for the software developers.

So exhausting watching people argue that trust is no longer necessary when they're simply pushing the trust around. You have to trust someone at the end of the day, and it's more practical to use the law to trust them (and enforce that trust) than cryptography. If I want to decentralize my storage, I would replicate data between multiple cloud object stores (Backblaze, S3, Wasabi, etc, a library to abstract PUTs and…

I think its great that most of the questions here are getting answered, but I'm still a little confused as to who this is for, and why adding bitcoins great great grand child to the project helps.

It doesn't look like its built for large data sets, since the capacity of the network is fairly low, and the providers fairly small. So if I want to store ~5TB of data, this isn't much of a fit and if I want to store ~5GB of data well there are tons of options and if I want it replicated, I'll replicate it.

Seriously though, I don't want to play game theory games with a list of storage providers, especially for tiny sums of data, I just want it to be there.

Re: Sia – Decentralized cloud storage network

#87

Post by SimianLogic2 »

Taek wrote: post #50
SimianLogic2 wrote: post #25

Can someone ELI5 why you need a blockchain for this? Another startup could make a non-blockchain version of this that uses similar syncing technology to GDrive / Dropbox file sync that just syncs other people's stuff instead of yours and pays you monthly.

Founder here. The blockchain gets you a couple of things. The key thing in our blockchain is the storage contract, which is something like a blockchain-SLA. The host has an obligation to store some data, and the host also puts up money out of pocket as a promise that the data will be stored faithfully. The blockchain will occasionally challenge the host, and if the host cannot create a proof that the data is still be…

How do renewals work? With S3 I upload something and then pay for it + bandwidth until I delete it. Would I need to know how long I need storage for ahead of time? Estimated bandwidth?

I run a video rendering service that currently uploads previews (call it 5-10mb) to S3 for ~3 days and then deletes them. Finished renders (~50-200mb) are stored for 2 weeks. I upload 500-1000 new files a day.

Amazon just bills me for this once a month. Would I need to load up a wallet and prepay each of these contracts individually? How would I send out a download link or play a video in a browser?

Re: Sia – Decentralized cloud storage network

#88

Post by Taek »

nisten wrote: post #73

There is one missing piece to all this, I/O. Given they publish their apps on all platforms they could probably add this in the future as an ssd or PCIe storage tier by doing regular disk benchmarks. However it seems like a major limiting factor, given that random access speeds on spinning drives or even cheap ssds can drop drastically by an order or two of magnitude (i.e. from 130MB/s to 2MB/s). Given how they're do…

Most 4K video is closer to 50mbps. If your home connection is that fast, Sia should be more than capable of delivering smooth 4K streaming to your home.

A single host with slow storage is not a blocking factor because Sia downloads in parallel from as many as 30 hosts at once when composing a stream of data. Even if each has an I/O bottleneck of 1 MB/s (8mbps), you'll be able to get your video out at 240mbps.

In practice today, Sia hosts are almost all network bottlenecked, most use HDDs which do not have trouble keeping up with network latencies (HDD seek times are 11ms, network latencies usually 50ms+) or throughputs (HDDs can usually get close to 100 MB/s, far more than the network throughput)

Can I ask, have you tried streaming 4K video from the latest version of Sia? Seek times might be 5-10 seconds, but I believe that once your stream has loaded you will see completely smooth playback. Our next release has a focus on reducing those seek times, but the code is not ready yet.

Re: Sia – Decentralized cloud storage network

#89

Post by wmf »

siavosh wrote: post #83

Can someone explain the difference between Sia vs Filecoin/IPFS. The latter seems to be still working on solving some fundamental crypto primitives before they can offer what seems like what Sia already claims to have solved?

Yes, Filecoin is trying to come up with a "more efficient" and possibly "elegant" implementation based on algorithms that are still being invented. Of course, with 100x the budget it's possible that they could still overtake Sia.

Re: Sia – Decentralized cloud storage network

#90

Post by Taek »

jchw wrote: post #74

> Today it is possible to purchase a refurbished 1 TB hard drive for about $20 — so buyers are essentially paying Amazon each month the entire cost of the hard drives used to store their data! What a refurbished hard disk does not give you: - Redundant storage - Redundant power/network/etc. - Geographic distribution - A network API - Any networking, or uplinks at all - Access control - A team ensuring the reliability…

One of the main innovations of Sia is providing a low cost means to turn that 1 TB refurbished hard drive into a useful cloud product.

Sia is able to provide, at 1/10th the cost of Amazon, high throughput, high reliability, high redundancy, geographically distributed cloud storage, and it can do that without high reliability drives or datacenters.

Post reply on HN