Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Filecoin: Proof of Storage Systems

blog.coinlist.co

Post reply on HN
Print view View original

31–40 of 53 posts

Re: Filecoin: Proof of Storage Systems

#31

Post by filleokus »

I don't really get the point of decentralised _storage_ solutions, where the stated goal is to compete with ≈ S3 [0].

S3 storage pricing, for most users is just a rounding error compared to e.g egress or compute cost. And it has proven to be tremendously reliable, both in durability and access. No sane person would point a CDN against a Filecoin backed origin, right?

For kinda-censorship resistant distribution of stuff like Wikileak dumps we already have torrents right now, where the seeders are aware of the content they spread.

For uses where S3 is not competitive, like storage of my media rips, spinning rust at places like Hetzner exist, or Backblaze et. al for more casual backup uses.

The only point I can see for decentralised "storage" of this kind is for ultra high durability, small data size, almost never read, almost some sort of "stakes". One could imagine something like storing contracts/land deeds, similar to Gwern's timestamping of URL's. The files should be small, the storage model optimised for maximal durability (every node store's almost everything?), and prices high enough to ensure durability.

[0]: Just look at this PR page https://filecoin.io/store/ [1]: https://www.gwern.net/Timestamping

Re: Filecoin: Proof of Storage Systems

#32

Post by rudolph9 »

acdha wrote: post #25
rudolph9 wrote: post #16

Earlier quoted context omitted.

There are a lot of nuanced ways a trustless distributed system can be more efficient. The article is pretty light on details but consider the content addressing mechanism. The data can be securely served by any device and independently verified by any recipient. The minimum latency of reads is theoretically much lower than the sever/client infrastructure commonly used today. It’s like edge computing with any device t…

> The minimum latency of reads is theoretically much lower than the sever/client infrastructure commonly used today. This is an oversimplification: it assumes that discovery is very low cost and that there's a peer with a copy enough closer on the network that it's faster than talking to server-class hardware in a data center with a high quality network connection. Given the number of assumptions which need to be tru…

> Obviously a lot goes into ensuring the data existing on a low latency connection to the distributed-client

This is admittedly a non-trivial task but perhaps an example can shed light on way it can be more efficient in practice.

Consider the Apple Photos app. It's on your phone, it's on your desktop, it's on the cloud (iCloud storage). Say you have so many photos they don't fit on your phone. When you try to access a photo that is not on your phone it gets it from icloud (these are literally features that are currently offered by photos/iCloud). Now suppose your desktop has lots of storage, you usually look at your photos while connected to your local network, why cant you get the data from there?

The key to the efficiency of file coin is not some magic cryptographic token, it's not even "blockchain", the key is the underlying protocols being unified for synchronizing devices across multiple tiers of networking. Filecoin mostly just makes sure the data doesn't disappear and gives you an optional substitute for cloud storage. Content addressing, p2p communication by default, flexible/future-proof standards for negotiated what data is and how it is formatted, and in modular plug-and-play utilization of many different communication/synchronization techniques is the key to applications being built in more effective and efficient way than conventional client/server architecture.

The word that comes to mind is "flexibility". Want you store to be S3 buckets? Great! Want to host a private network, only your devices can access? Great! Want to joint the public network? Great! Want to ensure persistence of data via a token? Great! Want to just use the address mechanism? Great! Want to run the software in the cloud, on you local machine, in a sandboxed browser tab and have them all seamlessly forming a p2p network over a multitude of network standards (tcp, udp, web-sockets, etc)? Great!

I worked extensively last year with various ipfs techs utilized in a private p2p network consisting of ARM nodes capture time-series measurement devices and cloud based persistent nodes managing the pinset of data and persisting long-term storage. There are definitely some rough edges but again it's open-source built is such a modular way it's almost difficult to develop your self into a corner.

The software coming out of protocol labs is overwhelmingly not new technology. Protocol lab large focuses on utilization of battle-tested existing standards and tech (many of which are relatively ancient like TCP, ssh, git, json, etc) unified, without breaking existing stands, via a common addressing mechanism.

tl;dr Filecoin/crypto-tokens just a side note! The vast majority of software coming out of protocol labs is independently useful and largely focuses on bridging together widely-used battle-tested tech.

Re: Filecoin: Proof of Storage Systems

#33

Post by rudolph9 »

zucker42 wrote: post #28
rudolph9 wrote: post #16

Earlier quoted context omitted.

There are a lot of nuanced ways a trustless distributed system can be more efficient. The article is pretty light on details but consider the content addressing mechanism. The data can be securely served by any device and independently verified by any recipient. The minimum latency of reads is theoretically much lower than the sever/client infrastructure commonly used today. It’s like edge computing with any device t…

Is this really an advantage of the fact that the system is trustless and the ownership of storage media is distributed? You can an imagine a system where Amazon owns all the hard drive but still distributes/duplicates files across many different hard drives, matching the theoretical performance characteristics of Filecoin. In fact storing data on S3 from my cursory understanding already distributes data across multip…

The advantage is the flexibility. I kind of went on a rant about this [here][1] but to answer your question more directly; the advantage is how the flexibility the addressing mechanism provides. Blockchain-tokens are frankly kind of boring, they solve the double spend problem. What's interesting is the trustless distributed system tech is being developed around them!

[1]: https://news.ycombinator.com/item?id=24218074

Re: Filecoin: Proof of Storage Systems

#34

Post by acdha »

rudolph9 wrote: post #32
acdha wrote: post #25

Earlier quoted context omitted.

> The minimum latency of reads is theoretically much lower than the sever/client infrastructure commonly used today. This is an oversimplification: it assumes that discovery is very low cost and that there's a peer with a copy enough closer on the network that it's faster than talking to server-class hardware in a data center with a high quality network connection. Given the number of assumptions which need to be tru…

> Obviously a lot goes into ensuring the data existing on a low latency connection to the distributed-client This is admittedly a non-trivial task but perhaps an example can shed light on way it can be more efficient in practice. Consider the Apple Photos app. It's on your phone, it's on your desktop, it's on the cloud (iCloud storage). Say you have so many photos they don't fit on your phone. When you try to access…

> Consider the Apple Photos app. It's on your phone, it's on your desktop, it's on the cloud (iCloud storage). Say you have so many photos they don't fit on your phone. When you try to access a photo that is not on your phone it gets it from icloud (these are literally features that are currently offered by photos/iCloud). Now suppose your desktop has lots of storage, you usually look at your photos while connected to your local network, why cant you get the data from there?

Yes, what Dropbox, Crashplan, etc. offered a decade ago — it's a neat sounding idea but there are two reasons why this isn't a huge win in practice: people are mobile so the number of times where you need an uncached file and happen to be on the same network is relatively low and increasingly few people have an always-on device with a ton of local storage free (phones are really good at generating high volumes of data so this is a non-trivial problem).

Working on things like this is really interesting but it involves a lot of work to handle unreliable clients or networks and bitrot (hashes don't solve this if your client helpfully replicates the bad sector from your desktop over the pristine copy on your phone). That overhead makes it a lot harder to beat conventional services, especially in cases where the time investment is greater than the possible savings.

Re: Filecoin: Proof of Storage Systems

#35

Post by dfischer »

simonebrunozzi wrote: post #29
zepearl wrote: post #24

General question: when somebody will push into Filecoin some "bad" files (anything, from docs needed by the mafia to children being sexually abused, to fake videos/claims/whatever about you, to any other bad things you can think of), how can that be detected and then deleted? (ignoring here the theme related to accountability) Question based on https://docs.filecoin.io/introduction/why-filecoin/ : Filecoin resists ce…

I think this is a serious problem - but probably not for the exact use case you list (mafia, sexually abused kids). It should be solved by allowing an "expensive" (a) "vote" (b) to override, or delete, or hide for a number of years. (a): expensive should discourage the option in general (b): proof-of-stake, proof-of-ownership, proof-of-work are some of the many ways in which you can define voting rights. Unfortunatel…

Too abusable... state actors, etc... could control network and remove stuff by spinning up probabilities to inject within quorum’s.

Not sure...

Re: Filecoin: Proof of Storage Systems

#36

Post by mifeng »

wmf wrote: post #13
mifeng wrote: post #10

This article seems like it was written in 2017, not 2020. Filecoin raised over $200m and hasn't shipped anything that I'm aware of in 3 years. Maybe it will be revolutionary, but the crypto industry has long moved on from white whale projects that don't ship.

https://github.com/filecoin-project/lotus https://docs.filecoin.io/how-to/install-filecoin/

This is what I'm talking about. It's been 3 years, and they're still doing testnets? What a joke.

Re: Filecoin: Proof of Storage Systems

#37

Post by macklemoreshair »

filleokus wrote: post #31

I don't really get the point of decentralised _storage_ solutions, where the stated goal is to compete with ≈ S3 [0]. S3 storage pricing, for most users is just a rounding error compared to e.g egress or compute cost. And it has proven to be tremendously reliable, both in durability and access. No sane person would point a CDN against a Filecoin backed origin, right? For kinda-censorship resistant distribution of stu…

Agreed the #1 issue with file coin from day one has been the idea that the value of a coin is tied to anything at all. It puts a limit on the imaginative value because it’s tied to something as commoditized as bulk storage

Re: Filecoin: Proof of Storage Systems

#38

Post by thefounder »

filleokus wrote: post #31

I don't really get the point of decentralised _storage_ solutions, where the stated goal is to compete with ≈ S3 [0]. S3 storage pricing, for most users is just a rounding error compared to e.g egress or compute cost. And it has proven to be tremendously reliable, both in durability and access. No sane person would point a CDN against a Filecoin backed origin, right? For kinda-censorship resistant distribution of stu…

Fuck S3 and all the lords gating the internet. I think that's what decentralised web wants to say and I'm pretty much on its side.

Re: Filecoin: Proof of Storage Systems

#39

Post by [deleted] »

rudolph9 wrote: post #32
acdha wrote: post #25

Earlier quoted context omitted.

> The minimum latency of reads is theoretically much lower than the sever/client infrastructure commonly used today. This is an oversimplification: it assumes that discovery is very low cost and that there's a peer with a copy enough closer on the network that it's faster than talking to server-class hardware in a data center with a high quality network connection. Given the number of assumptions which need to be tru…

> Obviously a lot goes into ensuring the data existing on a low latency connection to the distributed-client This is admittedly a non-trivial task but perhaps an example can shed light on way it can be more efficient in practice. Consider the Apple Photos app. It's on your phone, it's on your desktop, it's on the cloud (iCloud storage). Say you have so many photos they don't fit on your phone. When you try to access…

[deleted]

Re: Filecoin: Proof of Storage Systems

#40

Post by rudolph9 »

acdha wrote: post #34
rudolph9 wrote: post #32

Earlier quoted context omitted.

> Obviously a lot goes into ensuring the data existing on a low latency connection to the distributed-client This is admittedly a non-trivial task but perhaps an example can shed light on way it can be more efficient in practice. Consider the Apple Photos app. It's on your phone, it's on your desktop, it's on the cloud (iCloud storage). Say you have so many photos they don't fit on your phone. When you try to access…

> Consider the Apple Photos app. It's on your phone, it's on your desktop, it's on the cloud (iCloud storage). Say you have so many photos they don't fit on your phone. When you try to access a photo that is not on your phone it gets it from icloud (these are literally features that are currently offered by photos/iCloud). Now suppose your desktop has lots of storage, you usually look at your photos while connected t…

> Working on things like this is really interesting but it involves a lot of work to handle unreliable clients or networks and bitrot (hashes don't solve this if your client helpfully replicates the bad sector from your desktop over the pristine copy on your phone). That overhead makes it a lot harder to beat conventional services, especially in cases where the time investment is greater than the possible savings.

The idea that ipfs/related-project are trying to beat conventional services is big misconception. The overwhelming majority of the tech does not preclude usage in conventional services. For example, IPFS can 100% be configured as server/client offering roughly the same costs/reliability many are used to. The advantage is interoperability with many different "services", conventional and alternative (e.g. filecoin) alike.

Post reply on HN