Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Libgen Storage Decentralization on IPFS

freeread.org

Post reply on HN
Print view View original

71–80 of 115 posts

Re: Libgen Storage Decentralization on IPFS

#71

Post by eMGm4D0zgUAVXc7 »

chriswarbo wrote: post #55
derefr wrote: post #45

Earlier quoted context omitted.

If you take IPFS's model and add inherent anonymity, that's exactly what Freenet is. (To this day I'm confused about why IPFS is popular while Freenet isn't.)

I've not tried Freenet for a few years, but every time I did it was very slow; e.g. browsing basic HTML sites hosted on Freenet was quite painful. IPFS can serve streaming video, and host massive archives like Wikipedia. Also, Freenet actively seeks out data from the network to store locally, to give plausible deniability to users (there's no way to tell if something is locally cached because the user requested it, o…

> I've not tried Freenet for a few years, but every time I did it was very slow

It has become a lot faster recently, try again :) When I look at its statistics during active usage it can easily show 1 MiB/s traffic.

You still can't expect HTML to load in sub-second delays, for large multi-MiB sites it may take a low single-digit amount of seconds, but there is an inherent cost to anonymity which probably induces a boundary on how fast things can be:

To achieve anonymity, data needs to be redirected across multiple people so the sender and recipient can't determine who each other of them is.

Redirecting stuff across a longer path than necessary slows things down.

Re: Libgen Storage Decentralization on IPFS

#72

Post by georgyo »

josu wrote: post #30
diggan wrote: post #24

Earlier quoted context omitted.

Protocols don't have attitudes, everything you're using IPFS for you could also use BitTorrent for (almost). The protocols don't really care about what data you send, and no one can stop you from using them, BT or IPFS. The CoC you've linked covers the IPFS Community and other Protocol Labs initiatives, it doesn't cover usage of IPFS itself. freeread.org is not bound by that CoC.

The protocol can't, but the network can. https://discuss.ipfs.io/t/mirror-of-sci-hub-in-ipfs/1613

They are specifically asking for help and guidance to break the law. Any public community has to avoid those questions like the plague.

Remember youtube-dl got taken down because it's README file referenced copyrighted material.

It doesn't matter if a tool _could_ be used distribiting copyright material. The tool (and communities) cannot promote anything related to that use case at all.

The same is true with bittorrent. Bittorrent _could_ be used to download copyrighted music and movies, but you won't find a single reference to anything like that use case on their web page because that is illegal: https://www.bittorrent.com/

In fact the bittorrent terms of service says you may not use the tool for any illegal purposes: https://www.bittorrent.com/legal/terms-of-use/

Re: Libgen Storage Decentralization on IPFS

#73

Post by m-p-3 »

diggan wrote: post #25
elvis70 wrote: post #23

Earlier quoted context omitted.

Ah, thanks for mentioning that. If I understand well, it's a distributed filesystem but there is still an authority able to ban users or contents?

The answer for both BitTorrent and IPFS is: No.

They can blacklist hashes on the ipfs.io reverse proxy, but they cannot block you from accessing those through a different node like your own.

Re: Libgen Storage Decentralization on IPFS

#74

Post by eMGm4D0zgUAVXc7 »

chriswarbo wrote: post #68
petra wrote: post #5

What's missing from this system(and P2P systems in general), is automatic pinning - i.e. the system automatically choses which files to pin on which user, to ensure every file is available on the net. It's a bit of tricky problem, because people may not like unknown, random files on their pc, and probably, because such algorithm is distributed and will need to resist attacks. But i think that the payoff is big: what…

See the comment above about Freenet, which does something like this; although according to https://freenetproject.org/pages/documentation.html#content it seems to prioritise popular files rather than rare ones. IIRC eDonkey prioritised rare files, although I don't know if this was a property of the network or just a common choice in clients.

There is a Freenet "KeepAlive" plugin which can be used to selectively pin files which you want to preserve, even if they are completely unpopular.

Re: Libgen Storage Decentralization on IPFS

#75

Post by justarandomq »

mpfundstein wrote: post #60
justarandomq wrote: post #51

Is LibGen safe? I remember testing some of its mirror links on virustotal and saw some red flags, so I decided not to risk downloading.

what red flags did you see?

It was some book, and when submitting the download link, 3 of the antivirus results were red. Perhaps they were false positives.

Re: Libgen Storage Decentralization on IPFS

#76

Post by magila »

georgyo wrote: post #42
magila wrote: post #39

Earlier quoted context omitted.

This has been the state of IPFS for years now. The protocol and client haven't been developed to where they can work reliably in a real P2P scenario. IPFS only appears to work because most requests go strait to one of the big gateways which already has the content cached. Development of IPFS slowed to a crawl when most of Protocol Labs switched to working on Filecoin. It remains to be seen if either Filecoin or IPFS…

I'm not sure you are right about this. The following link will stream big buck bunny directly from the network with full P2P support. It is not using a gateway node for the video content. https://bbb.fu.io/ You can run the following in the console to see all your peers for await (const peer of await node.swarm.peers()) { console.log(peer.addr.toString()) } The console currently get's spammed pretty hard because they…

For me at least the vast majority of the data is being downloaded from various servers under bootstrap.libp2p.io or preload.ipfs.io. These are clearly dedicated servers not true peers.

The contact info for these servers appears to be retrieved via the DNS rather than the DHT. This isn't surprising because browser clients can't participate in the DHT, but the DHT is currently one of the biggest weak points of IPFS. IPFS's DHT is so chatty that it consumes over a megabit of throughput just to maintain an idle node. Lookups are excruciatingly slow and often fail completely.

Re: Libgen Storage Decentralization on IPFS

#77

Post by whimsicalism »

nemo wrote: post #69
whimsicalism wrote: post #64

Earlier quoted context omitted.

> the (very) common scenario is to get a couple warnings from your ISP and if you continue torrenting copyrighted material you will be dropped as a customer and banned. This is not "very" common - the ISPs don't really ban you, they just threaten you. I no longer do this, but I extensively torrented over multiple home ISPs and they would send me warnings but they would never actually do anything.

AT&T has banned a friend permanently so I would be careful about over-broad generalizations. Folks in my my household were torrenting and we got our service cut with the second offense until we went through some rigmarole promising we'd never do it again at which point they stopped. The cost to an ISP's lawyers to field these things isn't free and at least some really do respond with action.

I'm curious if the action taken depends on broadband competition in the area - do you live in a city?

Re: Libgen Storage Decentralization on IPFS

#78

Post by georgyo »

magila wrote: post #76
georgyo wrote: post #42

Earlier quoted context omitted.

I'm not sure you are right about this. The following link will stream big buck bunny directly from the network with full P2P support. It is not using a gateway node for the video content. https://bbb.fu.io/ You can run the following in the console to see all your peers for await (const peer of await node.swarm.peers()) { console.log(peer.addr.toString()) } The console currently get's spammed pretty hard because they…

For me at least the vast majority of the data is being downloaded from various servers under bootstrap.libp2p.io or preload.ipfs.io. These are clearly dedicated servers not true peers. The contact info for these servers appears to be retrieved via the DNS rather than the DHT. This isn't surprising because browser clients can't participate in the DHT, but the DHT is currently one of the biggest weak points of IPFS. IP…

Sorta, a peer can advertise dns addresses, and the browser can participate in the dht as a client. The example above uses webrtc-star for browser to browser discovery.

    $ ipfs dht findpeer 12D3KooWJxNHY6zE1KnzFdBJrTMCbhCVNtSLvjp7qR5Wsp49DnUC
    /dnsaddr/ipfs.scalable.io
    /dns4/ipfs.scalable.io/udp/4001/quic
    /dns6/ipfs.scalable.io/tcp/443/wss
    /dns6/ipfs.scalable.io/udp/4001/quic
    /dns4/ipfs.scalable.io/tcp/443/wss

Re: Libgen Storage Decentralization on IPFS

#79

Post by fsflover »

derefr wrote: post #45
fsflover wrote: post #3

I think such materials should better be placed to darknets protecting the privacy of users and seeders, e.g., to I2P.

If you take IPFS's model and add inherent anonymity, that's exactly what Freenet is. (To this day I'm confused about why IPFS is popular while Freenet isn't.)

What is the point of Freenet when we have I2P, which allows any protocol on top of it, not just file sharing?

Re: Libgen Storage Decentralization on IPFS

#80

Post by bawolff »

eMGm4D0zgUAVXc7 wrote: post #71
chriswarbo wrote: post #55

Earlier quoted context omitted.

I've not tried Freenet for a few years, but every time I did it was very slow; e.g. browsing basic HTML sites hosted on Freenet was quite painful. IPFS can serve streaming video, and host massive archives like Wikipedia. Also, Freenet actively seeks out data from the network to store locally, to give plausible deniability to users (there's no way to tell if something is locally cached because the user requested it, o…

> I've not tried Freenet for a few years, but every time I did it was very slow It has become a lot faster recently, try again :) When I look at its statistics during active usage it can easily show 1 MiB/s traffic. You still can't expect HTML to load in sub-second delays, for large multi-MiB sites it may take a low single-digit amount of seconds, but there is an inherent cost to anonymity which probably induces a bo…

> To achieve anonymity, data needs to be redirected across multiple people so the sender and recipient can't determine who each other of them is.

>

> Redirecting stuff across a longer path than necessary slows things down.

That's skipping past a lot of details. There are still choices there. Freenet traded speed, efficiency and scalsbility for a specific security model. There are anonoyminity systems (e.g. tor) that are much more efficient but have different trade-offs.

Post reply on HN