Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Libgen Storage Decentralization on IPFS

freeread.org

Post reply on HN
Print view View original

11–20 of 115 posts

Re: Libgen Storage Decentralization on IPFS

#11

Post by traverseda »

segfaultbuserr wrote: post #4

The IPFS mirror really gives a substantial performance boost. In the past, the files had to be fetched from a tired server far away in Russia, the speed can be as low as 0.5 Mbps in my experience (depends on your ISP and connectivity), now it can be lightning fast thanks to P2P, 2 Mbps+, more than adequate to download a 100 MiB+ file. You don't even have to install IPFS (but you probably should, to maintain decentral…

In my experience IPFS is only fast when you're using it through something like Cloudflare's IPFS proxy (which is basically a caching proxy). I haven't found IPFS to be actually usable any time I've tried running a node myself.

Is that unusual?

Re: Libgen Storage Decentralization on IPFS

#13

Post by segfaultbuserr »

dooglius wrote: post #12

A strange choice given that distributing copyrighted materials violates the IPFS Code of Conduct [0]. Why not use BitTorrent whose attitude toward copyright is much more in sync with libgen's? [0] https://github.com/ipfs/community/blob/master/code-of-conduc...

LibGen already has "torrent per 1000 files" for a long time, some books also have their individual torrents but it's not common. I'm not sure why, but I think the problem is that files on BitTorrent cannot be directly addressed by hashes, it had to go through an intermediate step called a "torrent", which does not uniquely identify individual files directly. You can always select the one file you need from a huge torrent, but it's not as convenient as directly addressing files by their hashes.

Re: Libgen Storage Decentralization on IPFS

#14

Post by mathnmusic »

segfaultbuserr wrote: post #4

The IPFS mirror really gives a substantial performance boost. In the past, the files had to be fetched from a tired server far away in Russia, the speed can be as low as 0.5 Mbps in my experience (depends on your ISP and connectivity), now it can be lightning fast thanks to P2P, 2 Mbps+, more than adequate to download a 100 MiB+ file. You don't even have to install IPFS (but you probably should, to maintain decentral…

And what about when browsers (like Brave) start supporting IPFS natively, and gateways are not needed any more?

Re: Libgen Storage Decentralization on IPFS

#15

Post by corey_moncure »

petra wrote: post #5

What's missing from this system(and P2P systems in general), is automatic pinning - i.e. the system automatically choses which files to pin on which user, to ensure every file is available on the net. It's a bit of tricky problem, because people may not like unknown, random files on their pc, and probably, because such algorithm is distributed and will need to resist attacks. But i think that the payoff is big: what…

If the pinning came with encryption where the key is not known to the pinned system, I think most would be OK with it. There's no way to know, or demonstrate that anyone could know, the contents of any given file. Furthermore, the file could be broken into encrypted chunks and distributed redundantly to many users. Kind of like BT turned inside out.

Re: Libgen Storage Decentralization on IPFS

#16

Post by bboygravity »

fsflover wrote: post #3

I think such materials should better be placed to darknets protecting the privacy of users and seeders, e.g., to I2P.

I agree. Actually in this day and age I think all information sharing should better be placed on darknets. I don't see the net advantage to anybody of being profiled constantly on clearnet.

Naive question: is there such a thing as Bittorrent over I2P or otherwise anonymized decentralized file sharing?

Re: Libgen Storage Decentralization on IPFS

#17

Post by segfaultbuserr »

traverseda wrote: post #11
segfaultbuserr wrote: post #4

The IPFS mirror really gives a substantial performance boost. In the past, the files had to be fetched from a tired server far away in Russia, the speed can be as low as 0.5 Mbps in my experience (depends on your ISP and connectivity), now it can be lightning fast thanks to P2P, 2 Mbps+, more than adequate to download a 100 MiB+ file. You don't even have to install IPFS (but you probably should, to maintain decentral…

In my experience IPFS is only fast when you're using it through something like Cloudflare's IPFS proxy (which is basically a caching proxy). I haven't found IPFS to be actually usable any time I've tried running a node myself. Is that unusual?

Interesting, didn't notice that before. I believed my downloads are too obscure to be cached by Cloudflare, but I may be wrong. I'll do a comparison once I fixed my server at home...

Re: Libgen Storage Decentralization on IPFS

#18

Post by dooglius »

segfaultbuserr wrote: post #13
dooglius wrote: post #12

A strange choice given that distributing copyrighted materials violates the IPFS Code of Conduct [0]. Why not use BitTorrent whose attitude toward copyright is much more in sync with libgen's? [0] https://github.com/ipfs/community/blob/master/code-of-conduc...

LibGen already has "torrent per 1000 files" for a long time, some books also have their individual torrents but it's not common. I'm not sure why, but I think the problem is that files on BitTorrent cannot be directly addressed by hashes, it had to go through an intermediate step called a "torrent", which does not uniquely identify individual files directly. You can always select the one file you need from a huge tor…

That's fair, BitTorrent v2 does solve this problem, but I can't say I've seen any v2 torrents around (not that I'm specifically looking)

Re: Libgen Storage Decentralization on IPFS

#19

Post by segfaultbuserr »

bboygravity wrote: post #16
fsflover wrote: post #3

I think such materials should better be placed to darknets protecting the privacy of users and seeders, e.g., to I2P.

I agree. Actually in this day and age I think all information sharing should better be placed on darknets. I don't see the net advantage to anybody of being profiled constantly on clearnet. Naive question: is there such a thing as Bittorrent over I2P or otherwise anonymized decentralized file sharing?

> Bittorrent over I2P

Yes. I2P is designed with P2P in mind. The default Java I2P client already comes with a bundled BitTorrent client, there is also a BitTorrent client called "Vuze" and is used by some non-anonymous users to help seeding a torrent across the clearnet and I2P simultaneously. Because of the nature of I2P, don't expect great speed. No seeder is "usual", 10 KiB/s is "normal" and 50 KiB/s with only 3 or 5 seeders is "good", overall, it feels as if it's still the early 2000s, but when you get used to it and adopt a "download and forget, come back in 3 days" mentality, it's not actually that bad. The fastest download I've ever seen was the movie Snowden, 20+ seeders and over 200 KiB/s+...

Re: Libgen Storage Decentralization on IPFS

#20

Post by [deleted] »

bboygravity wrote: post #16
fsflover wrote: post #3

I think such materials should better be placed to darknets protecting the privacy of users and seeders, e.g., to I2P.

I agree. Actually in this day and age I think all information sharing should better be placed on darknets. I don't see the net advantage to anybody of being profiled constantly on clearnet. Naive question: is there such a thing as Bittorrent over I2P or otherwise anonymized decentralized file sharing?

[deleted]
Post reply on HN