There is a huge amount of duplication there (i.e. books that have many scans), I wonder if it would be better to tackle that versus doing a straight backup.
I think the duplication issue is probably overstated. I doubt tackling that would shave off more than 20% of the total backup size.
Archivists Are Trying to Make Sure LibGen Never Goes Down
21–30 of 270 posts
Re: Archivists Are Trying to Make Sure LibGen Never Goes Down
#22one of the next interplanetary or Interstellar Probe should carry a copy of the sci-hub torrent in some kind of permanent storage
Keep in mind that they don't know our written or computational language and there's nothing about our technology that is inherently self-explaining/obvious.
Even the assumption that they'd use binary computers (rather than trinary, or other technology not based around electrical voltages) is open to debate.
Re: Archivists Are Trying to Make Sure LibGen Never Goes Down
#23Is there a way to just download the whole 32TB to your own machine? I see a ton of mirrors but the content seems to be highly fragmented between them
There are ways to do so. The archive is made up of many, many torrents (I believe it's a monthly if not biweekly update of the database). If you have the storage/bandwidth availability for the whole 32TBs, please get in touch and I may be able to help you get the whole deal without too much hassle. Otherwise, just pick some torrents (it would be best to pick them based on torrent health, but they are so many to check…
Re: Archivists Are Trying to Make Sure LibGen Never Goes Down
#24This is an extremely important effort. The LibGen archive contains around 32 TBs of books (by far the most common being scientific books and textbooks, with a healthy dose of non-STEM). The SciMag archive, backing up Sci-Hub, clocks in at around 67 TBs [0]. This is invaluable data that should not be lost. If you want to contribute, here's a few ways to do so. If you wish to donate bandwidth or storage, I personally k…
There's no easy solution for scanning physical books, is there?
Re: Archivists Are Trying to Make Sure LibGen Never Goes Down
#25For example right now in Germany I can get a WD 8TB USB 3.0 drive for 135€ but the cheapest internal 8TB drive costs 169€.
Any idea why? It's puzzling.
Re: Archivists Are Trying to Make Sure LibGen Never Goes Down
#26Why not publish the site over IPFS, that would make P2P hosting much simpler?
Currently (at least for the-eye) it's about IPFS's barrier of entry. I expect LibGen's case to be similar. Most people don't know about it, and if even those that knew about it had to learn how IPFS works etc, they would probably just try to find the book they're looking for elsewhere.
Re: Archivists Are Trying to Make Sure LibGen Never Goes Down
#27This is an extremely important effort. The LibGen archive contains around 32 TBs of books (by far the most common being scientific books and textbooks, with a healthy dose of non-STEM). The SciMag archive, backing up Sci-Hub, clocks in at around 67 TBs [0]. This is invaluable data that should not be lost. If you want to contribute, here's a few ways to do so. If you wish to donate bandwidth or storage, I personally k…
> Lastly, you can always contribute books. If you buy a textbook or book, consider uploading it (and scanning it, should it be a physical book) in case it isn't already present in the database. There's no easy solution for scanning physical books, is there?
[1] http://1dollarscan.com/ (no affiliation, just a satisfied customer, can't scan certain textbooks due to publisher threats of litigation)
Re: Archivists Are Trying to Make Sure LibGen Never Goes Down
#28This is an extremely important effort. The LibGen archive contains around 32 TBs of books (by far the most common being scientific books and textbooks, with a healthy dose of non-STEM). The SciMag archive, backing up Sci-Hub, clocks in at around 67 TBs [0]. This is invaluable data that should not be lost. If you want to contribute, here's a few ways to do so. If you wish to donate bandwidth or storage, I personally k…
I'd love to be proven wrong.
Re: Archivists Are Trying to Make Sure LibGen Never Goes Down
#29This is an extremely important effort. The LibGen archive contains around 32 TBs of books (by far the most common being scientific books and textbooks, with a healthy dose of non-STEM). The SciMag archive, backing up Sci-Hub, clocks in at around 67 TBs [0]. This is invaluable data that should not be lost. If you want to contribute, here's a few ways to do so. If you wish to donate bandwidth or storage, I personally k…
> Lastly, you can always contribute books. If you buy a textbook or book, consider uploading it (and scanning it, should it be a physical book) in case it isn't already present in the database. There's no easy solution for scanning physical books, is there?
Re: Archivists Are Trying to Make Sure LibGen Never Goes Down
#30Why not publish the site over IPFS, that would make P2P hosting much simpler?
Currently (at least for the-eye) it's about IPFS's barrier of entry. I expect LibGen's case to be similar. Most people don't know about it, and if even those that knew about it had to learn how IPFS works etc, they would probably just try to find the book they're looking for elsewhere.