Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Archivists Are Trying to Make Sure LibGen Never Goes Down

vice.com

Post reply on HN
Print view View original

51–60 of 270 posts

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#51

Post by JustSomeNobody »

LameRubberDucky wrote: post #35
Tepix wrote: post #25

Related: Looking at harddisk cost per terabyte, quite often extern drives are cheaper than internal ones. For example right now in Germany I can get a WD 8TB USB 3.0 drive for 135€ but the cheapest internal 8TB drive costs 169€. Any idea why? It's puzzling.

I noticed this yesterday while shopping for cyber Monday deals. If you want to load up a server with drives, perhaps the external drives can be removed from their cases and used internally?

/r/DataHoarder might be a good place to ask (or search). Also, this is known as "shucking".

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#52

Post by 0xdeadbeefbabe »

legatus wrote: post #2

This is an extremely important effort. The LibGen archive contains around 32 TBs of books (by far the most common being scientific books and textbooks, with a healthy dose of non-STEM). The SciMag archive, backing up Sci-Hub, clocks in at around 67 TBs [0]. This is invaluable data that should not be lost. If you want to contribute, here's a few ways to do so. If you wish to donate bandwidth or storage, I personally k…

I guess it's stunningly obvious to everyone else, but how are you certain the replacement isn't worse than the original system. I already see comments about the curation problem, for example. What's the point in making bad information (duplicate information etc.) highly available? Why put so much faith in this donation strategy i.e. donating bandwidth or donating money?

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#53

Post by guidoism »

lovecg wrote: post #40
woofcat wrote: post #31

Earlier quoted context omitted.

Look at the google books project. That got shutdown real hard due to copyright issues and litigation after they invested a ton of money in digitizing some of the most valuable library collections in the world.

It’s incredibly sad: https://www.theatlantic.com/technology/archive/2017/04/the-t...

There are also multiple petabytes of microfiche scans of old newspapers. And of course nobody cares about it. The problem was shut down 2011ish and the data became "owned" by a team that didn't care for it. There was talk of just deleting the data because the team didn't want to pay for it. Ugh.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#54

Post by traverseda »

Avamander wrote: post #16

Why not publish the site over IPFS, that would make P2P hosting much simpler?

In my experience ipfs doesn't actually work. I'd love to be proven wrong, but the reason why nobody uses ipfs even when it seems like a great fit is bect it's not really usable.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#55

Post by [deleted] »

legatus wrote: post #11
voldacar wrote: post #7

Is there a way to just download the whole 32TB to your own machine? I see a ton of mirrors but the content seems to be highly fragmented between them

There are ways to do so. The archive is made up of many, many torrents (I believe it's a monthly if not biweekly update of the database). If you have the storage/bandwidth availability for the whole 32TBs, please get in touch and I may be able to help you get the whole deal without too much hassle. Otherwise, just pick some torrents (it would be best to pick them based on torrent health, but they are so many to check…

[deleted]

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#56

Post by dewey »

legatus wrote: post #2

This is an extremely important effort. The LibGen archive contains around 32 TBs of books (by far the most common being scientific books and textbooks, with a healthy dose of non-STEM). The SciMag archive, backing up Sci-Hub, clocks in at around 67 TBs [0]. This is invaluable data that should not be lost. If you want to contribute, here's a few ways to do so. If you wish to donate bandwidth or storage, I personally k…

I just read the article and your comments here and I'm a bit unsure what's the difference to the Internet Archive. Is it that the IA can archive them but not make them public for legal reasons and The-Eye is more focused on keeping them online and accessible no matter what?

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#57

Post by sanxiyn »

Yongle Encyclopedia was a similar project of the 15th century China. It was the largest encyclopedia in the world for 600 years until surpassed by Wikipedia.

Alas, Yongle Encyclopedia is almost completely lost now. Archiving is harder than you think.

https://en.wikipedia.org/wiki/Yongle_Encyclopedia

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#58

Post by js8 »

penagwin wrote: post #39
ihuman wrote: post #23

Earlier quoted context omitted.

I'm pretty surprised by the lack of seeders. Out of the 2438 torrents listed, a third have 0 seeders, another third have 1 seeder, and all but 5 have less than 10. Hopefully the publicity boosts those numbers.

From what I've heard a good chunk of people rotate their seeds for LibGen because their seedboxes can't handle all the connections for every torrent at once.

Is there some tool or documentation describing this practice?

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#59

Post by toomuchtodo »

dewey wrote: post #56
legatus wrote: post #2

This is an extremely important effort. The LibGen archive contains around 32 TBs of books (by far the most common being scientific books and textbooks, with a healthy dose of non-STEM). The SciMag archive, backing up Sci-Hub, clocks in at around 67 TBs [0]. This is invaluable data that should not be lost. If you want to contribute, here's a few ways to do so. If you wish to donate bandwidth or storage, I personally k…

I just read the article and your comments here and I'm a bit unsure what's the difference to the Internet Archive. Is it that the IA can archive them but not make them public for legal reasons and The-Eye is more focused on keeping them online and accessible no matter what?

Yes. It is extremely likely IA has the LibGen corpus archived, but darked (inaccessible), to prevent litigation.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#60

Post by buboard »

saalweachter wrote: post #9
buboard wrote: post #5

one of the next interplanetary or Interstellar Probe should carry a copy of the sci-hub torrent in some kind of permanent storage

Do we have anything rated for a few millennia of interstellar radiation besides etched gold plates?

whats wrong with etched gold disks?
Post reply on HN