Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Archivists Are Trying to Make Sure LibGen Never Goes Down

vice.com

Post reply on HN
Print view View original

41–50 of 270 posts

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#41

Post by bumbledraven »

toomuchtodo wrote: post #27
canuckintime wrote: post #24

Earlier quoted context omitted.

> Lastly, you can always contribute books. If you buy a textbook or book, consider uploading it (and scanning it, should it be a physical book) in case it isn't already present in the database. There's no easy solution for scanning physical books, is there?

There are providers [1] that will destructively scan the book for you and return a PDF. If you want to preserve the book, you're stuck using a scanning rig [2]. The Internet Archive will also non-destructively scan as part of Open Library [3], but they only permit one checkout at a time of scanned works, and the latency can be high between sending them a book and it becoming available. FYI, 600 DPI is preferred for a…

A big +1 for 1dollarscan.com. They've scanned many hundreds of books for me. The quality of the resulting PDFs is uniformly excellent, their turnaround time is fast, and their prices are cheap ($1 per 100 pages).

I've visited their office -- located in an inexpensive industrial district of San Jose -- on multiple occasions. They have a convenient process for receiving books in person.

I believe the owners are Japanese and the operation reminds me of the businesses I visited in Tokyo: quiet, neat, and über-efficient.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#42

Post by javitury »

LameRubberDucky wrote: post #35
Tepix wrote: post #25

Related: Looking at harddisk cost per terabyte, quite often extern drives are cheaper than internal ones. For example right now in Germany I can get a WD 8TB USB 3.0 drive for 135€ but the cheapest internal 8TB drive costs 169€. Any idea why? It's puzzling.

I noticed this yesterday while shopping for cyber Monday deals. If you want to load up a server with drives, perhaps the external drives can be removed from their cases and used internally?

Check it before you buy it. Years ago I bought a 1TB WD external drive. The usb interface was connected directly to the drive.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#43

Post by miki123211 »

The new architecture of pirate sites, what I call the Hydra architecture, seems pretty interesting to me. There isn't a single site hosting the content, but a group of mirrors freely exchanging data between one another. In case some of them go down, the other ones still remain and new ones can appear, copying data from the remaining mirrors. This is like a hydra that grows two heads every time you chop one off. It's absolutely unkillable, as there's no single group or server to sue.

A more advanced version of this architecture is used by pirate addons for the Kodi media center software. Basically, you have a bunch of completely legal and above board services like Imdb that contain video metadata. They provide the search results, the artworks, the plot descriptions, episode lists for TV shows etc. Impossible to sue and shut down, as they're legal. Then, you have a large number of illegal services that, essentially, map IDs from websites like IMDB to links. Those links lead to websites like Openload, which let you host videos. They're in the gray area, if they comply with DMCA requests and are in a reasonably safe jurisdiction, they're unlikely to be shut down. On the Kodi side, you have a bunch of addons. There are the legitimate ones that access IMDB and give you the IDs, the not that legitimate ones that map IDs to URLs, and the half-legitimate ones that can actually play stuff ron those URLS (not an easy taks, as websites usually try to prevent you from playing something without seeing their ads). Those addons are distributed as libraries, and are used as dependencies by user-friendly frontends. Those frontends usually depend on several addons in each category, so, in case one goes down, all the other ones still remain. It's all so decentralized and ownerless that there's no single point of failure. The best you can do is killing the frontend addon, but it's easy to make a new one, and users are used to switching them every few months.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#44

Post by caleb-allen »

buboard wrote: post #33
Someone1234 wrote: post #22

Earlier quoted context omitted.

Storing that amount of information in a way that an unknown alien species would be able to read (even assuming technical expertise greater than our own) is a huge problem. Keep in mind that they don't know our written or computational language and there's nothing about our technology that is inherently self-explaining/obvious. Even the assumption that they'd use binary computers (rather than trinary, or other technol…

> an unknown alien species Not necessarily for aliens ... but why not keep a backup in a safe place outside the dangers of earthlings ? OTOH, I think a sufficiently advanced alien intelligence will be able to decipher the information structures we use regardless of differences in technology. It's possible though there will be missing links in that archive, which will need to be supplemented with a primary secondary a…

What about a satellite backup in orbit around earth? Maybe an elliptical orbit, coming around a few times a year or something

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#45

Post by caleb-allen »

Someone1234 wrote: post #22
buboard wrote: post #5

one of the next interplanetary or Interstellar Probe should carry a copy of the sci-hub torrent in some kind of permanent storage

Storing that amount of information in a way that an unknown alien species would be able to read (even assuming technical expertise greater than our own) is a huge problem. Keep in mind that they don't know our written or computational language and there's nothing about our technology that is inherently self-explaining/obvious. Even the assumption that they'd use binary computers (rather than trinary, or other technol…

/Godel, Escher, Bach/ has an interesting thought experiment about exactly this.

If one were to receive an object, how would it be indicated that there is a message embedded in there? Given that a intelligence could recognize that there was a message embedded, could it eventually be deciphered?

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#46

Post by roland00 »

driverdan wrote: post #38
Invictus0 wrote: post #14

Earlier quoted context omitted.

I think the duplication issue is probably overstated. I doubt tackling that would shave off more than 20% of the total backup size.

20% is not insignificant.

Forking the LibGen to save 20% of file sizes will be counterproductive. Yes you save some storage but the network effects is more important, for people willing to contribute to "the one true thing" actually provides more seeders than the 20%.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#47

Post by MiroF »

dooglius wrote: post #20
Invictus0 wrote: post #14

Earlier quoted context omitted.

I think the duplication issue is probably overstated. I doubt tackling that would shave off more than 20% of the total backup size.

Speaking from personal experience, I usually see several results for any search. Granted, there's a big selection bias there, but 20% seems way too small.

Usually the different results have meaningful differences - often times different edition or translator etc

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#48

Post by guidoism »

canuckintime wrote: post #24
legatus wrote: post #2

This is an extremely important effort. The LibGen archive contains around 32 TBs of books (by far the most common being scientific books and textbooks, with a healthy dose of non-STEM). The SciMag archive, backing up Sci-Hub, clocks in at around 67 TBs [0]. This is invaluable data that should not be lost. If you want to contribute, here's a few ways to do so. If you wish to donate bandwidth or storage, I personally k…

> Lastly, you can always contribute books. If you buy a textbook or book, consider uploading it (and scanning it, should it be a physical book) in case it isn't already present in the database. There's no easy solution for scanning physical books, is there?

Scanning with your phone is getting easier. At a minimum you can take a pic of each of the pages. Software can clean up the images, sorta. It's not ideal but it's better than nothing.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#49

Post by xur17 »

LameRubberDucky wrote: post #35
Tepix wrote: post #25

Related: Looking at harddisk cost per terabyte, quite often extern drives are cheaper than internal ones. For example right now in Germany I can get a WD 8TB USB 3.0 drive for 135€ but the cheapest internal 8TB drive costs 169€. Any idea why? It's puzzling.

I noticed this yesterday while shopping for cyber Monday deals. If you want to load up a server with drives, perhaps the external drives can be removed from their cases and used internally?

I did exactly this when setting up a NAS. Saved $50 / drive, and it took me a few minutes to remove the drive from each.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#50

Post by [deleted] »

miki123211 wrote: post #43

The new architecture of pirate sites, what I call the Hydra architecture, seems pretty interesting to me. There isn't a single site hosting the content, but a group of mirrors freely exchanging data between one another. In case some of them go down, the other ones still remain and new ones can appear, copying data from the remaining mirrors. This is like a hydra that grows two heads every time you chop one off. It's…

[deleted]
Post reply on HN