Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Archivists Are Trying to Make Sure LibGen Never Goes Down

vice.com

Post reply on HN
Print view View original

201–210 of 270 posts

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#201

Post by steve19 »

EthanHeilman wrote: post #197
TheOtherHobbes wrote: post #179

Earlier quoted context omitted.

This is not a solvable problem without technological continuity, or some unimaginably smart technology we can't imagine today. If you found a mysterious archive object and had no idea what it was - CD-R, hard drive, SSD, whatever - not only would you have to reinvent an entire hardware reader around it, you would also have to work out the file structure, extract the data (some of which could be damaged), and reverse…

Take a CD-R of some MP3 with English language file names stored on a FAT32 filesystem for example. Assume the reflective layer didn't rust since it was abandoned in a dry climate and our future archaeologist has access to roughly modern levels of technology. 1. Even if the CD-R has been crushed and shattered you could use a modern and cheap microscope to read continuous pits and lands off the disk [0,1]. It would be…

An engraved metal or stone tablet could be left along with the CDs to bootstrap the process. It could range from explaining the MP3 spec, to as simple as pictograms showing human speech being converted to microscopic pits. Explaining ASCII would be even easier.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#202

Post by kortex »

izzydata wrote: post #161
burtonator wrote: post #71

What's interesting is that 32TB is becoming more and more affordable and the research material is roughly staying about the same size. That might change though as people start including video + data within papers and have new notebook formats that are live and contain docker containers/ipython, etc. It's a shame we can't just mail these around.

The bandwidth to upload to people can get expensive depending on where you live. Most home connections don't have bi-directional fiber so you are stuck with crippling amounts of upload bandwidth.

I feel like this is the crux of the matter. You could easily get 32 people on this site to volunteer 1 TB each, if it were just cold storage. However, making those resources accessible and searchable (with all the pitfalls of compliance, uptime, legality, etc) is a totally different ballgame.

Encrypted shards partially solves this, but then you hit the quandry of "But what if I have a shard of something illegal or undesired enough to upset the wrong people?" which has not been thoroughly tested in our legal system.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#203

Post by kortex »

legatus wrote: post #17
saalweachter wrote: post #9

Earlier quoted context omitted.

Do we have anything rated for a few millennia of interstellar radiation besides etched gold plates?

Microsoft's project Silica [0] may hopefully provide really long term, large capacity archive grade storage on earth. I wonder what effects interstellar radiation has on them. [0] https://www.theverge.com/2019/11/4/20942040/microsoft-projec...

Glass is pretty inert, full stop. It would depend on the voxel size but I imagine as long as you have more than a few hundered atoms per voxel/bit you will have survivability on the order of millenia, even in high radiation environments. Someone would have to do the nuclear cross section calculations to get a real bit error rate but glass is very tough stuff.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#204

Post by zAy0LfpBZLC8mAC »

weinzierl wrote: post #110
Tepix wrote: post #25

Related: Looking at harddisk cost per terabyte, quite often extern drives are cheaper than internal ones. For example right now in Germany I can get a WD 8TB USB 3.0 drive for 135€ but the cheapest internal 8TB drive costs 169€. Any idea why? It's puzzling.

For me on amazon.de the WD 8TB USB 3.0 drive is currently at EUR 159.99. Where do you get it for EUR 135?

https://gh.de/?cat=hdx&xf=6_8000%7E840_USB+3.0&sort=p

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#205

Post by washadjeffmad »

StavrosK wrote: post #182
washadjeffmad wrote: post #129

Earlier quoted context omitted.

If you don't care about warranty, 8 and 12TB drives routinely go for $15/TB on sale inside WD Elements. I picked up 32TB for just under $500 with discount over the holiday that way.

Can you elaborate? What's the catch?

The only catch is that it's a minor lottery which model drive you're getting.

For instance, I got all white label WD80EMAZs (256MB cache, non-SMR, same firmware as the Reds) in this batch, so I had to insulate the 3.3V pins.

There are also true Reds, 128MB and 512MB cache drives, helium filleds, 7.2K HGSTs slowed to 5.4K, and other variants.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#206

Post by repiret »

abawany wrote: post #190
canuckintime wrote: post #24

Earlier quoted context omitted.

> Lastly, you can always contribute books. If you buy a textbook or book, consider uploading it (and scanning it, should it be a physical book) in case it isn't already present in the database. There's no easy solution for scanning physical books, is there?

I use bookscan.us for this purpose: I mail the physical book to them and they send me a file a few days later for a very reasonable price.

Unfortunately it’s a destructive process.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#207

Post by sillysaurusx »

MadWombat wrote: post #162
miki123211 wrote: post #43

The new architecture of pirate sites, what I call the Hydra architecture, seems pretty interesting to me. There isn't a single site hosting the content, but a group of mirrors freely exchanging data between one another. In case some of them go down, the other ones still remain and new ones can appear, copying data from the remaining mirrors. This is like a hydra that grows two heads every time you chop one off. It's…

> It's absolutely unkillable Just like any other distributed system, this is vulnerable to organized take downs and scare tactics. There was a whole bunch of mirrors of Pirate Bay, yet once most of Europe's legal systems adopted the "sharing is theft" mindset, it became pretty much impossible to find one.

I was just in Europe and used piratebay while I was there. The main site didn’t work, but searching “piratebay mirror” found one that did right away.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#208

Post by userbinator »

guidoism wrote: post #48
canuckintime wrote: post #24

Earlier quoted context omitted.

> Lastly, you can always contribute books. If you buy a textbook or book, consider uploading it (and scanning it, should it be a physical book) in case it isn't already present in the database. There's no easy solution for scanning physical books, is there?

Scanning with your phone is getting easier. At a minimum you can take a pic of each of the pages. Software can clean up the images, sorta. It's not ideal but it's better than nothing.

I remember when "cammed" books were bottom-tier and basically limited to things like 0day releases, even when done with an expensive DSLR. It's amazing how much camera technology has progressed since then; in less than a second you can get a high-resolution, extremely readable image of each page.

I used to participate in the "bookz scene", well over a decade ago. Raiding the local public libraries --- borrowing as many books as we could --- and having "scanparties" to digitise and upload them was incredibly fun, and we did it for the thrill, never thinking that one day almost all of our releases would end up in LibGen.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#209

Post by userbinator »

lioeters wrote: post #140
knzhou wrote: post #130

Libgen is one of the greatest contributors to scientific productivity worldwide, possibly beaten only by Sci-Hub. Just about everybody in academia knows about it. If it ever vanished, some of us could probably still get by trading files from person to person, but nothing could be as perfect as what we got now.

> possibly beaten only by Sci-Hub Today I learned that Library Genesis is actually "powered by Sci-Hub" as its primary source. So I guess they're sister projects by similarly minded people (who seem to be mostly/originally based in Slavic countries, which I find interesting culturally - perhaps it's due to a looser legal environment + activist academics?). > Just about everybody in academia knows about it. That reall…

who seem to be mostly/originally based in Slavic countries, which I find interesting culturally - perhaps it's due to a looser legal environment + activist academics?

You see the same situation with Asia --- it's a collectivist culture, they have a very different perspective on IP in general.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#210

Post by mikorym »

alteria wrote: post #193
gdy wrote: post #163

Earlier quoted context omitted.

"the USSR and their restrictive (if we are being euphemistic) policies towards academics." What do you mean?

Their policies were for more than "restrictive" is how I'm reading it See [1] [1] https://en.wikipedia.org/wiki/Suppressed_research_in_the_Sov...

Yes. I'm saying restrictive to describe the effect on academic papers. The effect on (oppression of) the academics themselves was much worse.
Post reply on HN