Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Archivists Are Trying to Make Sure LibGen Never Goes Down

vice.com

Post reply on HN
Print view View original

151–160 of 270 posts

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#151

Post by unicornporn »

EthanHeilman wrote: post #107

Maybe we should print this out on acid-free paper-thin flexible wood-pulp sheets stitched to together to form linear organized aggregations. Each aggregation would contain one or more works and be searchable using a SQL-like database. To make this plan really work there would need to be a collection of geographically distributed long term physical repositories that would receive periodic updates as new material becam…

In the GLAM sector the LOCKSS[1] is project is quite well-known. It tries to deal with some of the resiliency problems that is inherent in digital preservation. However, I'd guess this system does not offer the needed anonymity.

[1] https://www.lockss.org/ ; https://en.wikipedia.org/wiki/LOCKSS

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#152

Post by ancientworldnow »

washadjeffmad wrote: post #129
jbverschoor wrote: post #105

Earlier quoted context omitted.

You can buy 48TB (4x12TB) for €1000. Store some index on an SSD, and you have another full node.

If you don't care about warranty, 8 and 12TB drives routinely go for $15/TB on sale inside WD Elements. I picked up 32TB for just under $500 with discount over the holiday that way.

Even if you shuck the drive, as long as you keep the enclosure Western Digital will still honor the warranty.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#154

Post by guidoism »

whydoyoucare wrote: post #149

Isn't scanning a physical book and uploading a soft-copy, a landmine of hazards (both legal and moral)? Essentially you are encouraging (some) unlawful activity... I am not so sure I am onboard with this idea!

It’s easy to take this stance in a rich country. But what about the people in countries where one of these books cost the equivalent of a year’s wages. Not so black and white eh?

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#155

Post by morsch »

jonah-archive wrote: post #117
walrus01 wrote: post #83

Earlier quoted context omitted.

It is very common these days to buy the WD 8TB, 10TB and 12TB external USB3 hard drives and remove their cases, and put them in some sort of home built file server or NAS. There's a technique to put a thin section of kapton tape on one of the SATA pins so that they will power up from ordinary PC/ATX type power supplies with regular SATA power connectors. https://www.instructables.com/id/How-to-Fix-the-33V-Pin-Issu...…

> at no greater or lesser annual failure rate than the expensive enterprise hard drives. I've read these reports as well, but I can say that it's not my experience (we've gone through a few rounds of shucking at the Internet Archive, for economy and in one case necessity after the 2011 Thailand floods pinched the supply chain). Our raw failure rates on shucked drives are significantly higher, and the drives themselve…

I'm apparently out of the loop w.r.t. non-solid state storage. For people in the same boat:

Shingled magnetic recording (SMR) is a magnetic storage data recording technology used in hard disk drives (HDDs) to increase storage density and overall per-drive storage capacity ... The overlapping-tracks architecture may slow down the writing process since writing to one track overwrites adjacent tracks, and requires them to be rewritten as well.

https://en.wikipedia.org/wiki/Shingled_magnetic_recording

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#156

Post by guidoism »

agumonkey wrote: post #119
legatus wrote: post #6

Earlier quoted context omitted.

There are groups behind data curation as well, though it is much harder. LibGen sees an addition rate of about 230 GBs per month, while SciMag's is around 1.10 TBs per month. We should expect those numbers to increase in the future. The man-hours required to curate those database may very well cost much more than the storage and bandwidth required to store duplicates and incorrectly tagged files. In any case, as I sa…

Do you know if they process PDF to reduce file size ?

A lot of the data is in the djvu format which is very efficient for scanned books.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#157

Post by rollinDyno »

miki123211 wrote: post #43

The new architecture of pirate sites, what I call the Hydra architecture, seems pretty interesting to me. There isn't a single site hosting the content, but a group of mirrors freely exchanging data between one another. In case some of them go down, the other ones still remain and new ones can appear, copying data from the remaining mirrors. This is like a hydra that grows two heads every time you chop one off. It's…

I worry that if this system becames permanent, one in which it is practically impossible to stop piracy, followed by the loss of traditional incentives we might find ourselves in a place where no motivated investor will break even when producing quality and innocuous content.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#158

Post by guidoism »

BlueTemplar wrote: post #127
jandrese wrote: post #81

Earlier quoted context omitted.

I'm incredibly thankful that the public library system was invented before copyright maximalists got control of Congress.

In this case the issue seems to have come from "copyright minimalists" instead : wanting the books to be freely available, rather than making money for Google... I wonder why the Copyright Office didn't just buy Google Books, would only have cost a few hundred million $ ?

I’m fairly confident that google books is a huge money loser for google. The only reason it’s still online is because there are people within google willing to stick their necks out to spend the money on it.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#159

Post by ColanR »

rollinDyno wrote: post #157
miki123211 wrote: post #43

The new architecture of pirate sites, what I call the Hydra architecture, seems pretty interesting to me. There isn't a single site hosting the content, but a group of mirrors freely exchanging data between one another. In case some of them go down, the other ones still remain and new ones can appear, copying data from the remaining mirrors. This is like a hydra that grows two heads every time you chop one off. It's…

I worry that if this system becames permanent, one in which it is practically impossible to stop piracy, followed by the loss of traditional incentives we might find ourselves in a place where no motivated investor will break even when producing quality and innocuous content.

In that case, we will just pass around the same old stuff until we get bored enough that we'll actually pay for new stuff.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#160

Post by whydoyoucare »

guidoism wrote: post #154
whydoyoucare wrote: post #149

Isn't scanning a physical book and uploading a soft-copy, a landmine of hazards (both legal and moral)? Essentially you are encouraging (some) unlawful activity... I am not so sure I am onboard with this idea!

It’s easy to take this stance in a rich country. But what about the people in countries where one of these books cost the equivalent of a year’s wages. Not so black and white eh?

As far as I know, prices of books differ between rich and developing nations. For e.g., The C Programming Language that costs $50 in the US [1], is sold for Rs. 259 (~$4 US) in India. I believe that is the case with most "economy editions" specifically targeted at developing nations. It certainly isn't an "year's wages".

While I do understand your point, it still does not justify encouraging modern-day Robinhoods' and breaking the law.

[1] https://www.amazon.com/Programming-Language-2nd-Brian-Kernig... [2] https://www.amazon.in/Programming-Language-Kernighan-Dennis-...

Post reply on HN