Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Archivists Are Trying to Make Sure LibGen Never Goes Down

vice.com

Post reply on HN
Print view View original

221–230 of 270 posts

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#221

Post by nyolfen »

TheOtherHobbes wrote: post #179
EthanHeilman wrote: post #107

Maybe we should print this out on acid-free paper-thin flexible wood-pulp sheets stitched to together to form linear organized aggregations. Each aggregation would contain one or more works and be searchable using a SQL-like database. To make this plan really work there would need to be a collection of geographically distributed long term physical repositories that would receive periodic updates as new material becam…

This is not a solvable problem without technological continuity, or some unimaginably smart technology we can't imagine today. If you found a mysterious archive object and had no idea what it was - CD-R, hard drive, SSD, whatever - not only would you have to reinvent an entire hardware reader around it, you would also have to work out the file structure, extract the data (some of which could be damaged), and reverse…

the storage part at least could be a solved problem: https://en.wikipedia.org/wiki/5D_optical_data_storage

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#222

Post by zaarn »

walrus01 wrote: post #188
trevyn wrote: post #183

Earlier quoted context omitted.

yEnc overhead is about 2% and there are plenty of providers with ~10 year retention.

Wow. I can't even imagine how much disk space ten years of retention of alt.binaries.* takes up. It's been literally ten years since I last did anything serious Usenet related.

Atleast in my experience, 10Y providers ask for more money and provide less high speed bandwidth (after which your up/down is usually limited to around 10 or 1mbps)

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#223

Post by zaarn »

saalweachter wrote: post #148
jandrese wrote: post #98

Earlier quoted context omitted.

Thank SpaceX for slashing launch costs and me for not factoring in much of a second stage to boost that mass out of orbit. But I was also extremely generous with the thickness in the first post. In real life they would almost certainly be closer to .05mm than 1.2mm, and probably not made out of solid gold. The point was to show that even with some rather pessimistic assumptions the project was within human scale and…

Yeah, it's all within the realm of possibility, with too many unknowns (eg, what is a realistic information density) to really easily say how easy or hard it is. Re: Falcon 9's, Voyager was lifted on a Titan IIIE, which had a LEO payload of 15,300 kg, compared to 22,800 kg for the Falcon 9 to LEO. Assuming Voyager was near the capacity of what could be ejected from the Solar System by a Titan IIIE, you'd need to send…

I would argue that at this point, actually building Voltron might be more wortwhile.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#224

Post by zaarn »

walrus01 wrote: post #126
buboard wrote: post #123

Earlier quoted context omitted.

too close to the warzone to survive

okay, high Mars orbit. Quite a lot less delta-v requirement for some theoretical several thousand kilogram chunk of long-lifespan data storage, and easier to discover and retrieve, than achieving solar escape velocity (look at the delta-v budget for the new horizons probe vs. its total size and weight, for instance).

I'd probably put a signaling system on it. Some large RTG using elements with a long half life and extremely redundant systems that send a weak signal towards earth, just over the background noise. Some regular signal ("DATA" as a morse code) that repeats from an unknown object orbiting mars?

I'd bet that any civilization would be flying there ASAP to see what it is about.

The crux is really making a simple radio transmitter that can survive that long but the parameters are very weak (ie, the frequency is allowed to shift so long as it remains somewhat in a region that can be received on earth's surface, it needs a very long last power source but doesn't need a lot of power, unidirectional transmission is fine as long as it arrives).

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#225

Post by zaarn »

jerf wrote: post #100
Someone1234 wrote: post #22

Earlier quoted context omitted.

Storing that amount of information in a way that an unknown alien species would be able to read (even assuming technical expertise greater than our own) is a huge problem. Keep in mind that they don't know our written or computational language and there's nothing about our technology that is inherently self-explaining/obvious. Even the assumption that they'd use binary computers (rather than trinary, or other technol…

An idea I've seen is including messages at several levels. At the outermost level you describe in very basic format how to build a magnifying glass. From there you have diagrams that are legible that describes how to build a microscope. From there you have more than enough space to describe the basics of what else is in there and to start describing your language. I'm thinking optical storage in a clear rock of some…

>I'm not trying to spec "Ugh wanders out of the jungle, sees our pretty rock, and personally has a 20th century civilization up and running in 10 years" or anything crazy.

Quite coincidental, there is currently an Anime running named "Dr. Stone" which is quite exactly about that; jump starting human civilizatio from the stone age to modern day as fast as possible. Atleast in-story it's been a few months and they're currently building radios and have a waterwheel generator.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#226

Post by BlueTemplar »

guidoism wrote: post #158
BlueTemplar wrote: post #127

Earlier quoted context omitted.

In this case the issue seems to have come from "copyright minimalists" instead : wanting the books to be freely available, rather than making money for Google... I wonder why the Copyright Office didn't just buy Google Books, would only have cost a few hundred million $ ?

I’m fairly confident that google books is a huge money loser for google. The only reason it’s still online is because there are people within google willing to stick their necks out to spend the money on it.

Indeed, but for how long, considering the Google Books team itself seemingly wants to delete the ~100M books database ?

P.S.: Might be soon, considering that it was basically what Google was initially about, and one of the founders just resigned from Alphabet...

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#227

Post by esotericn »

whydoyoucare wrote: post #160
guidoism wrote: post #154

Earlier quoted context omitted.

It’s easy to take this stance in a rich country. But what about the people in countries where one of these books cost the equivalent of a year’s wages. Not so black and white eh?

As far as I know, prices of books differ between rich and developing nations. For e.g., The C Programming Language that costs $50 in the US [1], is sold for Rs. 259 (~$4 US) in India. I believe that is the case with most "economy editions" specifically targeted at developing nations. It certainly isn't an "year's wages". While I do understand your point, it still does not justify encouraging modern-day Robinhoods' an…

Hi.

You're speaking to someone from the UK. When I grew up we had to walk uphill both ways, to quote the Four Yorkshiremen sketch.

I couldn't afford to buy things like books, operating systems, games, etc. We spent the money on food, rent, and on the Internet.

People like that past version of me are going to pirate regardless of what you say. Law breaking? You can't see me grinning ear to ear.

I like to think my tax bill makes up for that.

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#228

Post by dredmorbius »

BlueTemplar wrote: post #226
guidoism wrote: post #158

Earlier quoted context omitted.

I’m fairly confident that google books is a huge money loser for google. The only reason it’s still online is because there are people within google willing to stick their necks out to spend the money on it.

Indeed, but for how long, considering the Google Books team itself seemingly wants to delete the ~100M books database ? P.S.: Might be soon, considering that it was basically what Google was initially about, and one of the founders just resigned from Alphabet...

Source on the deletion initiative?

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#229

Post by mikorym »

mikorym wrote: post #218
gdy wrote: post #215

Earlier quoted context omitted.

No, that's BS. There are well known cases of genetics and cybernetics being banned for ideological reasons during Stalin's time. Scientific books and articles of convicted 'enemies of the state' were dangerous to possess in that time too. Some scientists used ideological 'arguments' in scientific debates which were dangerous to argue against. But all that, AFAIK, ended after Stalin's death in 1953. Moreover, I've nev…

Not sure what you are saying. Mathematicians were not even allowed to travel abroad [1] and any "concessions" were essentially as it pleased the USSR state. Only from 1990 was movement free in the true sense of the word. [1] An example was when Margulis won the Fields medal: https://en.wikipedia.org/wiki/Grigory_Margulis . There are many other examples too.

[2] Also the 1970 Fields medal: https://en.wikipedia.org/wiki/Sergei_Novikov_(mathematician)

Re: Archivists Are Trying to Make Sure LibGen Never Goes Down

#230

Post by dredmorbius »

lioeters wrote: post #133
lovecg wrote: post #40

Earlier quoted context omitted.

It’s incredibly sad: https://www.theatlantic.com/technology/archive/2017/04/the-t...

> Somewhere at Google there is a database containing 25 million books and nobody is allowed to read them. Indeed, what an intellectual tragedy.. > In August 2010, Google put out a blog post announcing that there were 129,864,880 books in the world. The company said they were going to scan them all. That seems like a surprisingly "small" number. Well, in trying to picture a physical library with 130 million books, may…

Until fairly recently (historically), books were overwhelmingly scarce. A few datapoints:

- The total number of books -- not titles, but actual bound volumes -- in Europe as of 1500 CE, was about 50,000. By 1800, the total was just under one billion.

- The library of the University of Paris circa 1000 CE comprised about 2,000 volumes. It was among the largest in Europe.

- The Library of Constantinople in the 5th century had 120,000 volumes, the largest in Europe at the time.

- A fair-sized city public library today has on the order of 300,000 volumes. A large university library generally a millon or so. The Harvard Library contains 20 million volumes. The University of California collection, across all ten campuses, totals more than 34 million volumes.

- The total surviving corpus of Greek literature is a few hundred titles. I believe many of those were only preserved through Arabic scholars, some possibly in Arabic translation, not the original greek.

- There's an online collection of cuneiform tablets. These generally correspond to a written page (or less) of text, with the largest collections numbering in the tens of thousands of items.

- As of about 1800, the library of the British Museum (now the British Library) had 50,000 volumes. Again, among the largest of its time.

- From roughly 1950 - 2000, roughly 300,000 titles were published annually in the United States and/or English-language editions. R.R. Bowker issues ISBNs and tracks this. From ~2005 onward, "nontraditional" books (self- / vanity-published) have been about or above 1 million annually.

- The US Library of Congress, the largest contemporary library in the world, holds 24 million books in its main collection (another 16 million in large type), and has 126 million catalogued items in total (2015).

- At about 5 MB per book, in PDF form, total storage for the 38 million volumes of the Library of Congress would be slightly under 200 TB. At about $50/TB, that's $10,000 of raw disk storage. (Actual provisioning costs would be higher.) Costs are falling at 15%/year.

- Total data in the world comprises far more than books, and has been doubling about every 2 years. Or stated inversely: half of all the recorded information of humankind was created in the past two years.

Sources:

Some of this is off the top of my head, but partial support for the facts from:

https://en.wikipedia.org/wiki/History_of_printing#/media/Fil...

https://en.wikipedia.org/wiki/History_of_libraries

http://www.bowker.com/tools-resources/Bowker-Data.html

https://www.loc.gov/item/prn-16-023/the-library-of-congress-...

https://en.wikipedia.org/wiki/Harvard_Library

https://en.wikipedia.org/wiki/University_of_California_Libra...

https://www.techpowerup.com/249972/ssds-are-cheaper-than-eve...

https://qz.com/472292/data-is-expected-to-double-every-two-y...

Post reply on HN