Pirate Library Mirror: Preserving 7TB of books (that are not in Libgen)

From the perspective of how much you individually could read in a lifetime: that's about 4,000 weeks (to roughly age 80-ish, and presuming you're not already reading at birth). Multiply that by the number of books you plan on reading a week. You'll probably read fewer than 4,000 books over your entire life. Many people read few if any books after graduating secondary school, and even a highly-motivated reader might be challenged to crack, say, 40,000 (ten per week for life).

The US Library of Congress has the world's largest book collection at about 40 million distinct titles.[1] There are another 130 million total catalogued items, including photographs, films, audio recordings, maps, pamphlets, and other items. Not all are textual. I'll stick to books.

At 4,000 per lifetime, you'd need 10,000 people simply to read all of the LoC's collection at a reasonable pace.

(By comparison, at the turn of the 20th century, the LoC's annual report to Congress noted that its cataloguing department could handle about 3,000 books per cataloguer per year, or a pace of about 60/week or 15/day. Over a 40-year career, a cataloguer might handle, if not read completely, 120,000 books.)

As a rough rule of thumb, a digitised ebook in PDF format runs about 5 MB.

Every single book in the Library of Congress in digital format would occupy about 200 TB of storage.

Current disk prices are running about $5 -- $20 / TB.

200 TB in raw disk would set you back $1,000 -- $4,000. Figure 2-4x multiplier for a disk storage system all told, and it's still roughly $4k -- $16k to have as local storage every last book in the US Library of Congress. That's well within scope of a moderately wealthy US household budget, and would be reasonable to consider for a small-town city library.

That's the technical storage cost, obviously not the rights or aquisition costs for the materials.

The Library of Congress's budget is about $800 million/yr.

Those 4,000 books you might read in a lifetime? They'd fit on about 20 GB worth of disk. If you're a 10x reader, 200 GB, and a truly dedicated 100x reader, about 2 TB. Those are well within the range of present desktop / laptop disk allocations, and represent a few tens of dollars of storage expense.

The Internet Archive computes $2/GB for storage in perpetuity. That's roughly 400 books worth of storage.

But a small household NAS and a very modest server platform (most routers can effectively operate as media servers) could rival a mid-sized city library for about $100 or less in actual hardware outlay. Those prices are falling by half about every 18 to 36 months, as they have been for decades.

Books are not all published content, and all published content is not all human knowledge. But standard published books are a good proxy for total cultural knowledge, and in all likelihood, then some.

________________________________

Notes:

1. See: https://www.loc.gov/about/general-information/ That includes about 25 million books in the main collection, and 15 million in "nonclassified print collections, including books in large type and raised characters, incunabula (books printed before 1501), monographs and serials, music, bound newspapers, pamphlets, technical reports and other printed material". I suspect you could roughly halve my estimates above based on 25 million vs. 40 million volumes, as many of the nonclassified works may duplicate the main collection.

By copyright expiry, public domain in the US begins in 1927. Later works may be in the public domain, but all works published prior to 1927 are in the public domain in the United States.

(This may not be the case in other countries.)

There were not many published books prior to the invention of the printing press, and many of those didn't survive. The total number of books (not individual titles but actualy bound volumes entirely) in Western Europe as of 1400 may have been as few as 50,000.

By 1800, about 1 million titles had been printed.

Over the course of the 18th century, presses became vastly faster, as they evolved from hand-operated wooden screw-press to iron-frames to steam and electric-powered rotary and ultimately web presses. Paper became much cheaper (and less durable --- a factor commented on at length in the Librarian of Congress's annual reports to Congress in the late 19th century). Literacy exploded from ~25% to 95%+ over the 19th century (and probably accounted for numerous revolutions and political upheavals).

Through much of the 20th century, certainly by 1950, US publishers were issuing about 300,000 new titles per year, a rate which state remarkably constant through the early 21st century. By the aughts, "nontraditional" self-publishing (a/k/a "vanity press") was nearing or exceeding 1 million titles per year more than had been published through all time to 1800.

Reports that all recorded data was doubling every few years date to at least the 1960s. That would mean that in any two year period ... half of all recorded information was less than two years old.

The catch is that not all recorded data is published. So I'm not sure what the time-distribution of all publishing looks like. But I'm pretty confident it's skewed far more recently than 1927. And would thus tend to be copyrighted rather than uncopyrighted.

If you want to measure works by significance, you might make a different argument --- there are many great works of literature, philosophy, history, and religion which were first published before 1927. But ranking and tabulating these is more challenging than a simple enumeration.

Germany now is a leading copyright-maximalist power, unfortunately.

________________________________

Notes:

1. I say nation and not country as the German language and cultural nationality pre-existed the German state in 1871.