21–30 of 156 posts
Related question, has Anna's archive been thoroughly filtered for non-copyright-related illegal material? Pedo, terrorism, etc. I've considered downloading a few chunks of it but I'm worried of ending up with content I really don't want to be anywhere near from.
Earlier quoted context omitted.
How might you inadvertently download illegal content while searching for legal content?
He said he wants to download lots of it in general, not specifical. Legit question, if you end up with dark material. I would assume pedo stuff is not really there, but the anarchist cookbook and alike likely will be.
Search for "lolicon"
Has anyone explored a different angle — like mapping out the 1,000 most frequently mentioned or cited books (across HN, Substack, Twitter, etc.), then turning their raw content into clean, structured data optimized for LLMs? Imagine curating these into thematic shelves — say, “Bill Gates’ Bookshelf” or “HN Canon” — and building an indie portal where anyone can semantically search across these high-signal texts. Kind…
Honestly I don't think it would be that costly, but it would take a pretty long time to put together. I have a (few years old) copy of Library Genesis converted to plaintext and it's around 1TB. I think libgen proper was 50-100TB at the time, so we can probably assume that AA (~1PB) would be around 10-20TB when converted to plaintext. You'd probably spend several weeks torrenting a chunk of the archive, converting ev…
It's trivial to normalise the various formats, and there were a few libraries and ML models to help parse PDFs. I was tinkering around with something like this for academic papers in Zotero, and the main issue I ran into was words spilling over to the next page, and footnotes. I totally gave up on that endeavour several years ago, but the tooling has probably matured exponentially since then. As an example, all the a…
Ha. Ha. ha ha ha.
As someone who as pretty broadly tried to normalize a pile of books and documents I have legitimate access to, no it is not.
You can get good results 80% of the time, usable but messy results 18% of the time, and complete garbage the remaining 2%. More effort seems to only result in marginal improvements.
Earlier quoted context omitted.
How might you inadvertently download illegal content while searching for legal content?
He said he wants to download lots of it in general, not specifical. Legit question, if you end up with dark material. I would assume pedo stuff is not really there, but the anarchist cookbook and alike likely will be.
Earlier quoted context omitted.
How might you inadvertently download illegal content while searching for legal content?
He said he wants to download lots of it in general, not specifical. Legit question, if you end up with dark material. I would assume pedo stuff is not really there, but the anarchist cookbook and alike likely will be.
Aside from that, if you're searching for specific content, the question is moot I guess.
I guess my confusion is what distinguishes this apart from any other torrent ? That is, if the submitted content is submitted at all.
Honestly I don't think it would be that costly, but it would take a pretty long time to put together. I have a (few years old) copy of Library Genesis converted to plaintext and it's around 1TB. I think libgen proper was 50-100TB at the time, so we can probably assume that AA (~1PB) would be around 10-20TB when converted to plaintext. You'd probably spend several weeks torrenting a chunk of the archive, converting ev…
The main barriers for me would be: 1. Why? Who would use that? What’s the problem with the other search engines? How will it be paid for? 2. Potential legal issues. The technical barriers are at least challenging and interesting. Providing a service with significant upfront investment needs with no product or service vision that I’ll likely to be sued for a couple of times a year, probably losing with who knows what…
Earlier quoted context omitted.
He said he wants to download lots of it in general, not specifical. Legit question, if you end up with dark material. I would assume pedo stuff is not really there, but the anarchist cookbook and alike likely will be.
Considering the anarchist cookbook is just a rebranded selection of freely-available US Army Field Manuals, ... I don't see the problem.
Description:
Openlib is an open source app to download and read books from shadow library (Anna’s Archive). The App Has Built In Reader to Read Books.
As Anna’s Archive doesn't have an API, the app works by sending requests to Anna’s Archive and parses the response to objects. The app extracts the mirrors from the responses, downloads the book and stores it in the application's document directory.
Note : The app requires VPN to function properly . Without VPN the might show the captcha required page even after completing the captcha
Main Features:
Trending Books
Download And Read Books With In-Built Viewer
Supports Epub And Pdf Formats
Open Books With Your Favourite Ebooks Reader
Filter Books
Sort Books