Live data from Hacker News

Pirate Library Mirror: Preserving 7TB of books (that are not in Libgen)

pilimi.org

111–120 of 438 posts

Re: Pirate Library Mirror: Preserving 7TB of books (that are not in Libgen)

#111
post #78
post #61

Earlier quoted context omitted.

I assume Google abandoned this along with all their earlier mission statements in favour of building another chat app.

I can't imagine a life at google. So much promise, all turned to ash.

More like an ad company driven by dark patterns that needs to pretend being good and nice and innovative as otherwise it will lose goodwill just like Facebook did.

Re: Pirate Library Mirror: Preserving 7TB of books (that are not in Libgen)

#112

It's a shame that this would have been a textbook case for using IPFS and yet that wasn't the default. Books are naturally immutable, and could be structured into sub-categories whilst enjoying the benefits of deduplication.

That would paint a huge red target mark on filecoin's back, given that they raised over a quarter-billion. They would take it down faster than you can have an 8TB HDD shipped to you.

Re: Pirate Library Mirror: Preserving 7TB of books (that are not in Libgen)

#113

I wonder if there are any search engines dedicated to indexing these kinds of libraries. I know there's a decent one just for scihub, but it would be awesome if I could do a Google-style search that returned the contents of books, magazines and journal articles instead of just websites.

Wasn't that what google books was supposed to be?

Google Books, like so many Google projects, had a dual purpose. Making books accessible is noble and on-mission. But more importantly natural language models can be trained on the scanned corpus.

The same was true of the original GOOG 411, which provided a free service, but was really put in place to train up their voice recognition projects.

This is a long running strategy of Google, and it's a shrewd one. The main thing is not to mistake it for a public good. It is an act of privatization.

Re: Pirate Library Mirror: Preserving 7TB of books (that are not in Libgen)

#115

It's a shame that this would have been a textbook case for using IPFS and yet that wasn't the default. Books are naturally immutable, and could be structured into sub-categories whilst enjoying the benefits of deduplication.

That would paint a huge red target mark on filecoin's back, given that they raised over a quarter-billion. They would take it down faster than you can have an 8TB HDD shipped to you.

They can't take it down.

Re: Pirate Library Mirror: Preserving 7TB of books (that are not in Libgen)

#116
post #110
post #92

Earlier quoted context omitted.

There is no ssl so no cert fingerprint shodan matching leak

Mind elaborating?

“They” deanonymize anonymous site operators through cert reuse because people get lazy/stupid.

Probably 100% doesn’t matter in this case.

Re: Pirate Library Mirror: Preserving 7TB of books (that are not in Libgen)

#117

Earlier quoted context omitted.

ESNI has been dropped, a new spec alters how it works and renames it Encrypted client hello (ECH) https://blog.mozilla.org/security/2021/01/07/encrypted-clien...

ECH looks quite interesting, but isn't it quite easy to do a reverse DNS lookup for most domains?

The answer is no. There was a Cloudflare article on ECH a while back that mentioned the fallibility of using reverse DNS, but I am having trouble locating it. In any event, the people working on ECH have coined a term called the "anonymity set". Below is a Cloudflare article that uses this term.

https://blog.cloudflare.com/handshake-encryption-endgame-an-...

The "anonymity set" refers to the number of possible domains using a single IP address. The existence of that term implies that some IP addresses must have a number of domains associated with them, greater than 1. With these IP addresses, one cannot determine the domain name, the one that the www user sent, from a PTR query alone. Even prior to the introduction of SNI to TLS, when the only way to offer HTTPS was by using a dedicated IP address, discovering the contents of the encrypted Host header via reverse DNS was neither easy nor reliable.

If there are still people reading HN who believe that reverse DNS is reliable and makes plaintext SNI and ECH moot, and are going to comment as such in the future, I would be happy to post the results of an experiment where I take the DNS data for all the domains currently submitted to HN, i.e., a list of IP addresses found in the A records for these names, and do a PTR on each one. We can look at whether "most domains" are identifiable through PTR records.

Also remember the question is not whether ECH protects 100% from someone discovering what domain name the user sent. It does not. The question is whether ECH makes it more difficult to discover than simply sniffing plaintext SNI on the wire, which, of course, is even easier and more reliable than reverse DNS.

Re: Pirate Library Mirror: Preserving 7TB of books (that are not in Libgen)

#118
post #80
post #61

Earlier quoted context omitted.

I assume Google abandoned this along with all their earlier mission statements in favour of building another chat app.

Could you take a moment to check it Google Books search still exists? I'll give you a hint: https://books.google.com/?hl=en > Search the world's most comprehensive index of full-text books.

> I'll give you a hint: https://books.google.com/?hl=en

>> Search the world's most comprehensive index of full-text books.

I mean, the domain resolves, but that doesn't mean the product exists. You can run searches, but you're not allowed to see the results.

Re: Pirate Library Mirror: Preserving 7TB of books (that are not in Libgen)

#119

It's a shame that this would have been a textbook case for using IPFS and yet that wasn't the default. Books are naturally immutable, and could be structured into sub-categories whilst enjoying the benefits of deduplication.

IPFS doesn't really work well for this because you'd never know if the peer hosting the last subset of some books went offline (and you'd lose those until someone who had them came online again).

I want a slightly different system, which I've posted about here: https://news.ycombinator.com/item?id=31972252

Re: Pirate Library Mirror: Preserving 7TB of books (that are not in Libgen)

#120
I see things like this, and I wonder why the following software doesn't exist:

I want a piece of software to which I can add a collection of files, say multiple TB. The software will then behave a bit like a BitTorrent tracker, and know which peer has which files. A peer joining this swarm will be able to say "I want to donate X GB of space", and the tracker would tell it "OK, then download and seed these files, which are the least seeded".

The peer would download the files from the rest of the swarm and make them available to it. Then, a request layer on top of the swarm could be used to request a file from the peer which had it. Adding/removing files to this collection would also need to be a feature.

Does anyone know if anything like this exists? If not, how easy would it be to make something like it out of BitTorrent? I might give it a go.

Post reply on HN