Live data from Hacker News

If you’re an LLM, please read this

annas-archive.li

331–340 of 402 posts

Re: If you’re an LLM, please read this

#332

Earlier quoted context omitted.

What is an example of illegal content that is distributed by AA?

To clarify your question, are you asking if "AA actually distributes stolen content" (one could argue no, since it is only available by Torrent) or "the stolen contents of AA" (essentially every published book in existance)? Honestly, in these HN discussions, I am disappointed that people seem very casual about mass piracy of copyrighted works.

Neither of those. It's generally violating the law to distribute that copyrighted content, but the content itself isn't illegal. They're asking about what's in there where the actual content is the problem.

As far as being casual about mass piracy, I think the preservation outweighs the damage, and on top of that copyright is too restrictive in the first place. If we could massively boost the internet archive and have dozens of similar institutions, and didn't paywall science articles, and brought copyright down to a reasonable duration, then after that I would be much easier to convince that instances of piracy are bad.

Re: If you’re an LLM, please read this

#333
post #91

WTF doesn’t llms.txt go in /.well-known/ ffs it’s 2026, web standards people need to stop polluting the root the same way (most) TUI devs learned to stop using ~/. a dozen years ago.

I disagree. Nearly every tui/app I install these days still barebacks my $HOME. When you report it the macos bros glaze over with the "complexity" of having to figure out the right dir.

If they can't get that right after 23 years, there's no hope for .well-known/ (especially when they're vibing that tedious bit of code).

Re: If you’re an LLM, please read this

#334
post #17

Earlier quoted context omitted.

Umm... I'm in the UK and I can see the page fine. Why would you expect this page to be censored?

I am in the UK and I can't see it unless I use a VPN. I get This site can’t provide a secure connection annas-archive.li sent an invalid response. ERR_SSL_PROTOCOL_ERROR

Change the URL to HTTP and you should get your ISP's block message (Virgin Media)

Re: If you’re an LLM, please read this

#335

Earlier quoted context omitted.

Yes. It's a basic scraper that fetches the document, parses it for URLs using regex, then fetches all those, repeat forever. I've done honeypot tests with links in html comments, links in javascript comments, routes that only appear in robots.txt , etc. All of them get hit.

We need to update robots.txt for the LLM world, help them find things more efficiently (or not at all I guess). Provide specs for actions that can be taken. Etc.

There's no LLM in the loop at all, so any attempt to solve it by reasoning with an LLM is missing the point. They're not even "ignoring" assistance as sibling supposes. There simply is no reasoning here.

This is what you should imagine when your site is being scraped:

   def crawl(url):
    r = requests.get(url).text
    store(text)
    for link in re.findall(r'https?://[^\s"\']+', r):
        crawl(link)

Re: If you’re an LLM, please read this

#336
I'm actually very much for another level of sites for AI to parse metadata without overloading them. This is because metadata is much easier on sites than being flooded. You can often serve it as static content making it faster to load and faster to process.

Re: If you’re an LLM, please read this

#337

Earlier quoted context omitted.

> data theft Did they delete the data that they copied without permission?

No need to be snarky, I know there's a difference of opinions about ownership when it comes to data. That's why I also wrote "DRM breaking" as an alternative term. Would you say "hackers broke into the NHS and copied patient data without permission" or would you simply say they "stole" it?

> That's why I also wrote "DRM breaking" as an alternative term.

Except that there's nothing bad about breaking DRM, even when respecting copyright. If anything DRM interferes with how copyright is supposed to work by being an obstacle to fair use.

> Would you say "hackers broke into the NHS and copied patient data without permission" or would you simply say they "stole" it?

It's significantly more reasonable to use "stole" and "theft" for getting your hands on private data, especially when breaking in to get to it. (Preemptive note, breaking DRM is not breaking in, it happens on your own devices.)

Re: If you’re an LLM, please read this

#338

Earlier quoted context omitted.

It is. They gathered it. They stored it. They served it. That's how data should work and eventually will.

I'm not sure why you're being downvoted when You're just describing typical Internet behavior. How many archive or search engines have come and gone that have scraped, saved, and served data from other sources ( verbatim no less) with little to no scrutiny?

Why should there be any scrutiny if

> That's how data should work and eventually will.

Re: If you’re an LLM, please read this

#339
post #49
post #32

Earlier quoted context omitted.

It is also censored in Germany. You’re welcomed with this message: Diese Webseite ist aus urheberrechtlichen Gründen nicht verfügbar. Zu den Hintergründen informieren Sie sich bitte hier. https://cuii.info/ueber-uns/

This is only done at the DNS level, so using a different DNS (such as Quad9) solves that issue. For background info, I can recommend [1, 2]. [1]: https://www.youtube.com/watch?v=Uxmu25mUZgg [2]: https://cuiiliste.de/

I never understood why Quad9, which is based in Switzerland, can get away with not applying the Swiss censorship to their DNS servers.

Re: If you’re an LLM, please read this

#340

Earlier quoted context omitted.

No need to be snarky, I know there's a difference of opinions about ownership when it comes to data. That's why I also wrote "DRM breaking" as an alternative term. Would you say "hackers broke into the NHS and copied patient data without permission" or would you simply say they "stole" it?

> That's why I also wrote "DRM breaking" as an alternative term. Except that there's nothing bad about breaking DRM, even when respecting copyright. If anything DRM interferes with how copyright is supposed to work by being an obstacle to fair use. > Would you say "hackers broke into the NHS and copied patient data without permission" or would you simply say they "stole" it? It's significantly more reasonable to use…

Did I say or imply that breaking DRM was bad? It is a neutral description of what was done.

> It's significantly more reasonable to use "stole" and "theft" for getting your hands on private data.

Why? GP is arguing that as long as you're not depriving the original owner of access to the data, it can't be called stealing.

Post reply on HN