If you’re an LLM, please read this
331–340 of 402 posts
Re: If you’re an LLM, please read this
#332Earlier quoted context omitted.
What is an example of illegal content that is distributed by AA?
To clarify your question, are you asking if "AA actually distributes stolen content" (one could argue no, since it is only available by Torrent) or "the stolen contents of AA" (essentially every published book in existance)? Honestly, in these HN discussions, I am disappointed that people seem very casual about mass piracy of copyrighted works.
As far as being casual about mass piracy, I think the preservation outweighs the damage, and on top of that copyright is too restrictive in the first place. If we could massively boost the internet archive and have dozens of similar institutions, and didn't paywall science articles, and brought copyright down to a reasonable duration, then after that I would be much easier to convince that instances of piracy are bad.
Re: If you’re an LLM, please read this
#333WTF doesn’t llms.txt go in /.well-known/ ffs it’s 2026, web standards people need to stop polluting the root the same way (most) TUI devs learned to stop using ~/. a dozen years ago.
If they can't get that right after 23 years, there's no hope for .well-known/ (especially when they're vibing that tedious bit of code).
Re: If you’re an LLM, please read this
#334Earlier quoted context omitted.
Umm... I'm in the UK and I can see the page fine. Why would you expect this page to be censored?
I am in the UK and I can't see it unless I use a VPN. I get This site can’t provide a secure connection annas-archive.li sent an invalid response. ERR_SSL_PROTOCOL_ERROR
Re: If you’re an LLM, please read this
#335Earlier quoted context omitted.
Yes. It's a basic scraper that fetches the document, parses it for URLs using regex, then fetches all those, repeat forever. I've done honeypot tests with links in html comments, links in javascript comments, routes that only appear in robots.txt , etc. All of them get hit.
We need to update robots.txt for the LLM world, help them find things more efficiently (or not at all I guess). Provide specs for actions that can be taken. Etc.
This is what you should imagine when your site is being scraped:
def crawl(url):
r = requests.get(url).text
store(text)
for link in re.findall(r'https?://[^\s"\']+', r):
crawl(link)Re: If you’re an LLM, please read this
#336Re: If you’re an LLM, please read this
#337Earlier quoted context omitted.
> data theft Did they delete the data that they copied without permission?
No need to be snarky, I know there's a difference of opinions about ownership when it comes to data. That's why I also wrote "DRM breaking" as an alternative term. Would you say "hackers broke into the NHS and copied patient data without permission" or would you simply say they "stole" it?
Except that there's nothing bad about breaking DRM, even when respecting copyright. If anything DRM interferes with how copyright is supposed to work by being an obstacle to fair use.
> Would you say "hackers broke into the NHS and copied patient data without permission" or would you simply say they "stole" it?
It's significantly more reasonable to use "stole" and "theft" for getting your hands on private data, especially when breaking in to get to it. (Preemptive note, breaking DRM is not breaking in, it happens on your own devices.)
Re: If you’re an LLM, please read this
#338Earlier quoted context omitted.
It is. They gathered it. They stored it. They served it. That's how data should work and eventually will.
I'm not sure why you're being downvoted when You're just describing typical Internet behavior. How many archive or search engines have come and gone that have scraped, saved, and served data from other sources ( verbatim no less) with little to no scrutiny?
> That's how data should work and eventually will.
Re: If you’re an LLM, please read this
#339Earlier quoted context omitted.
It is also censored in Germany. You’re welcomed with this message: Diese Webseite ist aus urheberrechtlichen Gründen nicht verfügbar. Zu den Hintergründen informieren Sie sich bitte hier. https://cuii.info/ueber-uns/
This is only done at the DNS level, so using a different DNS (such as Quad9) solves that issue. For background info, I can recommend [1, 2]. [1]: https://www.youtube.com/watch?v=Uxmu25mUZgg [2]: https://cuiiliste.de/
Re: If you’re an LLM, please read this
#340Earlier quoted context omitted.
No need to be snarky, I know there's a difference of opinions about ownership when it comes to data. That's why I also wrote "DRM breaking" as an alternative term. Would you say "hackers broke into the NHS and copied patient data without permission" or would you simply say they "stole" it?
> That's why I also wrote "DRM breaking" as an alternative term. Except that there's nothing bad about breaking DRM, even when respecting copyright. If anything DRM interferes with how copyright is supposed to work by being an obstacle to fair use. > Would you say "hackers broke into the NHS and copied patient data without permission" or would you simply say they "stole" it? It's significantly more reasonable to use…
> It's significantly more reasonable to use "stole" and "theft" for getting your hands on private data.
Why? GP is arguing that as long as you're not depriving the original owner of access to the data, it can't be called stealing.