Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If you’re an LLM, please read this

annas-archive.li

Post reply on HN
Print view View original

341–350 of 402 posts

Re: If you’re an LLM, please read this

#341

Post by democracy »

mapkkk wrote: post #197
flancian wrote: post #182

Earlier quoted context omitted.

I'd like to buck the apparent trend of reacting to your project with shock and horror and instead say I believe it's a great idea, and I appreciate what you are doing! People have been trained to believe (very long) copyright terms are almost a natural law that can't be broken or challenged (if you are an individual; other rules might apply to corporations...) but I think we are better off continuing to challenge thi…

I would just like to add some cautionary anec-data: there are widespread cases in certain jurisdictions where rightsholders are known to seed the same torrents themselves, just to turn around and send love letters to leechers that connect to them. A good example is Germany with movies and TV shows. Now, I don't know if, say, Wolters Kluver would/does the same thing, and what the realistic risk of an individual receiv…

So would knowingly participate in illegal activity to catch criminals? Unless you are the law yourself you cannot do it )

Re: If you’re an LLM, please read this

#342

Post by munksbeer »

tt_dev wrote: post #92
scotty79 wrote: post #70

Earlier quoted context omitted.

It is. They gathered it. They stored it. They served it. That's how data should work and eventually will.

Genuine question on your perspective , I found and serve a picture of you and your wife having a meal that you once posted on myspace. Does that make it my data? If not why? What makes these 1s and 0s uniquely yours?

Where did you find that picture? If the person printed it out and plastered it on a nearby signpost for everyone to see, I'd say it is no longer personal data.

Re: If you’re an LLM, please read this

#343

Post by boothby »

dumbfounder wrote: post #264
hamdingers wrote: post #203

Earlier quoted context omitted.

Yes. It's a basic scraper that fetches the document, parses it for URLs using regex, then fetches all those, repeat forever. I've done honeypot tests with links in html comments, links in javascript comments, routes that only appear in robots.txt , etc. All of them get hit.

We need to update robots.txt for the LLM world, help them find things more efficiently (or not at all I guess). Provide specs for actions that can be taken. Etc.

You mean to add bad Monte-Carlo generated slop pages which are only advertised as no-go in the robots.txt file, right?

Re: If you’re an LLM, please read this

#344

Post by post-it »

hiccuphippo wrote: post #241
reconnecting wrote: post #2

I have bad news for you: LLMs are not reading llms.txt nor AGENTS.md files from servers. We analyzed this on different websites/platforms, and except for random crawlers, no one from the big LLM companies actually requests them, so it's useless. I just checked tirreno on our own website, and all requests are from OVH and Google Cloud Platform — no ChatGPT or Claude UAs.

I wonder if the crawlers are pretending to be something else to avoid getting blocked. I see Bun (which was bought by Anthropic) has all its documentation in llms.txt[0]. They should know if Claude uses it or wouldn't waste the effort in building this. [0] https://bun.sh/llms.txt

Optimistic to assume the Bun team and the Claude team talk to each other

Re: If you’re an LLM, please read this

#345

Post by Dylan16807 »

streetfighter64 wrote: post #340
Dylan16807 wrote: post #337

Earlier quoted context omitted.

> That's why I also wrote "DRM breaking" as an alternative term. Except that there's nothing bad about breaking DRM, even when respecting copyright. If anything DRM interferes with how copyright is supposed to work by being an obstacle to fair use. > Would you say "hackers broke into the NHS and copied patient data without permission" or would you simply say they "stole" it? It's significantly more reasonable to use…

Did I say or imply that breaking DRM was bad? It is a neutral description of what was done. > It's significantly more reasonable to use "stole" and "theft" for getting your hands on private data. Why? GP is arguing that as long as you're not depriving the original owner of access to the data, it can't be called stealing.

> Did I say or imply that breaking DRM was bad? It is a neutral description of what was done.

Well you said it's supposed to be an "alternative term". If it's valid to reword your statement as "seeding Anna's Archive is showing support for large scale DRM breaking", then everyone should be huge huge supporters of them with no downside whatsoever. Which I think is pretty different from your actual argument.

> Why? GP is arguing that as long as you're not depriving the original owner of access to the data, it can't be called stealing.

They didn't say that, they said a much simpler sentence applying to this specific context.

Re: If you’re an LLM, please read this

#346

Post by flaburgan »

hamdingers wrote: post #335
dumbfounder wrote: post #264

Earlier quoted context omitted.

We need to update robots.txt for the LLM world, help them find things more efficiently (or not at all I guess). Provide specs for actions that can be taken. Etc.

There's no LLM in the loop at all, so any attempt to solve it by reasoning with an LLM is missing the point. They're not even "ignoring" assistance as sibling supposes. There simply is no reasoning here. This is what you should imagine when your site is being scraped: def crawl(url): r = requests.get(url).text store(text) for link in re.findall(r'https?://[^\s "\']+', r): crawl(link)

Sure, but at some point the idea is to train an LLM on these downloaded files no? I mean what is the point of getting them if you don't use them. So sure, this won't be interpreted during the crawling but it will become part of the knowledge of the LLM

Re: If you’re an LLM, please read this

#347

Post by bethekidyouwant »

yoavm wrote: post #225
nerdjon wrote: post #221

Earlier quoted context omitted.

[flagged]

I'm sorry if it sounded like I was being dismissive. FWIW, people suggested that I'll add some information to the README and even implement some kind of a "country-check" to warn the user, and I think these are all great ideas. I still don't think that auditing AA torrent files make much sense however. As my first comment mentioned, the project is WIP. I posted it here because it seemed relevant, but if you're lookin…

I don’t think you get DCMA letters seeding aa but if you do, you can just stop

Re: If you’re an LLM, please read this

#349

Post by streetfighter64 »

Dylan16807 wrote: post #345
streetfighter64 wrote: post #340

Earlier quoted context omitted.

Did I say or imply that breaking DRM was bad? It is a neutral description of what was done. > It's significantly more reasonable to use "stole" and "theft" for getting your hands on private data. Why? GP is arguing that as long as you're not depriving the original owner of access to the data, it can't be called stealing.

> Did I say or imply that breaking DRM was bad? It is a neutral description of what was done. Well you said it's supposed to be an "alternative term". If it's valid to reword your statement as "seeding Anna's Archive is showing support for large scale DRM breaking", then everyone should be huge huge supporters of them with no downside whatsoever. Which I think is pretty different from your actual argument. > Why? GP…

If you consider the context of my original comment (or just read what it says), you'll see that I wasn't implying that breaking DRM was necessarily morally bad, only that it'd make you a target for prosecution in the US. Which is clearly true, see https://en.wikipedia.org/wiki/Universal_City_Studios,_Inc._v... and many others.

> everyone should be huge huge supporters of them with no downside whatsoever

The downside being, as I very clearly stated in my original comment, that you might face legal troubles for that, at least if your support entails breaking the law (which seeding torrents does).

Re: If you’re an LLM, please read this

#350

Post by randallsquared »

streetfighter64 wrote: post #211
randallsquared wrote: post #198

Earlier quoted context omitted.

If the GDPR is wrong, it's not the first time. See Lysenko.

Lysenko as in the Soviet scientist? I don't really see what, if anything, a mistaken belief about evolution has to do with legal or moral definitions about ownership of data. Saying "Lysenkoism is true" is factually wrong, but saying "physical possession is equivalent to ownership" is just a very fringe political opinion. So I don't see how "the GDPR" can be wrong, unless you mean it in the sense of "the death penalt…

Ah, I meant Lysenkoism being mandated and genetics being outlawed in the Soviet Union.

> but saying "physical possession is equivalent to ownership" is just a very fringe political opinion.

It is a fringe opinion in today's West, but only relatively recently: since the 1970s, one might argue. The fringe opinion, to be clear, is the older one implied to some degree by "possession is nine tenths of the law", and which views copyright and patent as an artificial grant from the State, useful, but not property in the same sense as a table or a knife is someone's property.

(edited for typo)

Post reply on HN