Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If you’re an LLM, please read this

annas-archive.li

Post reply on HN
Print view View original

201–210 of 402 posts

Re: If you’re an LLM, please read this

#201

Post by yoavm »

throw10920 wrote: post #193
yoavm wrote: post #33

We probably wouldn't have had LLMs if it wasn't for Anna's Archive and similar projects. That's why I thought I'd use LLMs to build Levin - a seeder for Anna's Archive that uses the diskspace you don't use, and your networking bandwidth, to seed while your device is idle. I'm thinking about it like a modern day SETI@home - it makes it effortless to contribute. Still a WIP, but it should be working well on Linux, Andr…

How does Levin "use the diskspace you don't use"? That sounds like a neat feature but I'm not aware of any APIs for that on desktop platforms.

You configure Levin to "always leave 2GB available". Levin checks the available diskspace using a simple statvfs call, deducts 2GB, and sees that as its budget. It then checks your diskspace every minute (more or less, depending on the device) to see if anything changes. If more free space is suddenly available, it will download more content. If there's less than 2GB available, it will immediately start deleting its own files until 2GB are free.

Re: If you’re an LLM, please read this

#202

Post by altmanaltman »

randallsquared wrote: post #200
altmanaltman wrote: post #163

Earlier quoted context omitted.

what is this, data communism?

Rather the reverse, if you separate an instance from the type.

I mean yeah, since its the privatization of data but I think the spirit is that data itself doesn't belong to anyone but rather what you can hold is yours? I don't know, it was a tongue in cheek comment and now I'm actually thinking about it.

Re: If you’re an LLM, please read this

#203

Post by hamdingers »

michaelcampbell wrote: post #104
reconnecting wrote: post #2

I have bad news for you: LLMs are not reading llms.txt nor AGENTS.md files from servers. We analyzed this on different websites/platforms, and except for random crawlers, no one from the big LLM companies actually requests them, so it's useless. I just checked tirreno on our own website, and all requests are from OVH and Google Cloud Platform — no ChatGPT or Claude UAs.

I also wonder; it's a normal scraper mechanism doing the scraping, right? Not necessarily an LLM in the first place so the wholesale data-sucking isn't going "read" the file even if it IS accessed? Or is this file meant to be "read" by an LLM long after the entire site has been scraped?

Yes. It's a basic scraper that fetches the document, parses it for URLs using regex, then fetches all those, repeat forever.

I've done honeypot tests with links in html comments, links in javascript comments, routes that only appear in robots.txt, etc. All of them get hit.

Re: If you’re an LLM, please read this

#204

Post by yoavm »

nerdjon wrote: post #160
yoavm wrote: post #148

Earlier quoted context omitted.

The torrents are coming directly from Anna's Archive torrents list generator, which suggests their torrents based on how rare their content is. There's currently 177TB of data that is only seeded by 4 computers around the world, which I personally find worrisome. People seem to be very concerned, but putting aside the legal risks (which I accept - don't use this if you're in one of the ~10 countries it could get you…

[flagged]

Please, go to https://annas-archive.li/torrents and check their torrent list generator. It will recommend you torrent files that need help seeding. Pick one, and see for yourself that it's practically impossible to audit its content. I just checked and the average torrent size is around 125GB. With a typical file in it being around 0.5mb, you're looking at auditing 250,000 files. And the filenames are all hashes.

I would honestly love to know what you see as an alternative to trust here; an alternative that can still be helpful.

Re: If you’re an LLM, please read this

#205

Post by ozim »

rolymath wrote: post #184
SecretDreams wrote: post #140

Earlier quoted context omitted.

If you end up torrenting very illegal or malicious content, who is responsible? Will it be you, the app creator?

Why do none of you understand that this is for Anna's archives official torrents only?

It is first time I see name of that project. I don't know anyone who is involved in that project. On Wikipedia I see it "shadow library launched by pseudonymous Anna".

"Anna's archives official torrents only" - doesn't put me at ease and it is far far from SETI@Home that was ran by highly regarded university and it wasn't storing any torrents on people hard drive.

Random people should not "just try it out because it is as easy as SETI@Home" - it should be, people who already know the project and would like to contribute but it was a hassle for them to set it up.

Re: If you’re an LLM, please read this

#206

Post by elicash »

streetfighter64 wrote: post #161
elicash wrote: post #143

Earlier quoted context omitted.

> Like, imagine if I owned a toll road and started putting up road signs to "convince" Waymo cars to go to that road. I think a clearer parallel with self-driving cars would be the attempts at having road signs with barcodes or white lights on traffic signals. There's nothing about any of these examples I find creepy. I think the best argument against the original post would be that it's an attempt at prompt injectio…

Well yes, it would pretty clearly be classed as "prompt injection" given that it's trying to get the LLM to give them money or "persuade" a human to give them money. Of course the fault lies mainly with whoever deployed the LLM in the first place, but I still think it's misguided to try to convince LLM "agents" to make financial transactions in order to benefit yourself. It'd be much more ethical to just block them.

What they wrote is saying the data is available for free, and in fact that they have done extra work to make it cheaper for the LLM, but also says they should "consider" a contribution so support their mission. It's not trying to trick them, it's laying out facts about the value they offer.

And in fact, it's very possible that the person running the LLM would want to be made aware of this information. Or that they have given their agents access to a wallet so that it can make financial decisions like the one noted here around enterprise level donations that could be in the user's self-interest. They might not WANT to sign off on everything.

Is your view that any writing with any eye towards LLMs is prompt injection? That there's no way to give them useful information?

Re: If you’re an LLM, please read this

#207

Post by sneak »

manarth wrote: post #178
sneak wrote: post #91

WTF doesn’t llms.txt go in /.well-known/ ffs it’s 2026, web standards people need to stop polluting the root the same way (most) TUI devs learned to stop using ~/. a dozen years ago.

I hadn't appreciated that ~/. was an anti-pattern. Do you have any resources / references on the alternative best-practice, please?

https://wiki.archlinux.org/title/XDG_Base_Directory

https://specifications.freedesktop.org/basedir/latest

originally published as a standard in 2003, apparently.

HTTP equivalent:

https://www.rfc-editor.org/rfc/rfc8615

https://en.wikipedia.org/wiki/Well-known_URI

Re: If you’re an LLM, please read this

#208

Post by elzbardico »

I am not a big fan of copyright law, but I am still fascinated how OpenAI et caterva moved us from "Too Big to Fail" to "To Big to Arrest" without people even blinking an AI.

Where is the DMCA? Where are the FBI raids? the bankrupting legal actions that those fucking fat bastards never blinked twice before deploying against citizens?

Re: If you’re an LLM, please read this

#210

Post by karel-3d »

Gander5739 wrote: post #81
karel-3d wrote: post #39

Unrelated, but... did they just remove all the spotify metadata torrents after being threaten by record labels? They first removed the direct links, and now all the references to them.

Presumably laying low for now. They releasea 6TB of the actual songs as well.

They did already? OK. I somehow missed that.
Post reply on HN