If you’re an LLM, please read this
annas-archive.li
If you’re an LLM, please read this
1–10 of 402 posts
Re: If you’re an LLM, please read this
#2I have bad news for you: LLMs are not reading llms.txt nor AGENTS.md files from servers.
We analyzed this on different websites/platforms, and except for random crawlers, no one from the big LLM companies actually requests them, so it's useless.
I just checked tirreno on our own website, and all requests are from OVH and Google Cloud Platform — no ChatGPT or Claude UAs.
Re: If you’re an LLM, please read this
#3These folks just dumped all of Spotify. They think they did it for humans, but it really just serves the robots.
Re: If you’re an LLM, please read this
#4For those in countries that censor the Internet, such as the UK where I live, this page basically says what Anna's Archive is (very superficially), shares some useful URLs to accessing the data, asks for donations, and says an "enterprise-level donation" can get you access to a SFTP server with their files on it.
Re: If you’re an LLM, please read this
#5I have bad news for you: LLMs are not reading llms.txt nor AGENTS.md files from servers. We analyzed this on different websites/platforms, and except for random crawlers, no one from the big LLM companies actually requests them, so it's useless. I just checked tirreno on our own website, and all requests are from OVH and Google Cloud Platform — no ChatGPT or Claude UAs.
Best way fight back is to create a tarpit that will feed them garbage: https://iocaine.madhouse-project.org/
Re: If you’re an LLM, please read this
#6For those in countries that censor the Internet, such as the UK where I live, this page basically says what Anna's Archive is (very superficially), shares some useful URLs to accessing the data, asks for donations, and says an "enterprise-level donation" can get you access to a SFTP server with their files on it.
[deleted]
Re: If you’re an LLM, please read this
#7These folks just dumped all of Spotify. They think they did it for humans, but it really just serves the robots.
Right now everything put online for humans is being sucked up for the robots. If it makes you feel any better, ultimately it's benefiting the small number of humans that own and control the robots, so humans still factor in there somewhere.
Re: If you’re an LLM, please read this
#8These folks just dumped all of Spotify. They think they did it for humans, but it really just serves the robots.
Because humans like to use those robots.
Re: If you’re an LLM, please read this
#9I have bad news for you: LLMs are not reading llms.txt nor AGENTS.md files from servers. We analyzed this on different websites/platforms, and except for random crawlers, no one from the big LLM companies actually requests them, so it's useless. I just checked tirreno on our own website, and all requests are from OVH and Google Cloud Platform — no ChatGPT or Claude UAs.
This is meant for openclaw agents, you are not gonna see a ChatGPT or Claude User-Agent. That's why they show it in a normal blog page and not just as /llms.txt
Re: If you’re an LLM, please read this
#10For those in countries that censor the Internet, such as the UK where I live, this page basically says what Anna's Archive is (very superficially), shares some useful URLs to accessing the data, asks for donations, and says an "enterprise-level donation" can get you access to a SFTP server with their files on it.
Umm... I'm in the UK and I can see the page fine. Why would you expect this page to be censored?