Earlier quoted context omitted.
I wonder why we broke the web.
It seems there's little agreement over how the web is broken.
People who love cookie banners either don't exist, or are alien invaders :)
Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
41–50 of 63 posts
Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#42We broke the web so badly for humans that we had to build a clean web for machines, and now humans will have to use machines to experience a clean web again.
Yeah, when browsers have a "reader mode", it's pretty obvious the plot has been lost somewhere.
Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#43Not really, but sounds interesting. Would you care to share some sites that offer better llms.txt than main web page? Or talk about some piece of info you easily found on llms.txt that was hard to navigate to on the regular website?
Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#44Earlier quoted context omitted.
I can definitively say llms.txt is not used by any AI players. I run a blogging platform with around 80k blogs and /llms.txt is not requested by anything (other than humans checking to see if there's an llms.txt path). All regular pages are aggressively scraped to the extent it's a problem I have to consistently manage, but not llms.txt.
> I can definitively say llms.txt is not used by any AI players. https://developers.openai.com/llms.txt https://docs.anthropic.com/llms.txt https://geminicli.com/llms.txt https://github.com/llms.txt https://docs.aws.amazon.com/llms.txt https://openrouter.ai/docs/llms.txt
OP clearly meant that the AI players are not reading and/or honouring llms.txt of other websites when scraping.
Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#45No, the spammers are just at the beginning of ruining that too
https://news.ycombinator.com/item?id=48411569
BTW why should Chrome even consider rendering a .txt file as markdown?
Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#46Earlier quoted context omitted.
> I can definitively say llms.txt is not used by any AI players. https://developers.openai.com/llms.txt https://docs.anthropic.com/llms.txt https://geminicli.com/llms.txt https://github.com/llms.txt https://docs.aws.amazon.com/llms.txt https://openrouter.ai/docs/llms.txt
OP clearly meant that the AI players are not reading and/or honouring llms.txt of other websites when scraping.
i stand corrected, but what was clear to you, obviously was not clear to me.
Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#47Why didn't they place it in .well-known? Also, I couldn't find a website that has it.
Putting it in .well-known/ was immediately raised as an issue from the beginning; it’s issue #2 in fact:
https://github.com/AnswerDotAI/llms-txt/issues/2
It’s been completely ignored ever since.
Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#48Not really, but sounds interesting. Would you care to share some sites that offer better llms.txt than main web page? Or talk about some piece of info you easily found on llms.txt that was hard to navigate to on the regular website?
llms.txt usually includes a clear sitemap and description of information available on a site.
There are also clear definition of the restful scheme and API/data access options.
One very basic example would be the weather channel https://weather.com/llms.txt