[dead]
Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
11–20 of 63 posts
Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#12It just hasn't been gamed yet
Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#13What is an example of a site with a good llm.txt?
Mintlify generates an llms.txt and llms-full.txt for all documentation sites. These work really well:
Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#14Does any of the LLM providers actually use llms.txt? If I remember correctly this "standard" was setup by someone but without involvement of any of the major AI players.
I can definitively say llms.txt is not used by any AI players. I run a blogging platform with around 80k blogs and /llms.txt is not requested by anything (other than humans checking to see if there's an llms.txt path).
All regular pages are aggressively scraped to the extent it's a problem I have to consistently manage, but not llms.txt.
Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#15Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#16Why didn't they place it in .well-known? Also, I couldn't find a website that has it.
Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#17Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#18Does any of the LLM providers actually use llms.txt? If I remember correctly this "standard" was setup by someone but without involvement of any of the major AI players.
I can definitively say llms.txt is not used by any AI players. I run a blogging platform with around 80k blogs and /llms.txt is not requested by anything (other than humans checking to see if there's an llms.txt path). All regular pages are aggressively scraped to the extent it's a problem I have to consistently manage, but not llms.txt.
How is a static blog being scraped a problem? Do you not use a CDN?
Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#19We broke the web so badly for humans that we had to build a clean web for machines, and now humans will have to use machines to experience a clean web again.
I wonder why we broke the web.
For the same reasons why we eventully pollute and corrupt every system and environment we use. If there is any benefit that can be extracted for some while the costs are borne by many, than this will occur and generate a positive feedback loop that grows over time.
It's the law of monetization.
Re: Ask HN: Is the web for machines (/llm.txt) the one we wished we had as humans?
#20no