Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llms.txt

llmstxt.org

Post reply on HN
Print view View original

121–130 of 191 posts

Re: Llms.txt

#121

Post by phito »

idf00 wrote: post #103
yochem wrote: post #94

Earlier quoted context omitted.

Yes, and .py is "plain" text too. The extension however helps with signaling the intend of the file. Also, there is something to say for the argument "there is no such thing as plain text" [0] [0]: https://youtu.be/gd5uJ7Nlvvo

If you had python code and you didn't want it to have syntax highlighting or be run/imported or any of the other normal things that you do with python files, it might make sense to have python code in a .txt. file. Same idea here IMO. .md would signal the wrong intent, as you don't want to render it to markdown formatting or read as a markdown file normally is. You want it to be read as plain unrendered text. Sam

Why even have an extension?

Unrelated, but the comment two steps above has the same username pattern as yours (3 letters+00)

Re: Llms.txt

#122

Post by sbergot »

Tepix wrote: post #91
JimDabell wrote: post #10

This is not how these kinds of things should be designed for the web. Instead of putting resources in the root of the web, this is what /.well-known/ was designed for. See RFC 5785: https://datatracker.ietf.org/doc/html/rfc5785 Instead of munging URLs to get alternate formats, this is what content negotiation or rel=alternate were designed for. I’m not sure making it easier to consume content is something that is nee…

I think the whole idea of extra instructions required for LLMs is unnecessary. A decent LLM should be able to handle browsing the site, if needed it can use the sitemap. It can hopefully also figure out what the various sections are about.

I agree. There are enough standard places to put metadata in a website.

Re: Llms.txt

#123

Post by tpoacher »

Eyas wrote: post #112

Was I the only one that found `docs.fastht.ml/llms.txt` more useful than both fastht.ml and docs.fastht.ml? Zooming out, it's interesting how many (especially dev-focused) tools & frameworks have landing sites that are so incomprehensible to me. They look like marketing sites but don't even explain what the thing they're offering does. llms.txt almost sounds like a forcing function for someone to write something that…

This! I would be in favour of this proposal, if only simply so that I can make the llms.txt file my next point of call for actual information when the human-facing page sucks.

Re: Llms.txt

#124

Post by nl »

Joker_vD wrote: post #19
JimDabell wrote: post #10

This is not how these kinds of things should be designed for the web. Instead of putting resources in the root of the web, this is what /.well-known/ was designed for. See RFC 5785: https://datatracker.ietf.org/doc/html/rfc5785 Instead of munging URLs to get alternate formats, this is what content negotiation or rel=alternate were designed for. I’m not sure making it easier to consume content is something that is nee…

If only this RFC was well-known among the people who actually put stuff out on the Web.

It's very broadly used.

For example:

- Apple uses it for their app to website association files

- OpenID Connect uses it for connection discovery

- security.txt is usually served from .well-known

- JSON Web Tokens uses it for Web Key Sets to verify public keys

- LetsEncrypt uses it for its ACME HTTPS verification protocol.

There's probably more, but these are ones I've personally used.

Re: Llms.txt

#126

Post by blenderob »

Anyone else worried how backward this sounds? I mean this is like totally giving up on the dismal state of website UXes these days and gladly accepting that website navigation and experience should remain utterly confusing for humans but machines (yes, machines) should get preferential treatment! Good UX is now for machines, not for humans!

Shouldn't something like this be first and foremost for humans ... which also benefits machines as an obvious side-effect?

Re: Llms.txt

#127

Post by spencerchubb »

blenderob wrote: post #126

Anyone else worried how backward this sounds? I mean this is like totally giving up on the dismal state of website UXes these days and gladly accepting that website navigation and experience should remain utterly confusing for humans but machines (yes, machines) should get preferential treatment! Good UX is now for machines, not for humans! Shouldn't something like this be first and foremost for humans ... which also…

It's recognizing that the needs of a human are different from the needs of an LLM.

Re: Llms.txt

#128

Post by lolinder »

JimDabell wrote: post #10

This is not how these kinds of things should be designed for the web. Instead of putting resources in the root of the web, this is what /.well-known/ was designed for. See RFC 5785: https://datatracker.ietf.org/doc/html/rfc5785 Instead of munging URLs to get alternate formats, this is what content negotiation or rel=alternate were designed for. I’m not sure making it easier to consume content is something that is nee…

OP replied to this complaint in a top-level comment:

https://news.ycombinator.com/item?id=41442092

They're correct that the RFC technically requires registration, and looking through the existing list of contact information for registrations I'd be likewise intimidated to attempt to register something that's experimental.

Re: Llms.txt

#129

Post by sodality2 »

idf00 wrote: post #100
mrweasel wrote: post #87

Earlier quoted context omitted.

> if you create an llms.txt you're not looking to manipulate anyone You don't know me :-) My suggestion is that someone might want taint the data that goes into an LLM. Let's say you have a website with guides, examples and tips and tricks for writing bash. What would prevent you from pointing the LLMs to separate content which would contain broken examples and code with a number of security issues, because you long…

llms.txt doesn't seem to make it any easier or harder to do that.

...it gives you a version only the LLM will see, as opposed to having to identify the visitor as a scraper or human and determining if they get the good or bad version.

Re: Llms.txt

#130

Post by nkozyra »

If you've been watching logs the past few years, you know that LLM data scrapers care less about robot directives than the scummiest of scraper bots of yore.

Your choices are: 1) give up 2) spend your days trying to detect and block agents and IPs that are known LLMs 3) try to spoil the pot with generated junk or 4) make it easier for them to scrape

1) is the easiest and frankly - not to be nihilistic - the only logical move

Post reply on HN