Can I say - For RAG based system using LLM, will it be more effective to structure the source documents in this way?
Yes, definitely!
Optimizing technical docs for LLMs
21–27 of 27 posts
Re: Optimizing technical docs for LLMs
#22Presumably all the structure and section headings that they recommend don't have to be rendered by a browser as visible to humans. The LLMs should be smart enough to understand HTML directives that don't add a lot of unnecessary visual structure.
They don't have to be rendered in the browser, but having all of the structures and section headings help humans to, so I would recommend it. :)
Re: Optimizing technical docs for LLMs
#23Re: Optimizing technical docs for LLMs
#24Isn’t there enough markdown floating around that the LLM can deduce structure from the three levels of headings?
Re: Optimizing technical docs for LLMs
#25Earlier quoted context omitted.
Exactly! It's very much garbage-in-garbage-out. One surprising (to me at least) benefit of hooking up an LLM to your docs is that it is actually a really useful way to find gaps in your docs. For example, when an LLM cannot answer a user question, there's a good chance it's because the answer is not documented anywhere.
What about when it answers with confabulation?
I guess then it becomes a threat to Google's business model.
Re: Optimizing technical docs for LLMs
#26Earlier quoted context omitted.
Just like how (it used to be) that writing good content would rank high in search results. But writing good content is hard. So we'll kid ourselves into writing good content for LLMs. Which is equally hard, but we'll feel like we're getting a leg up over everyone else -- who are all also doing the same thing. It's unfortunate that people are more motivated to write for LLMs that are then used by humans than write for…
Fair point. Although good writing for humans = good writing for LLMs and vice versa. So I'm hopeful this new excitement around AI for docs if anything will just encourage folks to put even more effort into writing great docs.
No, it's not a fair point IMHO. LLMs are arguably the best way we've found to organize and represent textual information.
A document you can hold a meaningful conversation with is a big freaking deal, far superior to any conventional resource when you're trying to learn how to do something new.
Re: Optimizing technical docs for LLMs
#27Presumably all the structure and section headings that they recommend don't have to be rendered by a browser as visible to humans. The LLMs should be smart enough to understand HTML directives that don't add a lot of unnecessary visual structure.
They don't have to be rendered in the browser, but having all of the structures and section headings help humans to, so I would recommend it. :)
It can go too far. Too many section headings and it becomes unreadable, like an undergraduate textbook where you're constantly being distracted by sections and boxes.