Live data from Hacker News

Optimizing technical docs for LLMs

docs.kapa.ai

1–10 of 27 posts

Re: Optimizing technical docs for LLMs

#5
I love that writing LLM-friendly docs is just... writing good docs. There's a ton of overlap between accessibility work and preparing things to be used by LLMs.

I wonder if an unintended side effect of this AI hype cycle is a huge investment in more accessible applications.

Re: Optimizing technical docs for LLMs

#6

I love that writing LLM-friendly docs is just... writing good docs. There's a ton of overlap between accessibility work and preparing things to be used by LLMs. I wonder if an unintended side effect of this AI hype cycle is a huge investment in more accessible applications.

Exactly! It's very much garbage-in-garbage-out.

One surprising (to me at least) benefit of hooking up an LLM to your docs is that it is actually a really useful way to find gaps in your docs. For example, when an LLM cannot answer a user question, there's a good chance it's because the answer is not documented anywhere.

Re: Optimizing technical docs for LLMs

#7

I love that writing LLM-friendly docs is just... writing good docs. There's a ton of overlap between accessibility work and preparing things to be used by LLMs. I wonder if an unintended side effect of this AI hype cycle is a huge investment in more accessible applications.

My current working hypothesis is that the way to get the best out of an LLM (and any AI which uses them as the human interface layer) is the same way to get the best out of a human — because it's trained on humans interacting with other humans.

If you yell and swear at the chatbot, you'll get the response most similar to how a human would respond to yelling and swearing. I know the stereotype about drill instructors, but does that even work for marines, or is it just an exercise in learning to cope with stress?

Re: Optimizing technical docs for LLMs

#8
I wonder how many of these groups went the opposite direction — creating the structure of those web pages by using an LLM?

I've (obviously, like almost everyone) experimented with creating stuff with ChatGPT, and… hmm. I was going to write "it made web pages like that", but: Clever Hans. I don't know if I might have subconsciously primed it to, because that's also something I like.

Re: Optimizing technical docs for LLMs

#9
I felt most of this was just plain common sense. People read things by headings and sub headings. People look for relevant documentation for product x under product x not y. QA and code samples primes the LLM for what most developers like myself hunt for (a quick answer or a simple code snippet). The forum part got me. Seeing as forums tend to be variable in the quality and quantity of info. If the author(s) suggest forums why not discord servers and gitter chat as well? I know of serveral projects where the real documentation, examples and help is locked up on the discord/gitter channels. Also, in the same vein, why not Github PRs/issues as well? Having the LLM diagnose when an issue was cleared up, migration strats, etc. from github PRs/issues (as I've had to use from time to time) would be great too. Of course, Github/Discord/Gitter would require some kind of filtering to make sure it is data worth ingesting into the LLM but if it can identify it was worth ingesting then perhaps it could also suggest to the documentation team something worth documenting.

Re: Optimizing technical docs for LLMs

#10
post #9

I felt most of this was just plain common sense. People read things by headings and sub headings. People look for relevant documentation for product x under product x not y. QA and code samples primes the LLM for what most developers like myself hunt for (a quick answer or a simple code snippet). The forum part got me. Seeing as forums tend to be variable in the quality and quantity of info. If the author(s) suggest…

A lot of innovation is common sense because oddly enough, common sense is not widely distributed.
Post reply on HN