Everyone says documenting code is important yet no one seems to want to take the time to write such documentation. My question is wouldn't it be possible to just let a LLM keep reading your repo and generate the necessary documentation? People already use LLMs to code and are trying to make LLMs work as full developers. If we expect them to work as independent developers in the near future, can't we get them to at least write useful documentation first?
Ask HN: Why can't we have LLMs writing documentation?
1–10 of 10 posts
Re: Ask HN: Why can't we have LLMs writing documentation?
#2I expect the LLM would need more context to make the documentation useful for readers.
Re: Ask HN: Why can't we have LLMs writing documentation?
#3Good documentation is no different than good software architecture. Neither are challenging yet most people in the work force cannot do either effectively, because these things require extensive practice making original organizational decisions. Practice isn’t something that can be wished away by some ineffective hallucination tool.
If you think that lack of experience can be replaced by AI then good luck. Microsoft has very recently lost all faith in AI to do any of this for reason.
Re: Ask HN: Why can't we have LLMs writing documentation?
#4Good documentation is no different than good software architecture. Neither are challenging yet most people in the work force cannot do either effectively, because these things require extensive practice making original organizational decisions. Practice isn’t something that can be wished away by some ineffective hallucination tool. If you think that lack of experience can be replaced by AI then good luck. Microsoft…
What do you mean by Microsoft has lost all faith in AI to do any of this?
Re: Ask HN: Why can't we have LLMs writing documentation?
#5This is the first thing I do when using LLM's to create code. So we can and I do have LLMs writing documentation.
In my domain IIoT, I even created a yaml structure to make this easy. https://unsframework.com (wip)
Re: Ask HN: Why can't we have LLMs writing documentation?
#6Writing documentation after coding should strongly feature why things were done the way they were. How could an LLM do that?
Re: Ask HN: Why can't we have LLMs writing documentation?
#7Writing documentation after coding should strongly feature why things were done the way they were. How could an LLM do that?
Very true. Generated docs will never be able to do that part.
That being said from what I've seen at work (and I think other people experience the same), the bar is not that high. Meaning if the LLM generated documentation was able to explain what is happening without the "why" that would already be a significant improvement.
Re: Ask HN: Why can't we have LLMs writing documentation?
#8Writing documentation after coding should strongly feature why things were done the way they were. How could an LLM do that?
Very true. Generated docs will never be able to do that part. That being said from what I've seen at work (and I think other people experience the same), the bar is not that high. Meaning if the LLM generated documentation was able to explain what is happening without the "why" that would already be a significant improvement.
I’d imagine it wouldn’t be hard to have a kind of hybrid method. The developer comments the code for a reason, and ai can expound on that comment if necessary.
Re: Ask HN: Why can't we have LLMs writing documentation?
#9Good documentation is no different than good software architecture. Neither are challenging yet most people in the work force cannot do either effectively, because these things require extensive practice making original organizational decisions. Practice isn’t something that can be wished away by some ineffective hallucination tool. If you think that lack of experience can be replaced by AI then good luck. Microsoft…
What do you mean by Microsoft has lost all faith in AI to do any of this?
Microsoft has been investing billions into OpenAI and new data centers specifically for AI. Recently that ended. Microsoft has seen no return from the investment and simply concluded AI isn’t worth it and is not the future of software.
Re: Ask HN: Why can't we have LLMs writing documentation?
#10I'd break things down into two broad kind of documentation:
1. Comments on code
2. User-facing manuals
There's no need for an LLM to write the former. Comments are left by a person to explain decisions when they're no longer present. LLMs don't go away. You can always ask the LLM why the code was written that way.
As for users... they famously don't read. UX designers take that into account, and try to make their interfaces self-documenting. A user who can't figure it out from the design in front of them probably won't find the right part of the manual to read, either. They'd rather dive in than read cover-to-cover.
The "for dummies" books seem to sell well. I wonder if you could get an LLM to write those.