Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI documentation you can talk to, for every repo

deepwiki.com

Post reply on HN
Print view View original

111–120 of 132 posts

Re: AI documentation you can talk to, for every repo

#111

Post by aDyslecticCrow »

andybak wrote: post #48
aDyslecticCrow wrote: post #32

This is a nice idea in theory. But you need excellent docs in the firstplace for it to work. And if a human spent painstaking effort writing excellent docs, the least bit of respect i can give them is read it.

> But you need excellent docs in the first place for it to work. Are you sure? I just tried it on projects of mine that have almost zero documentation it did a fairly good job.

Really? How large is your project?

There is a very clear point in codebase size where LLMs tend to falter without very clear written down overview descriptions of the system structure. I have a hard time seeing that this system would be immune to that.

i have encountered LLMs seeminly knowing more about a system than it should because there are many similar in its training set; but that just lead me to be extra sceptical when it pulls up functions that dont exist. (Ive fought LLMs about json libraries quite a bit)

Re: AI documentation you can talk to, for every repo

#112

Post by theletterf »

alansammarone wrote: post #57
theletterf wrote: post #46

You need great pre-existing docs for something like this to work properly. AI must RTFM. https://passo.uno/from-tech-writers-to-ai-context-curators/

It certainly helps, but in my experience you get 60-80% of the benefit just with code (except in legacy or otherwise terrible code, for example with misleading/outdated comments everywhere, bad variable/function names, etc - in that case more like 40%).

Why stopping at 60%?

Re: AI documentation you can talk to, for every repo

#113

Post by ninininino »

blopker wrote: post #108
NewsaHackO wrote: post #28

Earlier quoted context omitted.

>I dunno, it seems to be real excited about a VS Code extension that doesn't exist and isn't mentioned in the actual documentation. There's just too many factual errors to list. There is a folder for a VS Code extension here[0]. It seems to have a README with installation instructions. There is also an extension.ts file, which seems to me to be at least the initial prototype for the extension. Did you forget that you…

I brought up this issue because I thought it illustrated my previous points nicely. Yes, there is a VS Code folder in that repo. However, it doesn't exist as an actual extension. It's an experiment that does not remotely work. The LLM generated docs has confidently decided that not only does it exist, but it is the primary installation method. This is wrong. Edit: I've now had to go into the Readme of this extension…

Is it possible that a random person who discovered your repo from Google search would make the same mistake the LLM did and assume it works and not realize it was an unfinished experiment?

Re: AI documentation you can talk to, for every repo

#114

Post by blibble »

blopker wrote: post #16

I took a look at a project I maintain[0], and wow. It's so wrong in every section I saw. The generated diagrams make no sense. The text sections take implementation details that don't matter and present them to the user like they need to know them. It's also outdated. I hope actual users never see this. I dread thinking about having to go around to various LLM generated sites to correct documentation I never approved…

they will

it's the first result on google for just about anything technical I search for

Re: AI documentation you can talk to, for every repo

#115

Post by rng-concern »

ninininino wrote: post #113
blopker wrote: post #108

Earlier quoted context omitted.

I brought up this issue because I thought it illustrated my previous points nicely. Yes, there is a VS Code folder in that repo. However, it doesn't exist as an actual extension. It's an experiment that does not remotely work. The LLM generated docs has confidently decided that not only does it exist, but it is the primary installation method. This is wrong. Edit: I've now had to go into the Readme of this extension…

Is it possible that a random person who discovered your repo from Google search would make the same mistake the LLM did and assume it works and not realize it was an unfinished experiment?

Yes, and so the value of the persons opinions on the repo is low. Far lower than real documentation written by someone who knows more, that would not have made that mistake.

The value proposition here is that these llm docs would be useful, however in this case they were not.

Re: AI documentation you can talk to, for every repo

#116

Post by NewsaHackO »

rng-concern wrote: post #115
ninininino wrote: post #113

Earlier quoted context omitted.

Is it possible that a random person who discovered your repo from Google search would make the same mistake the LLM did and assume it works and not realize it was an unfinished experiment?

Yes, and so the value of the persons opinions on the repo is low. Far lower than real documentation written by someone who knows more, that would not have made that mistake. The value proposition here is that these llm docs would be useful, however in this case they were not.

>Far lower than real documentation written by someone who knows more, that would not have made that mistake.

But his own documentation did said that there was a VSCode extension, with installation instructions, a README, changelog, etc. From what he said, it doesn't even compile or remotely work. It would be extremely aggravating to attempt to build the project with the maintainer's own documentation, spend an hour trying to figure out what's wrong, and then contact the maintainer for him to say, "oh yeah, that documentation not correct, that doesn't even compile even though I said it did 2 months ago lol." It is extremely ironic that he is so gungho about DeepWiki getting this wrong.

Re: AI documentation you can talk to, for every repo

#117

Post by ninininino »

NewsaHackO wrote: post #116
rng-concern wrote: post #115

Earlier quoted context omitted.

Yes, and so the value of the persons opinions on the repo is low. Far lower than real documentation written by someone who knows more, that would not have made that mistake. The value proposition here is that these llm docs would be useful, however in this case they were not.

>Far lower than real documentation written by someone who knows more, that would not have made that mistake. But his own documentation did said that there was a VSCode extension, with installation instructions, a README, changelog, etc. From what he said, it doesn't even compile or remotely work. It would be extremely aggravating to attempt to build the project with the maintainer's own documentation, spend an hour t…

Yes, this is my point. It seems like the creator was a little bit lazy to create such a full fledged readme.md with so much polish but -entirely neglect to mention the whole thing is broken and unfinished-.

That seems about as annoying as a random wiki mis-explaining your system.

That being said, I am still biased towards empathizing with the library author since contributing to open source should be seen as being a great service already in and of itself, and I'd default to avoiding casting blame at an author for not doing things "perfectly" or whatever when they are already doing volunteer work/sharing code they could just keep private.

Re: AI documentation you can talk to, for every repo

#118

Post by NewsaHackO »

ninininino wrote: post #117
NewsaHackO wrote: post #116

Earlier quoted context omitted.

>Far lower than real documentation written by someone who knows more, that would not have made that mistake. But his own documentation did said that there was a VSCode extension, with installation instructions, a README, changelog, etc. From what he said, it doesn't even compile or remotely work. It would be extremely aggravating to attempt to build the project with the maintainer's own documentation, spend an hour t…

Yes, this is my point. It seems like the creator was a little bit lazy to create such a full fledged readme.md with so much polish but -entirely neglect to mention the whole thing is broken and unfinished-. That seems about as annoying as a random wiki mis-explaining your system. That being said, I am still biased towards empathizing with the library author since contributing to open source should be seen as being a…

>That being said, I am still biased towards empathizing with the library author since contributing to open source should be seen as being a great service already in and of itself, and I'd default to avoiding casting blame at an author for not doing things "perfectly" or whatever when they are already doing volunteer work/sharing code they could just keep private

This is true, and the only reason for this was more so his dismissive view of DeepWiki than a criticism of the project itself or of the author as a programmer. LLMs hallucinate all the time, but there is usually a method to the way they do so. Particularly, for it to just say a repo had a VSCode extension portion with nothing pointing to it would not be typical at all for an LLM like DeepWiki.

Re: AI documentation you can talk to, for every repo

#119

Post by blopker »

ninininino wrote: post #117
NewsaHackO wrote: post #116

Earlier quoted context omitted.

>Far lower than real documentation written by someone who knows more, that would not have made that mistake. But his own documentation did said that there was a VSCode extension, with installation instructions, a README, changelog, etc. From what he said, it doesn't even compile or remotely work. It would be extremely aggravating to attempt to build the project with the maintainer's own documentation, spend an hour t…

Yes, this is my point. It seems like the creator was a little bit lazy to create such a full fledged readme.md with so much polish but -entirely neglect to mention the whole thing is broken and unfinished-. That seems about as annoying as a random wiki mis-explaining your system. That being said, I am still biased towards empathizing with the library author since contributing to open source should be seen as being a…

This.

The WIP code was committed with the expectation that very few people would see it because it was not linked anywhere in the main readme. It's a calculated risk, so that the code wouldn't get out of date with main. The risk changed when their LLM (wrongly) decided to elevate it to users before it was ready.

It's clear DeepWiki is just a sales funnel for Devin, so all of this is being done in bad faith anyway. I don't expect them to care much.

Re: AI documentation you can talk to, for every repo

#120

Post by bulbar »

blopker wrote: post #16

I took a look at a project I maintain[0], and wow. It's so wrong in every section I saw. The generated diagrams make no sense. The text sections take implementation details that don't matter and present them to the user like they need to know them. It's also outdated. I hope actual users never see this. I dread thinking about having to go around to various LLM generated sites to correct documentation I never approved…

Please don't correct the AI documentation. Just let those projects die as they deserve.
Post reply on HN