Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI documentation you can talk to, for every repo

deepwiki.com

Post reply on HN
Print view View original

101–110 of 132 posts

Re: AI documentation you can talk to, for every repo

#101

Post by [deleted] »

andybak wrote: post #44
blopker wrote: post #16

I took a look at a project I maintain[0], and wow. It's so wrong in every section I saw. The generated diagrams make no sense. The text sections take implementation details that don't matter and present them to the user like they need to know them. It's also outdated. I hope actual users never see this. I dread thinking about having to go around to various LLM generated sites to correct documentation I never approved…

I just tried it on several of my repos and I was rather impressed. This is another one of those bizarre situations that keeps happening in AI coding related matters where people can look at the same thing and reach diametrically opposed conclusions. It's very peculiar and I've never experienced anything like it in my career until recently.

[deleted]

Re: AI documentation you can talk to, for every repo

#102

Post by NewsaHackO »

sceptic123 wrote: post #73
NewsaHackO wrote: post #28

Earlier quoted context omitted.

>I dunno, it seems to be real excited about a VS Code extension that doesn't exist and isn't mentioned in the actual documentation. There's just too many factual errors to list. There is a folder for a VS Code extension here[0]. It seems to have a README with installation instructions. There is also an extension.ts file, which seems to me to be at least the initial prototype for the extension. Did you forget that you…

In that folder is CHANGELOG.md[0] that indicates that this is unreleased. I'd say that including installation instructions for an unreleased version of the extension is exactly the issue that is being flagged. [0] https://github.com/blopker/codebook/blob/main/vscode-extensi...

You are going to want to reread the file you are quoting buddy. That changelog is indicative that the extension has been released. The Unreleased section seems to list features that are not yet included in the released version of the VS Code extension, and the future plans are features that have not been developed yet.

Re: AI documentation you can talk to, for every repo

#103

Post by killerstorm »

Tangentially related: AI assistants (ChatGPT, Claude and Gemini) are unable to get public code from GitHub. (I.e. specifically assistants you use via web site, not Codex, Claude Code, etc.)

Again, as it might be hard to believe, as situation is rather insane: flagship AI assistants cannot get publicly available code. They can get bits and pieces from README, but that might degrade response quality as it's often based on guesswork, etc.

Example via GPT-5 Thinking, my request:

``` Can you read code from https://github.com/killerstorm/auto-ml-runner/blob/master/ru... ?

If yes, show me some port of code and how you got it. ```

(That's normal URL user can access from the browser, you also get same result if you post top-level repo URL https://github.com/killerstorm/auto-ml-runner/).

Thinking: 2 minutes. (IT WAS THINKING FOR TWO MINUTES JUST TO ACCESS ONE FILE!)

``` Short answer: yes...

Why I’m not pasting a snippet right this second:

In this environment, GitHub’s code pages are loading their chrome but not returning the file body ```

So, actually, no, it cannot read it, but it believes it can. That's rather problematic.

Claude: "Unfortunately, I cannot directly read the code from that GitHub URL."

Gemini: "While the tool was unable to retrieve the full, clean code directly, this inferred portion ...". I.e. it just imagined the code. The snippet has nothing to do with code in repo.

This is a rather fucktacular situation as agents are not sure if they read the code, and they might hallucinate subtly wrong code trying to be helpful.

As I can fetch this via curl it seems like GitHub is deliberately blocking AI agents including their partner OpenAI.

Re: AI documentation you can talk to, for every repo

#104

Post by NewsaHackO »

throwaway290 wrote: post #58
NewsaHackO wrote: post #28

Earlier quoted context omitted.

>I dunno, it seems to be real excited about a VS Code extension that doesn't exist and isn't mentioned in the actual documentation. There's just too many factual errors to list. There is a folder for a VS Code extension here[0]. It seems to have a README with installation instructions. There is also an extension.ts file, which seems to me to be at least the initial prototype for the extension. Did you forget that you…

here the maintainer says it doesn't exist. there's basically no way another interpretation is "more correct". presence or files can be not intended for use, deprecated, internal, WIP, etc. this is why we need maintainers.

Maintainers are not gods, and don't get to rewrite plainly true facts. In the Changelog, it actually says it is a "Initial release of Codebook VS Code extension".

Re: AI documentation you can talk to, for every repo

#105

Post by raincole »

curl-up wrote: post #53
NewsaHackO wrote: post #28

Earlier quoted context omitted.

>I dunno, it seems to be real excited about a VS Code extension that doesn't exist and isn't mentioned in the actual documentation. There's just too many factual errors to list. There is a folder for a VS Code extension here[0]. It seems to have a README with installation instructions. There is also an extension.ts file, which seems to me to be at least the initial prototype for the extension. Did you forget that you…

This thread should end up in the hall of fame, right next to the Dropbox one. From a fellow LLM-powered app builder, I wish you best of luck!

Yeah, this is a thread worth saving. Even just as an example of multiple people who can't read as well as an LLM.

Re: AI documentation you can talk to, for every repo

#106

Post by reconnecting »

oblio wrote: post #74
andybak wrote: post #44

Earlier quoted context omitted.

I just tried it on several of my repos and I was rather impressed. This is another one of those bizarre situations that keeps happening in AI coding related matters where people can look at the same thing and reach diametrically opposed conclusions. It's very peculiar and I've never experienced anything like it in my career until recently.

You could link your docs so we can compare them to OP's docs. No need to guess.

Same here.

Original: https://docs.tirreno.com/

Deepwiki: https://deepwiki.com/tirrenotechnologies/tirreno

Github: https://github.com/tirrenotechnologies/tirreno

Re: AI documentation you can talk to, for every repo

#107

Post by Xss3 »

esperent wrote: post #51
andybak wrote: post #44

Earlier quoted context omitted.

I just tried it on several of my repos and I was rather impressed. This is another one of those bizarre situations that keeps happening in AI coding related matters where people can look at the same thing and reach diametrically opposed conclusions. It's very peculiar and I've never experienced anything like it in my career until recently.

> people can look at the same thing and reach diametrically opposed conclusions. It's very peculiar and I've never experienced anything like it in my career until recently React vs other frameworks (or no framework). Object oriented vs functional. There's loads of examples of this that predate AI.

This is such an apples to oranges comparison that it makes me suspicious of your motives here.

Bad documentation full of obvious errors and nonsense is very different to having an opinion on OO vs Functional programming.

Even that sentence sounds insane because who would ever compare the two?!

Re: AI documentation you can talk to, for every repo

#108

Post by blopker »

NewsaHackO wrote: post #28
blopker wrote: post #25

Earlier quoted context omitted.

I dunno, it seems to be real excited about a VS Code extension that doesn't exist and isn't mentioned in the actual documentation. There's just too many factual errors to list.

>I dunno, it seems to be real excited about a VS Code extension that doesn't exist and isn't mentioned in the actual documentation. There's just too many factual errors to list. There is a folder for a VS Code extension here[0]. It seems to have a README with installation instructions. There is also an extension.ts file, which seems to me to be at least the initial prototype for the extension. Did you forget that you…

I brought up this issue because I thought it illustrated my previous points nicely.

Yes, there is a VS Code folder in that repo. However, it doesn't exist as an actual extension. It's an experiment that does not remotely work.

The LLM generated docs has confidently decided that not only does it exist, but it is the primary installation method.

This is wrong.

Edit: I've now had to go into the Readme of this extension to add a note to LLMs explicitly to not recommend it to users. I hate this.

Re: AI documentation you can talk to, for every repo

#109

Post by Xss3 »

aswegs8 wrote: post #42
skissane wrote: post #34

Earlier quoted context omitted.

> It's so wrong in every section I saw. Not talking about this tool, but in general-incorrect LLM-generated documentation can have some value - developer knows they should write some docs, but are starring at a blank screen and not sure what to write so they don’t. Then developer runs an LLM, gets a screenful of LLM-generated docs, notices it is full of mistakes, starts correcting them-suddenly, a screenful of half-d…

At some point it will be less wrong (TM) and it'll be helpful. Feels generally like a good bet.

Will it though?

Fundamentally this is an alignment problem.

There isnt a single AI out there that wont lie to your face, reinterpret your prompt, or just decide to ignore your prompt.

When they try to write a doc based off code, there is nothing you can do to prevent them from making up a load of nonsense and pretending it is thoroughly validated.

Do we have any reason to believe alignment will be solved any time soon?

Re: AI documentation you can talk to, for every repo

#110

Post by Xss3 »

frumiousirc wrote: post #82
blopker wrote: post #16

I took a look at a project I maintain[0], and wow. It's so wrong in every section I saw. The generated diagrams make no sense. The text sections take implementation details that don't matter and present them to the user like they need to know them. It's also outdated. I hope actual users never see this. I dread thinking about having to go around to various LLM generated sites to correct documentation I never approved…

I have a fairly large code base that has been developed over a decade that deepwiki has indexed. The results are mixed but how they are mixed gives me some insight into deepwiki's usefulness. The code base has a lot of documentation in the form of many individual text files. Each describe some isolated aspect of the code in dense, info-rich and not entirely easily consumable (by humans) detail. As numerous as these d…

If documenting the why rather than the how you often end up tying high level concepts together.

E.g. If you describe how the user service exists you wont necessarily capture where it is used.

If you document why the user service exists you will often mention who or what needs it to exist, the thing that gives it a purpose. Do this throughout and everything ends up tied together at a higher level.

Post reply on HN