Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM Wiki – example of an "idea file"

gist.github.com

Post reply on HN
Print view View original

61–70 of 109 posts

Re: LLM Wiki – example of an "idea file"

#61

Post by nidnogg »

loveparade wrote: post #60
nidnogg wrote: post #52

I've recently lazied out big time on a company project going down a similar rabbit hole. After having a burnout episode and dealing with sole caregiver woes in the family for the past year, I've had less and less energy to piece together intense, correct thought sequences at work. As such I've taken to delegating substantial parts architecture and discovery to multiagent workflows that always refer back to a wiki-lik…

That has been my experience as well. Most of the value of writing docs or a wiki is not in the final artifacts, it's that the process of writing docs updates your own mental models and knowledge so that you can make better decisions down the road. Even if you can get an LLM to output good artifacts that don't eventually evolve into slop, which is questionable, it's really not that useful, especially not for a persona…

And what happens when the bucket of knowledge gets too big and starts to overflow? I feel as if, by delegating that process of building knowledge too much, I end up accruing knowledge gaps of my own. Funnily enough it mirrors the LLM/agent's performance.

Maybe my recent prompts reflects how badly up to speed I am at a given time? I don't know. A slightly related note - I recently heard the term AI de-skilling, this treads close to it imo.

Re: LLM Wiki – example of an "idea file"

#63

Post by nidnogg »

loveparade wrote: post #60
nidnogg wrote: post #52

I've recently lazied out big time on a company project going down a similar rabbit hole. After having a burnout episode and dealing with sole caregiver woes in the family for the past year, I've had less and less energy to piece together intense, correct thought sequences at work. As such I've taken to delegating substantial parts architecture and discovery to multiagent workflows that always refer back to a wiki-lik…

That has been my experience as well. Most of the value of writing docs or a wiki is not in the final artifacts, it's that the process of writing docs updates your own mental models and knowledge so that you can make better decisions down the road. Even if you can get an LLM to output good artifacts that don't eventually evolve into slop, which is questionable, it's really not that useful, especially not for a persona…

The worst part to me, by far, is having nothing more than a bunch of "smart" markdown files to show as my deliverables for the day. Sometimes this stacks for many days on end. Usually the bigger the knowledge gaps are, the more I procrastinate on real work.

Talk about back to school feelings (!)

Re: LLM Wiki – example of an "idea file"

#64

Post by gbro3n »

I built AS Notes for VS Code (https://www.asnotes.io) with the option for this usage pattern in mind. By augmenting VS Code so it has the tooling we use in personal knowledge management systems, it makes it easy to write, link and update markdown / wikilinked notes manually (with mermaid / LaTeX rendering capability also) - but by using VS Code we have easy access to an Agent harness that we can direct to work on, or use our notes as context. Others have pointed out that context bloat is an issue, but no more so than when you use the copilot harness (or any other) inside a large codebase. I find I get more value from my AI conversations when I persist the outputs in markdown like this.

Re: LLM Wiki – example of an "idea file"

#65

Post by kilroy123 »

loveparade wrote: post #60
nidnogg wrote: post #52

I've recently lazied out big time on a company project going down a similar rabbit hole. After having a burnout episode and dealing with sole caregiver woes in the family for the past year, I've had less and less energy to piece together intense, correct thought sequences at work. As such I've taken to delegating substantial parts architecture and discovery to multiagent workflows that always refer back to a wiki-lik…

That has been my experience as well. Most of the value of writing docs or a wiki is not in the final artifacts, it's that the process of writing docs updates your own mental models and knowledge so that you can make better decisions down the road. Even if you can get an LLM to output good artifacts that don't eventually evolve into slop, which is questionable, it's really not that useful, especially not for a persona…

Makes me think of all these tools that use AI to make fancy flashcards for you to study.

It seems rather silly to me, as _creating_ those flashcards is what helps you learn, with the studying after, cementing that knowledge in your brain.

Re: LLM Wiki – example of an "idea file"

#66

Post by lelanthran »

Imanari wrote: post #43

Isn’t this just kicking the can down the road? > but the LLM is rediscovering knowledge from scratch on every question Unless the wiki stays fully in context now the LLM hast to re-read the wiki instead of re-reading the source files. Also this will introduce and accumulate subtle errors as we start to regurgitate 2nd-order information. I totally get the idea but I think next gen models with 10M context and/or 1000tp…

> I totally get the idea but I think next gen models with 10M context and/or 1000tps will make this obsolete.

We've already got 1m context, 800k context, and they still start "forgetting" things around the 200k - 300k mark.

What use is 10M context if degradation starts at 200k - 300k?

Re: LLM Wiki – example of an "idea file"

#67

Post by kubb »

nidnogg wrote: post #52

I've recently lazied out big time on a company project going down a similar rabbit hole. After having a burnout episode and dealing with sole caregiver woes in the family for the past year, I've had less and less energy to piece together intense, correct thought sequences at work. As such I've taken to delegating substantial parts architecture and discovery to multiagent workflows that always refer back to a wiki-lik…

You’re not doing anything wrong. This isn’t a bulletproof idea. It can work, and this is what a lot of people end up with to manage complexity, but there’s a critical point beyond which things collapse: the agent can’t keep the wiki up to date anymore, the developer can’t grok it anymore.

Re: LLM Wiki – example of an "idea file"

#68

Post by cookiengineer »

This is essentially the hack that Claude Code did with their dedicated sub-agent that summarizes a list of previous messages and maintains different types of memory (temporary, persistent, cross-agent). They also built tools so that agents can talk to each other and use that summary to propagate knowledge to other agents if needed.

Setting aside that their codebase is absolute slopcrap, I think something like this might work nicely if it's built from the ground up.

For my own test environment I'm relying on Golang and its conventions (go build, go test, go fmt, gopls etc) which saves a lot of prompts and tokens down the line. Additionally I think that spec driven development might be more successful but I haven't found out yet what the right amount of details for specifications is, so that semantic anchors can help summarize it better.

Anyways if you're curious, it's made for short agent lifecycles and it kinda works every time most of the time: https://github.com/cookiengineer/exocomp

Still need to implement the summarizing agent and memory parts, it's a little fiddlework to get that right so I'm currently experimenting a lot locally with both ollama and vllm as an inference engine.

Re: LLM Wiki – example of an "idea file"

#69

Post by kaashif »

kubb wrote: post #67
nidnogg wrote: post #52

I've recently lazied out big time on a company project going down a similar rabbit hole. After having a burnout episode and dealing with sole caregiver woes in the family for the past year, I've had less and less energy to piece together intense, correct thought sequences at work. As such I've taken to delegating substantial parts architecture and discovery to multiagent workflows that always refer back to a wiki-lik…

You’re not doing anything wrong. This isn’t a bulletproof idea. It can work, and this is what a lot of people end up with to manage complexity, but there’s a critical point beyond which things collapse: the agent can’t keep the wiki up to date anymore, the developer can’t grok it anymore.

Managing complexity, modularity, separation of concerns, were already critical for ensuring humans could still hold enough of the system in their brains to do something useful.

People who do not understand that will continue to not understand that it also applies to AI right now. Maybe at some point in the future it won't, not sure. But my impression is that systems grow in complexity far past the point where the system is gummed up and no-one can do anything, unless it's actively managed.

If a human can understand 10 units of complexity and their LLM can do 20, then they might just build a system that's 30 complex and not understand the failure modes until it's too late.

Re: LLM Wiki – example of an "idea file"

#70

Post by kaashif »

kilroy123 wrote: post #65
loveparade wrote: post #60

Earlier quoted context omitted.

That has been my experience as well. Most of the value of writing docs or a wiki is not in the final artifacts, it's that the process of writing docs updates your own mental models and knowledge so that you can make better decisions down the road. Even if you can get an LLM to output good artifacts that don't eventually evolve into slop, which is questionable, it's really not that useful, especially not for a persona…

Makes me think of all these tools that use AI to make fancy flashcards for you to study. It seems rather silly to me, as _creating_ those flashcards is what helps you learn, with the studying after, cementing that knowledge in your brain.

The ultimate connecting up of the dots would be brain implants that just give you knowledge with zero effort.

I don't know if I'd ever be comfortable with that, hopefully I'll just be retired or dead when that takes off.

Post reply on HN