Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: A Karpathy-style LLM wiki your agents maintain (Markdown and Git)

github.com

Post reply on HN
Print view View original

61–70 of 140 posts

Re: Show HN: A Karpathy-style LLM wiki your agents maintain (Markdown and Git)

#61

Post by 0123456789ABCDE »

armcat wrote: post #17

Any particular reason for BM25? Why not just a table of contents or index structure (json, md, whatever) that is updated automatically and fed in context at query time? I know bag of words is great for speed but even at 1000s of documents, the index can be quite cheap and will maximise precision

do you want to pollute the context with blurbs for docs in disparate topics? cascade filtering, even with naïve bm25, helps reduce the amount of _noise_ that's pushed into the context window. if we reduce the amount of results to consider, further filtering or reranking, with more expensive options, becomes realistic. one could even put a cheaper model in front to further clean the results.

Re: Show HN: A Karpathy-style LLM wiki your agents maintain (Markdown and Git)

#62

Post by zby »

Reviewed: https://zby.github.io/commonplace/agent-memory-systems/revie...

It is a third llm wiki on front page in 24 hours! Obviously it is a hot topic. I have my own horse in that race - so I might not be objective - but I've compiled a wishlist for these system: https://zby.github.io/commonplace/notes/designing-agent-memo...

I wish there was a chance for collaboration - everybody coding their own system seems like a lot of effort duplication.

Re: Show HN: A Karpathy-style LLM wiki your agents maintain (Markdown and Git)

#63

Post by big_man_ting »

portly wrote: post #20

I don't understand the point of automating note taking. It never worked for me to copy paste text into my notes and now you can 100x that? The whole point of taking notes for me is to read a source critically, fit it in my mental model, and then document that. Then sometimes I look it up for the details. But for me the shaping of the mental model is what counts

Totally agree re note taking. We treat our notes way too lightly, just as an attic or a basement leads to hoarding more stuff than you'll ever need.

Most things do not need to end up in your notes, and LLMs add too much noise, one that you likely never personally verify/filter out at all.

JA Westenberg made a good video essay about it a few days ago:

https://youtube.com/watch?v=3E00ZNdFbEk

Re: Show HN: A Karpathy-style LLM wiki your agents maintain (Markdown and Git)

#64

Post by iterateoften »

dataviz1000 wrote: post #25

LLM models and the agents that use them are probabilistic, not deterministic. They accomplish something a percentage of the time, never every time. That means the longer an agent runs on a task, the more likely it will fail the task. Running agents like this will always fail and burn a ton of token cash in the process. One thing that LLM agents are good at is writing their own instructions. The trick is to limit the…

It’s also that agents and ML reach local maximima unless external feedback is given. So your wiki will reach a state and get stuck there.

Re: Show HN: A Karpathy-style LLM wiki your agents maintain (Markdown and Git)

#66

Post by Abby_101 »

The "garbage facts in, garbage briefs out" caveat is the part I'd want stress tested. In my own LLM features the context that decays fastest is what agents wrote without a human glance. Six months in, you have entries that are confidently wrong and the lint pass can't tell which. Does the promotion flow require human review or can agents self promote?

Re: Show HN: A Karpathy-style LLM wiki your agents maintain (Markdown and Git)

#67

Post by dataviz1000 »

iterateoften wrote: post #64
dataviz1000 wrote: post #25

LLM models and the agents that use them are probabilistic, not deterministic. They accomplish something a percentage of the time, never every time. That means the longer an agent runs on a task, the more likely it will fail the task. Running agents like this will always fail and burn a ton of token cash in the process. One thing that LLM agents are good at is writing their own instructions. The trick is to limit the…

It’s also that agents and ML reach local maximima unless external feedback is given. So your wiki will reach a state and get stuck there.

Here is an iteresting thing.

> "The LLM model's attention doesn't distinguish between "instructions I'm writing" and "instructions I'm following" -- they're both just tokens in context."

That means all these SOTA models are very capable of updating their own prompts. Update prompt. Copy entire repository in 1ms into /tmp/*. Run again. Evaluate. Update prompt. Copy entire repository ....

That is recursion, like Karpathy's autoresearch, it requires a deterministic termination condition.

Or have the prompt / agent make 5 copies of itself and solve for 5 different situations to ensure the update didn't introduce any regressions.

> reach local maximima unless external feedback is given

The agents can update themselves with human permission. So the external feedback is another agent and selection bias of a human. It is close to the right idea. I, however, am having huge success with the external feedback being the agent itself. The big difference is that a recursive agent can evaluate performance within confidence interval rather than chaos.

Re: Show HN: A Karpathy-style LLM wiki your agents maintain (Markdown and Git)

#70

Post by rglover »

ting0 wrote: post #47
batoga wrote: post #22

Put AI in your product name, make billion dollars. Put Karpathy in your blog article, get hired by Anthropic as Principal engineer. Milk money as long as fad last. No one is thinking about customer needs, everyone is trying to wash hands in the wave as it last.

Hey man, if it works it works. There's a reason everyone is creating AI tools. We're all buying them. I'm still waiting for someone to make a world-class cli harness that can replace Claude Code but solves the memory and design problem. Web design is still a nightmare with LLMs.

Cline. Works as a CLI and VSCode plugin.
Post reply on HN