Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM Wiki – example of an "idea file"

gist.github.com

Post reply on HN
Print view View original

51–60 of 109 posts

Re: LLM Wiki – example of an "idea file"

#51

Post by jahala »

devnullbrain wrote: post #20

I don't see why this wouldn't just lead to model collapse: https://www.nature.com/articles/s41586-024-07566-y If you've spent any time using LLMs to write documentation you'll see this for yourself: the compounding will just be rewriting valid information with less terse information. I find it concerning Karpathy doesn't see this. But I'm not surprised, because AI maximalists seem to find it really difficult to be...…

I did a proof of concept for self-updating html files (polyglot bash/html) some weeks ago. It actually works quite well, with simple prompting it seems to not just go in circles (https://github.com/jahala/o-o)

Re: LLM Wiki – example of an "idea file"

#52

Post by nidnogg »

I've recently lazied out big time on a company project going down a similar rabbit hole. After having a burnout episode and dealing with sole caregiver woes in the family for the past year, I've had less and less energy to piece together intense, correct thought sequences at work.

As such I've taken to delegating substantial parts architecture and discovery to multiagent workflows that always refer back to a wiki-like castle of markdown files that I've built over time with them, fronted by Obsidian so I can peep efficiently often enough.

Now I'm certainly doing something wrong, but the gaps are just too many to count. If anything, this creates a weird new type of tech debt. Almost like a persistent brain gap. I miss thinking harder and I think it would get me out of this one for sure. But the wiki workflow is just too addictive to stop.

Re: LLM Wiki – example of an "idea file"

#53

Post by nidnogg »

Imanari wrote: post #43

Isn’t this just kicking the can down the road? > but the LLM is rediscovering knowledge from scratch on every question Unless the wiki stays fully in context now the LLM hast to re-read the wiki instead of re-reading the source files. Also this will introduce and accumulate subtle errors as we start to regurgitate 2nd-order information. I totally get the idea but I think next gen models with 10M context and/or 1000tp…

It is how I feel when I do it. And it certainly shows over time.

Re: LLM Wiki – example of an "idea file"

#54

Post by 0123456789ABCDE »

Imanari wrote: post #43

Isn’t this just kicking the can down the road? > but the LLM is rediscovering knowledge from scratch on every question Unless the wiki stays fully in context now the LLM hast to re-read the wiki instead of re-reading the source files. Also this will introduce and accumulate subtle errors as we start to regurgitate 2nd-order information. I totally get the idea but I think next gen models with 10M context and/or 1000tp…

this solves for now, and this solves for the future.

now you get to condense the findings that interest from a handful of papers

in the future it solves for condensing your interests in a whole field to a handful of papers or less

Re: LLM Wiki – example of an "idea file"

#55

Post by iamflimflam1 »

moralestapia wrote: post #32
sebmellen wrote: post #29

Earlier quoted context omitted.

Edit for context: the sibling comment from karpathy is gone after being flagged to oblivion. Not sure if he deleted it or if it was just removed based on the number of flags? He had copy-pasted a few snarky responses from Claude and essentially said “Claude has this to say to you:” followed by a super long run on paragraph of slop. ———— Wow, I respect karpathy so much and have learned a ton from him. But WTF is the s…

Lol at that. It's weird how some people cover the whole range of putting out some really good stuff and other times the complete opposite. Feels as if they were two different people ... or three, or four.

Emotional state, tiredness, drunkenness, a goods nights sleep… the number of factors that drive our responses is ridiculous.

Re: LLM Wiki – example of an "idea file"

#56

Post by khalic »

Imanari wrote: post #43

Isn’t this just kicking the can down the road? > but the LLM is rediscovering knowledge from scratch on every question Unless the wiki stays fully in context now the LLM hast to re-read the wiki instead of re-reading the source files. Also this will introduce and accumulate subtle errors as we start to regurgitate 2nd-order information. I totally get the idea but I think next gen models with 10M context and/or 1000tp…

The goal isn’t to keep the context every time, it’s to make the memory queryable. Like a data lake but for your ideas and decisions

Re: LLM Wiki – example of an "idea file"

#58

Post by TeMPOraL »

locknitpicker wrote: post #35
nurettin wrote: post #33

He really wants to shine, but how is this different than claude memory or skills? When I encounter something it had difficulty doing, or consistently start off with incorrect assumptions, I solve for it and tell it to remember this. If it goes on a long trial and error loop to accomplish something, once it works I tell it to create a skill.

> He really wants to shine, but how is this different than claude memory or skills? It isn't different. This just tries to reinvent the wheel that all mainstream coding assistants have been providing for over a year. Even ChatGPT rolled out chat memory in their free tier.

The difference obviously being, his way you own the memories; in what's currently deployed, it's the platform that owns them.

Re: LLM Wiki – example of an "idea file"

#59

Post by khalic »

ChrisGreenHeur wrote: post #24
devnullbrain wrote: post #20

I don't see why this wouldn't just lead to model collapse: https://www.nature.com/articles/s41586-024-07566-y If you've spent any time using LLMs to write documentation you'll see this for yourself: the compounding will just be rewriting valid information with less terse information. I find it concerning Karpathy doesn't see this. But I'm not surprised, because AI maximalists seem to find it really difficult to be...…

The article is not on training LLMs. it is about using LLMs to write a wiki for personal use. The article assumes a fully trained LLM such as ChatGPT or Claude already exists to be used.

Don't even try, after vibe coding, people seem to be adopting vibe thinking. "Model Collapse sounds cool, I'm gonna use it without looking up"

Re: LLM Wiki – example of an "idea file"

#60

Post by loveparade »

nidnogg wrote: post #52

I've recently lazied out big time on a company project going down a similar rabbit hole. After having a burnout episode and dealing with sole caregiver woes in the family for the past year, I've had less and less energy to piece together intense, correct thought sequences at work. As such I've taken to delegating substantial parts architecture and discovery to multiagent workflows that always refer back to a wiki-lik…

That has been my experience as well. Most of the value of writing docs or a wiki is not in the final artifacts, it's that the process of writing docs updates your own mental models and knowledge so that you can make better decisions down the road.

Even if you can get an LLM to output good artifacts that don't eventually evolve into slop, which is questionable, it's really not that useful, especially not for a personal wiki.

Post reply on HN