Similar: https://v-it.org/doctrine/
LLM Wiki – example of an "idea file"
41–50 of 109 posts
Re: LLM Wiki – example of an "idea file"
#42The challenge to me seems quality assurance:
I'd rather have it source the original document everytime, then an LLM-generated wiki which I most likely wouldn't have the time to fact-check and review myself.
Re: LLM Wiki – example of an "idea file"
#43Isn’t this just kicking the can down the road?
> but the LLM is rediscovering knowledge from scratch on every question
Unless the wiki stays fully in context now the LLM hast to re-read the wiki instead of re-reading the source files. Also this will introduce and accumulate subtle errors as we start to regurgitate 2nd-order information.
I totally get the idea but I think next gen models with 10M context and/or 1000tps will make this obsolete.
Re: LLM Wiki – example of an "idea file"
#44Sounds like a solution in search for a problem.
Re: LLM Wiki – example of an "idea file"
#45I built a very similar system into my own assistant type project. In all honesty though I've not used it enough to know how well it works out in practice.
Re: LLM Wiki – example of an "idea file"
#46Isn’t this just kicking the can down the road? > but the LLM is rediscovering knowledge from scratch on every question Unless the wiki stays fully in context now the LLM hast to re-read the wiki instead of re-reading the source files. Also this will introduce and accumulate subtle errors as we start to regurgitate 2nd-order information. I totally get the idea but I think next gen models with 10M context and/or 1000tp…
I use a home baked system based on obsidian that is essentially just “obsidian but with structured format on top with schemas” and I deploy this in multiple places with ranges of end users. It is more valuable than you think. The intermediary layer is great for capturing intent of design and determining when implementation diverges from that. There will always be a divergence from the intent of a system and how it actually behaves, and the code itself doesn’t capture that. The intermediate layer is lossy, it’s messy, it goes out of date, but it’s highly effective.
It’s not what this person is describing though. A self referential layer like this that’s entirely autonomous does feel completely valueless - because what is it actually solving? Making itself more efficient? The frontier model providers will be here in 3 weeks doing it better than you on that front. The real value is having a system that supports a human coming in and saying “this is how the system should actually behave”, and having the system be reasonably responsive to that.
I feel like a lot of the exercises like op are interesting but ultimately futile. You will not have the money these frontier providers do, and you do not have remotely the amount of information that they do on how to squeeze the most efficiency in how they work. Best bet is to just stick with the vanilla shit until the firehose of innovation slows down to something manageable, because otherwise the abstraction you build is gonna be completely irrelevant in two months
Re: LLM Wiki – example of an "idea file"
#47this is so validating• https://grimoire-pt5.sprites.app/
what is this?
Re: LLM Wiki – example of an "idea file"
#48I built a tool to exactly help an LLM navigate and search a knowledgebase of md files. It helps a lot to build such a wiki by providing basic content search à la grep but also structured search for frontmatter properties. It also helps to move files around without breaking links and to fix links automatically. It is a CLI tool, mainly meant to be driven by AI tools.
Check it out: https://github.com/ractive/hyalo
Re: LLM Wiki – example of an "idea file"
#49I don't see why this wouldn't just lead to model collapse: https://www.nature.com/articles/s41586-024-07566-y If you've spent any time using LLMs to write documentation you'll see this for yourself: the compounding will just be rewriting valid information with less terse information. I find it concerning Karpathy doesn't see this. But I'm not surprised, because AI maximalists seem to find it really difficult to be...…
Edit for context: the sibling comment from karpathy is gone after being flagged to oblivion. Not sure if he deleted it or if it was just removed based on the number of flags? He had copy-pasted a few snarky responses from Claude and essentially said “Claude has this to say to you:” followed by a super long run on paragraph of slop. ———— Wow, I respect karpathy so much and have learned a ton from him. But WTF is the s…
Eh, he’s just a person. I’m not surprised he posted a rude comment haha, and it got rightfully flagged off the site for being AI slop.
Appreciate the gift link, I’ll give it a read!
Re: LLM Wiki – example of an "idea file"
#50> You never (or rarely) write the wiki yourself — the LLM writes and maintains all of it.
Then what is the point? Why be averse so to use your own brain so much? Why are tech bros like this?