11–20 of 30 posts
To make this easier to discuss and think about, here's a short name for that thing, and a longer definition that probably describes that thing.
.
Oh look, if I take the literal definitions of some words in , there's a thing that fits!
Therefore is solved!
> If you appeared in a puff of smoke before the authors of that paper, just after publication — a few months before half of them cleaved from OpenAI to form Anthropic — and carried with you a laptop linked through time to the big models of 2026, what would their appraisal be ? There’s no doubt in my mind they would say: Wow, we really did it ! This is obviously AGI! I really don't think this would be the reaction. I'…
I think this is on point to the next phase of LLMs or a different neural network architecture that improves on top of them, alongside continual learning.
Adding memory capabilities would mostly benefit local "reasoning" models than online ones as you would be saving tokens to do more tasks, than generating more tokens to use more "skills" or tools. (Unless you pay more for memory capabilities to Anthropic or OpenAI).
It's kind of why you see LLMs being unable to play certain games or doing hundreds of visual tasks very quickly without adding lots of harnesses and tools or giving it a pre-defined map to help it understand the visual setting.
As I said before [0], the easiest way to understand the memory limitations with LLMs is Claude Playing Pokemon with it struggling with basic tasks that a 5 year old can learn continuously.
I'll say AGI is "here" when they are agents in our Slack/chat, emails, and planning tools doing that strategic work that management does. As a dev, "AI" is currently just a handy English language interface to web search.
You could argue LLMs already achieve the same work that management does.
> If you appeared in a puff of smoke before the authors of that paper, just after publication — a few months before half of them cleaved from OpenAI to form Anthropic — and carried with you a laptop linked through time to the big models of 2026, what would their appraisal be ? There’s no doubt in my mind they would say: Wow, we really did it ! This is obviously AGI! I really don't think this would be the reaction. I'…
> If you appeared in a puff of smoke before the authors of that paper, just after publication — a few months before half of them cleaved from OpenAI to form Anthropic — and carried with you a laptop linked through time to the big models of 2026, what would their appraisal be ? There’s no doubt in my mind they would say: Wow, we really did it ! This is obviously AGI! I really don't think this would be the reaction. I'…
with beads, or shoving it in git, or .MD files, it's not clear that we do.
A cat doesn't know its way around a house when it's born, but it also doesn't have to flick through markdown files to find its way around. A child can touch a hot stove once and be neurotic about touching hot things for the rest of their life, without having to read flash cards each morning or think for a few minutes about "what do I know about stoves" every time they're in the kitchen.
> If you appeared in a puff of smoke before the authors of that paper, just after publication — a few months before half of them cleaved from OpenAI to form Anthropic — and carried with you a laptop linked through time to the big models of 2026, what would their appraisal be ? There’s no doubt in my mind they would say: Wow, we really did it ! This is obviously AGI! I really don't think this would be the reaction. I'…
> It's clear that we need a paradigm shift on memory to unlock the next level of performance. I think this is on point to the next phase of LLMs or a different neural network architecture that improves on top of them, alongside continual learning. Adding memory capabilities would mostly benefit local "reasoning" models than online ones as you would be saving tokens to do more tasks, than generating more tokens to use…
> If you appeared in a puff of smoke before the authors of that paper, just after publication — a few months before half of them cleaved from OpenAI to form Anthropic — and carried with you a laptop linked through time to the big models of 2026, what would their appraisal be ? There’s no doubt in my mind they would say: Wow, we really did it ! This is obviously AGI! I really don't think this would be the reaction. I'…