Andrej Karpathy – It will take a decade to work through the issues with agents

I don't agree with information-theoretic models in this context but we come to the same conclusion.

Loss only makes sense if there was a fixed “original” but there is not. The information-theoretic model creates a solvable engineering problem. We just aren't solving the right problem then with LLMs.

I think it is more than that. The path forward with a use theory of meaning is even less clear.

The driving example is actually a great example of the use theory of meaning and not the information-theoretic.

The meaning of “driving” emerges from this lived activity, not from abstract definitions. You don't encode an abstract meaning of driving that is then transmitted on a noisy channel of language.

The meaning of driving emerges from the physical act of driving. If you only ever mount a camera on the headrest and operate the steering wheel and pedals remotely from a distance you still don't "understand" the meaning of "driving".

Whatever data stream you want to come up with, trying to extract the meaning of "driving" from that data stream makes no sense.

Trying to extract the "meaning" of driving from driving language game syntax with language models is just complete nonsense. There is no meaning to be found even if scaled in the limit.

I'm surprised you missed it, given there's several other models in this space:

From NVIDIA: https://www.nvidia.com/en-us/high-performance-computing/eart...

Google: https://deepmind.google/science/weathernext/

And this is different model from Microsoft, this time a collaboration with Cambridge University: https://www.microsoft.com/en-us/research/blog/introducing-au...

The word for creating that statistical map is "learning".

Now, you could argue that gradient descent or genetic algorithms or whatever else we have are "slow learners", I'd agree with that, but the weights and biases in any ML model are most definitely "learned".

I want something far more interactive that leaves me far more in control and forces me to be responsible for the choices.

For the last two months as I've been out of paid work I've been working like mad on my open source project, and fell into the pattern of heavily using Claude Code and some of the results have been amazing but some I have let my judgment and oversight lapse and come back later with a completely "WTF did it do here?" surprise.

That shouldn't be allowed to happen. A responsible SWE culture would demand that these tools engage in a way that encourages heavy oversight review and engagement.

Almost everybody does mandatory code review process these days (they didn't earlier in my career) ... despite its lower velocity... because of lessons learned -- and yet now we're allowing agent coding to produce large quantities of code that doesn't even lend itself to review by the party in charge of producing it.

Simulating that is a long way away - so the only possibility is that brains have some sort of redundancy and we can optimise that away. Though computers are faster than brains so its possible maybe, how much faster? So lets say a neuron does its work in a mS and we can simulate this work in 1uS, ie a thousand times faster - thats still a lot. Can we get to a million times faster? even then its still a lot. Not to mention the power required for this.

Even if we can fit a million neurons in a CPU thats still 90 million CPU's. Only 10% are active say, still 9 million CPU's, a thousand times faster - 9,000 cpu's nearly there but still a while away.