What Carmack is doing is right. More people need to get away from training their models just with words. AI need the physicality.
John Carmack talk at Upper Bound 2025
51–60 of 387 posts
Re: John Carmack talk at Upper Bound 2025
#52It's always a treat to watch a Carmack lecture or read anything he writes, and his notes here are no exception. He writes as an engineer, for engineers and documents all his thought processes and misteps in the exact detailed yet concise way you'd want a colleague to who was handing off some work. One question I would have about the research direction is the emphasis on realtime. If I understand correctly he's doing…
From the notes:
"A reality check for people that think full embodied AGI is right around the corner is to ask your dancing humanoid robot to pick up a joystick and learn how to play an obscure video game."
Re: John Carmack talk at Upper Bound 2025
#53I feel top level AI creation is beyond his skill set. He’s a AAA software engineer but the prerequisites to build out cutting edge AI require deep formal math that is beyond his education and years at this point. Nothing to stop him playing around with AI models though.
I think you overestimate the level of math required in AI and at the same time I think you underestimate the math skills of John. AI runs on GPUs, Quake 2 engine was one of the first to optimized for GPUs (OpenGL). I'm pretty excited to see him in this domain. I think he'll focus on some DeepSeek style improvements.
This feels like an understatement. At the time, young me had the impression Carmack came first, then the industry created 3dfx/OpenGL to run his games better. I still have nothing but respect for his skills decades later.
Re: John Carmack talk at Upper Bound 2025
#54A lot of the problems John mentioned (camera jpeg, latency, real time decisions) have been worked on by comma.ai for many years. He could have just used their stack and build on it the general learning parts that comma is not focusing on.
Carmack himself has done a lot of work on end to end latency, during his time at oculus.
Re: John Carmack talk at Upper Bound 2025
#55Earlier quoted context omitted.
It means “programmer I don’t like”. Very versatile insult and vague enough that it’s impossible to defend against.
No, I'm pretty sure it's just a programmer that understands the world in terms of bros.
"in the terms of bros" what does that even mean? I think bro is a term that's used pretty widely, for different things, in different cultures and contexts, I call my brother bro.. I've heard people call their friends bro.. I've heard someone tell a police officer "don't tase me, bro"..
Re: John Carmack talk at Upper Bound 2025
#56Earlier quoted context omitted.
> But it still feels like some key ingredient is missing. Continuous training is the key ingredient. Humans can use existing knowledge and apply it to new scenarios, and so can most AI. But AI cannot permanently remember the result of its actions in the real world, and so its body of knowledge cannot expand. Take a toddler and an oven. The toddler has no concept of what an oven is other than maybe that it smells nice…
This kind of context management is not that hard, even when building LLMs. Especially when you have huge windows like we do today. Look at how ChatGPT can remember things permanently after you said them once using a function call to edit the permanent memory section inside the context. You can also see that in Anthropic's latest post on Claude 4 where it learns to play Pokemon. The only remaining issue here is maybe…
> This kind of context management is not that hard, even when building LLMs.
It is, at least if you wish to be in the meatspace, that's my point. Every day has 86400 seconds during which a human brain constantly adapts to and learns from external input - either directly as it's being awake or indirectly during nighttime cleanup processes.
On top of that, humans have built-in filters for training. Basically, we see some drunkard shouting about the Hollow Earth on the sidewalk... our brain knows that this is a drunkard and that Hollow Earth is absolutely crackpot material, so if it stores anything at all then the fact that there is a drunkard on that street and one might take another route next time, but the drunkard's rambling is forgotten maybe five minutes later.
AI, in contrast, needs to be hand-held by humans during training that annotate, "grade" or weigh information during the compilation of the training dataset, in order that the AI knows what is written in "Mein Kampf" so it can answer questions upon it, but that it also knows (or at least: won't openly regurgitate) that the solution to economic problems isn't to just deport Jews.
And huge context windows aren't the answer either. My wife says me, she would like to have a fruit cake for her next birthday. I'll probably remember that piece of information (or at the very least I'll write it down)... but an AI butler? I'd be really surprised if this is still in its context space in a year, and even if it is, I would not be surprised if it weren't able to recall that fact.
And the final thing is prompts... also not the answer. We've seen it just a few days ago with Grok - someone messed with the system prompt so it randomly interjected "white genocide" claims into completely unrelated conversation [1] despite hopefully being trained on a ... more civilised dataset, and to the contrary, we've also seen Grok reply to Twitter questions in a way that suggest that it is aware its training data is biased.
[1] https://www.reuters.com/business/musks-xai-updates-grok-chat...
Re: John Carmack talk at Upper Bound 2025
#57It's always a treat to watch a Carmack lecture or read anything he writes, and his notes here are no exception. He writes as an engineer, for engineers and documents all his thought processes and misteps in the exact detailed yet concise way you'd want a colleague to who was handing off some work. One question I would have about the research direction is the emphasis on realtime. If I understand correctly he's doing…
> the human brain, of which we don't have a clear understanding of the compute capacity
Neurons have finite (very low) speed of signal transfer, so just by measuring cognitive reaction time we can deduce upper bounds on how many _consecutive_ neuron connections are involved in reception, cognitive processing, and resulting reaction via muscles, even for very complex cognitive processes. And the number is just around 100 consecutive neurons involved one after another. So “the algorithm” could not be _that_ complex in the end (100x matmul+tanh?)
Granted, a lot of parallelism and feedback loops are involved, but overall it gives me (and many others) an impression that when the AGI algorithm is ever found, it’s “mini” version should be able to run on modest 2025 hardware in real time.
Re: John Carmack talk at Upper Bound 2025
#58Earlier quoted context omitted.
> But it still feels like some key ingredient is missing. Continuous training is the key ingredient. Humans can use existing knowledge and apply it to new scenarios, and so can most AI. But AI cannot permanently remember the result of its actions in the real world, and so its body of knowledge cannot expand. Take a toddler and an oven. The toddler has no concept of what an oven is other than maybe that it smells nice…
This kind of context management is not that hard, even when building LLMs. Especially when you have huge windows like we do today. Look at how ChatGPT can remember things permanently after you said them once using a function call to edit the permanent memory section inside the context. You can also see that in Anthropic's latest post on Claude 4 where it learns to play Pokemon. The only remaining issue here is maybe…
Having used vectorDBs before, we're very much not there yet. We don't have any appreciable amounts of context for any reasonable real-life memory. It works if that is the most recent thing you did. Have you talked to an LLM for a day? Stuff is gone before the first hour. You have to use every trick currently in the book, treat context like it's your precious pet
Re: John Carmack talk at Upper Bound 2025
#59Earlier quoted context omitted.
This kind of context management is not that hard, even when building LLMs. Especially when you have huge windows like we do today. Look at how ChatGPT can remember things permanently after you said them once using a function call to edit the permanent memory section inside the context. You can also see that in Anthropic's latest post on Claude 4 where it learns to play Pokemon. The only remaining issue here is maybe…
> This kind of context management is not that hard, even when building LLMs. It is, at least if you wish to be in the meatspace, that's my point. Every day has 86400 seconds during which a human brain constantly adapts to and learns from external input - either directly as it's being awake or indirectly during nighttime cleanup processes. On top of that, humans have built-in filters for training. Basically, we see so…
>Every day has 86400 seconds during which a human brain constantly adapts to and learns from external
That's not even remotely true. At least not in the sense that it is for context in transformer models. Or can you tell me all the visual and auditory inputs you experienced yesterday at the 45232nd second? You only learn permanently and effectively from particular stimulation coupled with surprise. That has a sample rate which is orders of magnitude lower. And it's exactly the kind of sampling that can be replicated with a run-of-the-mill persistent memory system for an LLM. I would wager that you could fit most people's core experiences and memories that they can randomly access at any moment into a 1000 page book - something that fits well into state of the art context windows. For deeper more detailed things you can always fall back to another system.
Re: John Carmack talk at Upper Bound 2025
#60Earlier quoted context omitted.
This kind of context management is not that hard, even when building LLMs. Especially when you have huge windows like we do today. Look at how ChatGPT can remember things permanently after you said them once using a function call to edit the permanent memory section inside the context. You can also see that in Anthropic's latest post on Claude 4 where it learns to play Pokemon. The only remaining issue here is maybe…
Having used vectorDBs before, we're very much not there yet. We don't have any appreciable amounts of context for any reasonable real-life memory. It works if that is the most recent thing you did. Have you talked to an LLM for a day? Stuff is gone before the first hour. You have to use every trick currently in the book, treat context like it's your precious pet
VectorDBs are basically just one excuse of many to make up for a part of the system that is lacking capability due to technical limitations. I'm currently at 50:50 if the problems will be overcome directly by the models or by such support systems. Used to be 80:20 but models have grown in usefulness much faster than all the tools we built around them.