The quite good blog post mentioned by Karpathy for working with LLMs when building software:
- https://blog.nilenso.com/blog/2025/05/29/ai-assisted-coding/
See also:
Andrej Karpathy: Software in the era of AI [video]
111–120 of 827 posts
Re: Andrej Karpathy: Software in the era of AI [video]
#112It’s fascinating to think about what true GUI for LLM could be like. It immediately makes me think a LLM that can generate a customized GUI for the topic at hand where you can interact with in a non-linear way.
Fun demo of an early idea was posted by Oriol just yesterday :) https://x.com/OriolVinyalsML/status/1935005985070084197
[flagged]
Re: Andrej Karpathy: Software in the era of AI [video]
#113I think it's interesting to juxtapose traditional coding, neural network weights and prompts because in many areas -- like the example of the self driving module having code being replaced by neural networks tuned to the target dataset representing the domain -- this will be quite useful. However I think it's important to make it clear that given the hardware constraints of many environments the applicability of what…
Weights are code being replaced by data; something I've been making heavy use of since the early 00s. After coding for 10 years you start to see the benefits of it and understand where you should use it.
LLMs give us another tool only this time it's far more accessible and powerful.
Re: Andrej Karpathy: Software in the era of AI [video]
#114Software 3.0 is the code generated by the machine, not the prompts that generated it. The prompts don't even yield the same output; there is randomness.
The new software world is the massive amount of code that will be burped out by these agents, and it should quickly dwarf the human output.
Re: Andrej Karpathy: Software in the era of AI [video]
#115Where do these analogies break down? 1. Similar cost structure to electricity, but non-essential utility (currently)? 2. Like an operating system, but with non-determinism? 3. Like programming, but ...? Where does the programming analogy break down?
only in English, and also non-deterministic.
Re: Andrej Karpathy: Software in the era of AI [video]
#116Earlier quoted context omitted.
You don't "solve" autonomous driving as such. There's a long, slow grind of gradually improving things until failures become rare enough.
I wonder at what point all the self-driving code becomes replaceable with a multimodal generalist model with the prompt “drive safely”
This is (in part) what "world models" are about. While some companies like Tesla bring together a fleet of small specialised models, others like CommaAI and Wayve train generalist models.
Re: Andrej Karpathy: Software in the era of AI [video]
#117[dead]
Re: Andrej Karpathy: Software in the era of AI [video]
#118Re: Andrej Karpathy: Software in the era of AI [video]
#119Software 3.0 is the code generated by the machine, not the prompts that generated it. The prompts don't even yield the same output; there is randomness. The new software world is the massive amount of code that will be burped out by these agents, and it should quickly dwarf the human output.
I think that if you give the same task to three different developers you'll get three different implementations. It's not a random result if you do get the functionality that was expected, and at that, I do think the prompt plays an important role in offering a view of how the result was achieved.
Re: Andrej Karpathy: Software in the era of AI [video]
#120The beginning was painful to watch as is the cheering in this comment section.
The 1.0, 2.0, and 3.0 simply aren't making sense. They imply a kind of a succession and replacement and demonstrate a lack of how programming works. It sounds as marketing oriented as "Web 3.0" that has been born inside an echo chamber. And yet halfway through, the need for determinism/validation is now being reinvented.
The analogies make use of cherry picked properties, which could apply to anything.