Should we not treat LLMs more as a UX feature to interact with a domain specific model (highly contextual), rather than expecting LLMs to provide the intelligence needed for software to act as partner to Humans.
He's selling something.
Andrej Karpathy: Software in the era of AI [video]
161–170 of 827 posts
Re: Andrej Karpathy: Software in the era of AI [video]
#162The slide at 13m claims that LLMs flip the script on technology diffusion and give power to the people. Nothing could be further from the truth.
Large corporations, which have become governments in all but name, are the only ones with the capability to create ML models of any real value. They're the only ones with access to vast amounts of information and resources to train the models. They introduce biases into the models, whether deliberately or not, that reinforces their own agenda. This means that the models will either avoid or promote certain topics. It doesn't take a genius to imagine what will happen when the advertising industry inevitably extends its reach into AI companies, if it hasn't already.
Even open weights models which technically users can self-host are opaque blobs of data that only large companies can create, and have the same biases. Even most truly open source models are useless since no individual has access to the same large datasets that corporations use for training.
So, no, LLMs are the same as any other technology, and actually make governments and corporations even more powerful than anything that came before. The users benefit tangentially, if at all, but will mostly be exploited as usual. Though it's unsurprising that someone deeply embedded in the AI industry would claim otherwise.
Re: Andrej Karpathy: Software in the era of AI [video]
#163Earlier quoted context omitted.
You can't just put things there any time you want - the RFC requires that they go through a registration process. Having said that, this won't work for llms.txt, since in the next version of the proposal they'll be allowed at any level of the path, not only the root.
> You can't just put things there any time you want - the RFC requires that they go through a registration process. Actually, I can for two reasons. First is of course the RFC mentions that items can be registered after the fact, if it's found that a particular well-known suffix is being widely used. But the second is a bit more chaotic - website owners are under no obligation to consult a registry, much like port re…
There was no "large group of very smart people" behind llms.txt. It was just me. And I'm very familiar with the registry, and it doesn't work for this particular case IMO (although other folks are welcome to register it if they feel otherwise, of course).
Re: Andrej Karpathy: Software in the era of AI [video]
#164Earlier quoted context omitted.
You can't just put things there any time you want - the RFC requires that they go through a registration process. Having said that, this won't work for llms.txt, since in the next version of the proposal they'll be allowed at any level of the path, not only the root.
> You can't just put things there any time you want - the RFC requires that they go through a registration process. Excuse me???
From the RFC:
""" A well-known URI is a URI [RFC3986] whose path component begins with the characters "/.well-known/", and whose scheme is "HTTP", "HTTPS", or another scheme that has explicitly been specified to use well- known URIs.
Applications that wish to mint new well-known URIs MUST register them, following the procedures in Section 5.1. """
Re: Andrej Karpathy: Software in the era of AI [video]
#165Tight feedback loops are the key in working productively with software. I see that in codebases up to 700k lines of code (legacy 30yo 4GL ERP systems). The best part is that AI-driven systems are fine with running even more tight loops than what a sane human would tolerate. Eg. running full linting, testing and E2E/simulation suite after any minor change. Or generating 4 versions of PR for the same task so that the h…
I don't think the human is the problem here, but the time it takes to run the full testing suite.
I work in web dev, so people sometimes hook code formatting as a git commit hook or sometimes even upon file save.
The tests are problematic tho. If you work at huge project it's a no go idea at all. If you work at medium then the tests are long enough to block you, but short enough for you not to be able to focus on anything else in the meantime.
Re: Andrej Karpathy: Software in the era of AI [video]
#166The slide at 13m claims that LLMs flip the script on technology diffusion and give power to the people. Nothing could be further from the truth. Large corporations, which have become governments in all but name, are the only ones with the capability to create ML models of any real value. They're the only ones with access to vast amounts of information and resources to train the models. They introduce biases into the…
Well there are cases like OLMo where the process, dataset, and model are all open source. As expected though, it doesn't really compare well to the worst closed model since the dataset can't contain vast amounts of stolen copyrighted data that noticeably improves the model. Llama is not good because Meta knows what they're doing, it's good because it was pretrained on the entirety of Anna's Archive and every pirated ebook they could get their hands on. Same goes for Elevenlabs and pirated audiobooks.
Lack of compute on the Ai2's side also means the context OLMo is trained for is miniscule, the other thing that you need to throw brazillions of dollars at to make model that's maybe useful in the end if you're very lucky. Training needs high GPU interconnect bandwidth, it can't be done in distributed horde in any meaningful way even if people wanted to.
The only ones who have the power now are the Chinese, since they can easily ignore copyright for datasets, patents for compute, and have infinite state funding.
Re: Andrej Karpathy: Software in the era of AI [video]
#167Earlier quoted context omitted.
As "jit" to you means running code, and not "building and executing machine code", maybe you could vibe code this. And enjoy the segfaults.
In a way he's making sense. If the "code" is the prompt, the output of the llm is an intermediate artifact, like the intermediate steps of gcc. So why should we still need gcc? The answer is of course, that we need it because llm's output is shit 90% of the time and debugging assembly or binary directly is even harder, so putting asides the difficulties of training the model, the output would be unusable.
Probably too much snark from me. But the gulf between interpreter and compiler can be decades of work, often discovering new mathematical principles along the way.
The idea that you're fine to risk everything, in the way agentic things allow [0], and want that messing around with raw memory is... A return to DOS' crashes, but with HAL along for the ride.
[0] https://msrc.microsoft.com/update-guide/vulnerability/CVE-20...
Re: Andrej Karpathy: Software in the era of AI [video]
#168There were some cool ideas- I particularly liked "psychology of AI" Overall though I really feel like he is selling the idea that we are going to have to pay large corporations to be able to write code. Which is... terrifying. Also, as a lazy developer who is always trying to make AI do my job for me, it still kind of sucks, and its not clear that it will make my life easier any time soon.
I think it used to be like that before the GNU people made gcc, completely destroying the market of compilers. > Also, as a lazy developer who is always trying to make AI do my job for me, it still kind of sucks, and its not clear that it will make my life easier any time soon. Every time I have to write a simple self contained couple of functions I try… and it gets it completely wrong. It's easier to just write it m…
At least proprietary compilers were software you owned and could be airgapped from any network. You didn't create software by tediously negotiating with compilers running on remote machines controlled by a tech corp that can undercut you on whatever you are trying to build (but of course they will not, it says so in the Agreement, and other tales of the fantastic).
Re: Andrej Karpathy: Software in the era of AI [video]
#169Meanwhile, I asked this morning Claude 4 to write a simple EXIF normalizer. After two rounds of prompting it to double-check its code, I still had to point out that it makes no sense to load the entire image for re-orientating if the EXIF orientation is fine in the first place. Vibe vs reality, and anyone actually working in the space daily can attest how brittle these systems are. Maybe this changes in SWE with more…
> Meanwhile
What do you mean "meanwhile", that's exactly (among other things) the kind of stuff he's talking about? The various frictions and how you need to approach it
> anyone actually working in the space
Is this trying to say that Karpathy doesn't "actually work" with LLMs or in the ML space?
I feel like your whole comment is just reacting to the title of the YouTube video, rather than actually thinking and reflecting on the content itself.
Re: Andrej Karpathy: Software in the era of AI [video]
#170Meanwhile, I asked this morning Claude 4 to write a simple EXIF normalizer. After two rounds of prompting it to double-check its code, I still had to point out that it makes no sense to load the entire image for re-orientating if the EXIF orientation is fine in the first place. Vibe vs reality, and anyone actually working in the space daily can attest how brittle these systems are. Maybe this changes in SWE with more…