https://github.com/EvolvingAgentsLabs/llmunix An experiment to explore Kaparthy ideas
how do i install this thing?
Andrej Karpathy: Software in the era of AI [video]
331–340 of 827 posts
Re: Andrej Karpathy: Software in the era of AI [video]
#332Earlier quoted context omitted.
I can recognize images in one look. How about that 400 Line change that touches 7 files?
In my prompt I ask the LLM to write a short summary of how it solved the problem, run multiple instances of LLM concurrently, compare their summaries, and use the output of whichever LLM seems to have interpreted instructions the best, or arrived at the best solution.
And you trust that the summary matches what was actually done? Your experience with the level of LLMs understanding of code changes must significantly differ from mine.
Re: Andrej Karpathy: Software in the era of AI [video]
#333Earlier quoted context omitted.
Here’s a few problems I foresee: 1. People get lazy when presented with four choices they had no hand in creating, and they don’t look over the four and just click one, ignoring the others. Why? Because they have ten more of these on the go at once, diminishing their overall focus. 2. Automated tests, end-to-end sim., linting, etc—tools already exist and work at scale. They should be robust and THOROUGHLY reviewed by…
With lazy people the same applies for everything, code they do write, or code they review from peers. The issue is not the tooling, but the hands.
The more tedious the work is, the less motivation and passion you get for doing it, and the more "lazy" you become.
Laziness does not just come from within, there are situations that promote behaving lazy, and others that don't. Some people are just lazy most of the time, but most people are "lazy" in some scenarios and not in others.
Re: Andrej Karpathy: Software in the era of AI [video]
#334Earlier quoted context omitted.
We'll need to boil a few more lakes before we get to that stage I'm afraid, who needs water when you can have your AI hallucinate some for you after all?
Who needs water when all these hot takes come from sources so dense, they're about to collapse into black holes.
Is me not wanting the UI of my OS to shift with every mouse click a hot take? If me wanting to have the consistent "When I click here, X happens" behavior instead of the "I click here and I'm Feeling Lucky happens" behavior is equal to me being dense, so be it I guess.
Re: Andrej Karpathy: Software in the era of AI [video]
#335Earlier quoted context omitted.
I'm not sure that AI code has to be sloppy. I've had some success with hand coding some examples and then asking codex to rigorously adhere to prior conventions. This can end up with very self consistent code. Agree though on the "pick the best PR" workflow. This is pure model training work and you should be compensated for it.
Yep this is what Andrej talks about around 20 minutes into this talk. You have to be extremely verbose in describing all of your requirements. There is seemingly no such thing as too much detail. The second you start being vague, even if it WOULD be clear to a person with common sense, the LLM views that vagueness as a potential aspect of it's own creative liberty.
> You have to be extremely verbose in describing all of your requirements. There is seemingly no such thing as too much detail
I understand YMMV, but I have yet to find a use case where this takes me less time than writing the code myself.
Re: Andrej Karpathy: Software in the era of AI [video]
#336Tight feedback loops are the key in working productively with software. I see that in codebases up to 700k lines of code (legacy 30yo 4GL ERP systems). The best part is that AI-driven systems are fine with running even more tight loops than what a sane human would tolerate. Eg. running full linting, testing and E2E/simulation suite after any minor change. Or generating 4 versions of PR for the same task so that the h…
> Or generating 4 versions of PR for the same task so that the human could just pick the best one. That sounds awful. A truly terrible and demotivating way to work and produce anything of real quality. Why are we doing this to ourselves and embracing it? A few years ago, it would have been seen as a joke to say “the future of software development will be to have a million monkey interns banging on one million keyboar…
I say this all the time!
Does anybody really want to be an assembly line QA reviewer for an automated code factory? Sounds like shit.
Also I can’t really imagine that in the first place. At my current job, each task is like 95% understanding all the little bits, and then 5% writing the code. If you’re reviewing PRs from a bot all day, you’ll still need to understand all the bits before you accept it. So how much time is that really gonna save?
Re: Andrej Karpathy: Software in the era of AI [video]
#337Earlier quoted context omitted.
In my prompt I ask the LLM to write a short summary of how it solved the problem, run multiple instances of LLM concurrently, compare their summaries, and use the output of whichever LLM seems to have interpreted instructions the best, or arrived at the best solution.
And you trust that the summary matches what was actually done? Your experience with the level of LLMs understanding of code changes must significantly differ from mine.
It matched every time so far.
Re: Andrej Karpathy: Software in the era of AI [video]
#338the fanboying for this dudes opinion is insane.
Re: Andrej Karpathy: Software in the era of AI [video]
#339It's fascinating to see his gears grinding at 22:55 when acknowledging that a human still has to review the thousand lines of LLM-generated code for bugs and security issues if they're "actually trying to get work done". Yet these are the tools that are supposed to make us hyperproductive? This is "Software 3.0"? Give me a break.
Plus coding is the fun bit, reviewing code is the hard and not fun bit, arguing with an overconfident machine sound like it'll be worse even than that. Thankfully I'm going to retire soon.
Agreed. Hell, even reviewing code can be fun and engaging, especially if done in person. But it helps when the other party can actually think, instead of automatically responding with "You're right!", followed by changes that may or may not make things worse.
It's as if software developers secretly hated their jobs and found most tasks a chore, so they hired someone else to poorly do the mechanical tasks for them, while ignoring the tasks that actually matter. That's not software engineering, programming, nor coding. It's some process of producing shitty software for which we need new terminology to describe.
I envy you for retiring. Good luck!
Re: Andrej Karpathy: Software in the era of AI [video]
#340Earlier quoted context omitted.
I think type theory is exactly right for this! Being so similar to programming languages, it can piggy back on the huge amount of training the LLMs have on source code. I am not sure lean in part is the right language, there might be challengers rising (or old incumbents like Agda or Roq can find a boost). But type theory definitely has the most robust formal systems at the moment.
> Being so similar to programming languages I think it is more important to be close to English than to programming languages, because that is the critical part: "As close to a programming language as necessary, as close to English as possible" is the goal, in my opinion, without sacrificing constraints such as simplicity.
Why? Why would the language used to express proof of correctness have anything to do with English?
English was not developed to facilitate exact and formal reasoning. In natural language ambiguity is a feature, in formal languages it is unwanted. Just look at maths. The reasons for all the symbols is not only brevity but also precision. (I dont think the symbolism of mathematics is something to strive for though, we can use sensible names in our languages, but the structure will need to be formal and specialised to the domain.)
I think there could be meaningful work done to render the statements of the results automatically into (a restricted subset of) English for ease of human verification that the results proven are actually the results one wanted. I know there has been work in this direction. This might be viable. But I think the actual language of expressing results and proofs would have to be specialised for precision. And there I think type theory has the upper hand.