It's interesting to see people here and on Blind are more wary? of AI than people in say, Reddit or Youtube comments
Andrej Karpathy: Software in the era of AI [video]
471–480 of 827 posts
Re: Andrej Karpathy: Software in the era of AI [video]
#472when I started coding at the age of 11 in machine code and assembly on the C64, the dream was to create software that creates software. Nowadays it's almost reality, almost because the devil is always in the details. When you're used to write code, writing code is relatively fast. You need this knowledge to debug issues with generated code. However you're now telling AI to fix the bugs in the generated code. I see it…
This is most definitely not toys for rich people. Now perhaps depending on your country it may be considered rich but I would comfortably say that for most of the developed world, the costs for these tools are absolutely attainable, there is a reason ChatGPT has such a large subscriber base. Also the disconnect for me here is I think back on the cost of electronics, prices for the level of compute have generally gone…
[flagged]
Re: Andrej Karpathy: Software in the era of AI [video]
#473Re: Andrej Karpathy: Software in the era of AI [video]
#474Great talk, thanks for putting it online so quickly. I liked the idea of making the generation / verification loop go brrr, and one way to do this is to make verification not just a human task, but a machine task, where possible. Yes, I am talking about formal verification, of course! That also goes nicely together with "keeping the AI on a tight leash". It seems to clash though with "English is the new programming l…
> Use informal English on the surface, while some of it is interpreted as a formal expression, put to work, and then reflected back in English? I think that is possible, if you have a formal language and logic that is flexible enough, and close enough to informal English.
That sounds like a paradox.
Formal verification can prove that constraints are held. English cannot. mapping between them necessarily requires disambiguation. How would you construct such a disambiguation algorithm which must, by its nature, be deterministic?
Re: Andrej Karpathy: Software in the era of AI [video]
#475Earlier quoted context omitted.
I think this is the moment you're referring to? https://youtu.be/LCEmiRjPEtQ?si=QWkimLapX6oIqAjI&t=236 > maybe you've seen a lot of GitHub code is not just like code anymore there's a bunch of like English interspersed with code and so I think kind of there's a growing category of new kind of code so not only is it a new programming paradigm it's also remarkable to me that it's in our native language of English and s…
> That's a lot of people to talk to in a day more or less, since the talk happened. Were they all there and you too, or you all had a watch party or something? hehe, I wish. The topics in the talk are not new. They have been explored and pondered up for quite a while now. As for the outcome of the donut experiment, I don't know. You tell me. Apply it repeatedly at a big scale and see if you should alter the initial o…
> The topics in the talk are not new.
Sure, but your initial dismissal ("95% X, 5% Y") is literally about this talk no? And when you say 'it's "in one ear, out the other" with most of his audience' that's based on some previous experience, rather than the talk itself? I guess I got confused what applied to what event.
> As for the outcome of the donut experiment, I don't know. You tell me. Apply it repeatedly at a big scale and see if you should alter the initial offer for best outcomes (as relative as "best" might be).
Maybe I'm extra slow today, how does this tie into our conversation so far? Does it have anything to do with determinism or what was the idea behind bringing it up? I'm afraid you're gonna have to spell it out for me, sorry about that :)
Re: Andrej Karpathy: Software in the era of AI [video]
#476Earlier quoted context omitted.
> Does anybody really want to be an assembly line QA reviewer for an automated code factory? Sounds like shit. On the other hand, does anyone really wanna be a code-monkey implementing CRUD applications over and over by following product specifications by "product managers" that barely seem to understand the product they're "managing"? See, we can make bad faith arguments both ways, but what's the point?
Issue is if product people will do the “coding” and you have to fix it is miserable
Even worse would be if we asked the accountants to do the coding, then you'll learn what miserable means.
What was the point again?
Re: Andrej Karpathy: Software in the era of AI [video]
#477Earlier quoted context omitted.
This is why I call all this AI stuff BS. Using a formal language is a feature, not a bug. It is a cornerstone of all human engineering and scientific activity and is the _reason_ why these disciplines are successful. What you are describing (ie. ditching formal and using natural language) is moving humanity back towards magical thinking, shamanism and witchcraft.
> is the _reason_ why these disciplines Would you say that ML isn't a successful discipline? ML is basically balancing between "formal language" (papers/algorithms) and "non-deterministic outcomes" (weights/inference) yet it seems useful in a wide range of applications, even if you don't think about LLMs at all. > towards magical thinking, shamanism and witchcraft. I kind of feel like if you want to make a point abou…
> Would you say that ML isn't a successful discipline?
Not yet it isn't; all I am seeing are tools to replace programmers and artists :-/
Where are the tools to take in 400 recipes and spit out all of them in a formal structure (poster upthread literally gave up on trying to get an LLM to do this). Tools that can replace the 90% of office staff who aren't programmers?
Maybe it's a successful low-code industry right now, it's not really a successful AI industry.
Re: Andrej Karpathy: Software in the era of AI [video]
#47895% terrible expression of the landscape, 5% neatly dumbed down analogies. English is a terrible language for deterministic outcomes in complex/complicated systems. Vibe coders won't understand this until they are 2 years into building the thing. LLMs have their merits and he sometimes aludes to them, although it almost feels accidental. Also, you don't spend years studying computer science to learn the language/synt…
Who said I wanted my outcomes to be deterministic. Why is it that the only way we accept programming is for completely deterministic outcomes, when the reality is that is an implementation detail. I am a real user and I am on a general purpose e-commerce site and my ask is "I want a TV that is not that expensive", then by definition the user request is barely deterministic. User requests are normally like this for an…
Agreed. This e-commerce example is quite a good highlight for LLMs.
Same can't be applied when your supplier needs 300 68 x 34 mm gaskets by the BS10 standard, to give a random, more precise example.
Re: Andrej Karpathy: Software in the era of AI [video]
#479Generally, people behind big revolutionary tech are the worst suited for understanding how it will do "in the wild". Forest for the trees and all that.
Some good nuggets in this talk, specifically his concept that Software 1.0, 2.0 and 3.0 will all persist and all have unique use cases. I definitely agree with that. I disagree with his belief that "anyone can vibe code" mindset - this works to a certain level of fidelity ("make an asteroids clone") but what he overlooks is his ability, honed over many years, to precisely document requirements that will translate directly to code that works in an expected way. If you can't write up a Jira epic that covers all bases of a project, you probably can't vibe code something beyond a toy project (or an obvious clone). LLM code falls apart under its own weight without a solid structure, and I don't think that will ever fundamentally change.
Where we are going next, and a lot of effort is being put behind, is figuring out exactly how to "lengthen the leash" of AI through smart framing, careful context manipulation and structured requests. We obviously can have anyone vibe code a lot further if we abstract different elements into known areas and simply allow LLMs to stitch things together. This would allow much larger projects with a much higher success rate. In other words, I expect an AI Zapier/Yahoo Pipes evolution.
Lastly, I think his concept of only having AI pushing "under 1000 line PRs" that he carefully reviews is more short-sighted. We are very, very early in learning how to control these big stupid brains. Incrementally, we will define sub-tasks that the AI can take over completely without anyone ever having to look at the code, because the output will always be within an accepted and tested range. The revolution will be at the middleware level.
Re: Andrej Karpathy: Software in the era of AI [video]
#480Earlier quoted context omitted.
> is the _reason_ why these disciplines Would you say that ML isn't a successful discipline? ML is basically balancing between "formal language" (papers/algorithms) and "non-deterministic outcomes" (weights/inference) yet it seems useful in a wide range of applications, even if you don't think about LLMs at all. > towards magical thinking, shamanism and witchcraft. I kind of feel like if you want to make a point abou…
> Would you say that ML isn't a successful discipline? Not yet it isn't; all I am seeing are tools to replace programmers and artists :-/ Where are the tools to take in 400 recipes and spit out all of them in a formal structure (poster upthread literally gave up on trying to get an LLM to do this). Tools that can replace the 90% of office staff who aren't programmers? Maybe it's a successful low-code industry right n…
> Not yet it isn't; all I am seeing are tools to replace programmers and artists :-/
You're missing a huge part of the ecosystem, ML is so much more than just "generative AI", which seems to be the extent of your experience so far.
Weather predictions, computer vision, speech recognition, medicine research and more are already improved by various machine learning techniques, and already was before the current LLM/generative AI. Wikipedia has a list of ~50 topics where ML is already being used, in production, today ( https://en.wikipedia.org/wiki/Machine_learning#Applications ) if you're feeling curious about exploring the ecosystem more.