Live data from Hacker News

Andrej Karpathy: Software in the era of AI [video]

youtube.com

381–390 of 827 posts

Re: Andrej Karpathy: Software in the era of AI [video]

#381
post #369

It's going to be very interesting to see how things evolve in enterprise IT, especially but not exclusively in regulated industries. As more SaaS services are at least partly vibe coded, how are CIOs going to understand and mitigate risk? As more internal developers are using LLM-powered coding interfaces and become less clear on exactly how their resulting code works, how will that codebase be maintained and increme…

I've found that as LLMs improve, some of their bugs become increasingly slippery - I think of it as the uncanny valley of code. Put another way, when I cause bugs, they are often glaring (more typos, fewer logic mistakes). Plus, as the author it's often straightforward to debug since you already have a deep sense for how the code works - you lived through it. So far, using LLMs has downgraded my productivity. The bug…

I'm curious where that expectation of the flip comes from? Your experience (and mine, frankly) would seem to indicate the opposite, so from whence comes this certainty that one day it'll change entirely and become reliable instead?

I ask (and I'll keep asking) because it really seems like the prevailing narrative is that these tools have improved substantially in a short period of time, and that is seemingly enough justification to claim that they will continue to improve until perfection because...? waves hands vaguely

Nobody ever seems to have any good justification for how we're going to overcome the fundamental issues with this tech, just a belief that comes from SOMEWHERE that it'll happen anyway, and I'm very curious to drill down into that belief and see if it comes from somewhere concrete or it's just something that gets said enough that it "becomes true", regardless of reality.

Re: Andrej Karpathy: Software in the era of AI [video]

#382
post #369

It's going to be very interesting to see how things evolve in enterprise IT, especially but not exclusively in regulated industries. As more SaaS services are at least partly vibe coded, how are CIOs going to understand and mitigate risk? As more internal developers are using LLM-powered coding interfaces and become less clear on exactly how their resulting code works, how will that codebase be maintained and increme…

I've found that as LLMs improve, some of their bugs become increasingly slippery - I think of it as the uncanny valley of code. Put another way, when I cause bugs, they are often glaring (more typos, fewer logic mistakes). Plus, as the author it's often straightforward to debug since you already have a deep sense for how the code works - you lived through it. So far, using LLMs has downgraded my productivity. The bug…

I have been using LLMs for coding a lot during the past year, and I've been writing down my observations by task. I have a lot of tasks where my first entry is thoroughly impressed by how e.g. Claude helped me with a task, and then the second entry is a few days after when I'm thoroughly irritated by chasing down subtle and just _strange_ bugs it introduced along the way. As a rule, these are incredibly hard to find and tedious to debug, because they lurk in the weirdest places, and the root cause is usually some weird confabulation that a human brain would never concoct.

Re: Andrej Karpathy: Software in the era of AI [video]

#383

I think it's interesting to juxtapose traditional coding, neural network weights and prompts because in many areas -- like the example of the self driving module having code being replaced by neural networks tuned to the target dataset representing the domain -- this will be quite useful. However I think it's important to make it clear that given the hardware constraints of many environments the applicability of what…

I kind of say it in words (agreeing with you) but I agree the versioning is a bit confusing analogy because it usually additionally implies some kind of improvement. When I’m just trying to distinguish them as very different software categories.

Andrej, maybe Software 3.0 is not written in spoken language like code or prompts. Software 3.0 is recorded in behavior, a behavior that today's software lacks. That behavior is written and consumed by machine and annotated by human interaction. Skipping to 3.0 is premature, but Software 2.0 is a ramp.

Re: Andrej Karpathy: Software in the era of AI [video]

#385

It's going to be very interesting to see how things evolve in enterprise IT, especially but not exclusively in regulated industries. As more SaaS services are at least partly vibe coded, how are CIOs going to understand and mitigate risk? As more internal developers are using LLM-powered coding interfaces and become less clear on exactly how their resulting code works, how will that codebase be maintained and increme…

It will succeed due to the same reason other sloppy strategies succeed, it has large short term gains and moves risk into the nebulous future. Management LOVES these types of things.

Re: Andrej Karpathy: Software in the era of AI [video]

#386

He's talking about "LLM Utility companies going down and the world becoming dumber" as a sign of humanity's progress. This if anything should be a huge red flag

He lives in a GenAI bubble where everyone is self-congratulating about the usage of LLMs. The reality is that there's not a single critical component anywhere that is built on LLMs. There's absolutely no reliance on models, and ChatGPT being down has absolutely no impact on anything beside teenagers not being able to cheat on their homeworks and LLM wrappers not being able to wrap.

Even an LLM could tell you that that's an unknowable thing, perhaps you should rely on them more.

Re: Andrej Karpathy: Software in the era of AI [video]

#387
post #310

Earlier quoted context omitted.

LLM APIs are pretty darn cheap for most of the developed worlds income levels.

It's cheap now. But if you take into account all the training costs, then at such prices they cannot make a profit in any way. This is called dumping to capture the market.

> But if you take into account all the training costs

Not everyone has to paid that cost, as some companies are releasing weights for download and local use (like Llama) and then some other companies are going even further and releasing open source models+weights (like OLMo). If you're a provider hosting those, I don't think it makes sense to take the training cost into account when planning your own infrastructure.

Although I don't it makes much sense personally, seemingly it makes sense for other companies.

Re: Andrej Karpathy: Software in the era of AI [video]

#388

Earlier quoted context omitted.

I kind of say it in words (agreeing with you) but I agree the versioning is a bit confusing analogy because it usually additionally implies some kind of improvement. When I’m just trying to distinguish them as very different software categories.

Andrej, maybe Software 3.0 is not written in spoken language like code or prompts. Software 3.0 is recorded in behavior, a behavior that today's software lacks. That behavior is written and consumed by machine and annotated by human interaction. Skipping to 3.0 is premature, but Software 2.0 is a ramp.

Would this also be more of a push towards robotics and getting physical AI in our every day lives

Re: Andrej Karpathy: Software in the era of AI [video]

#390
post #363

Earlier quoted context omitted.

Replace with "Water Utility going down and the world becoming less sanitary", etc. Still a red flag?

You're making leap of logic. Before water sanitization technology we had no way of sanitizing water on a large scale. Before LLMs, we could still write software. Arguably we were collectively better at it.

LLMs are general-purpose tools used for great many tasks, most of them not related to writing code.
Post reply on HN