Andrej Karpathy's talk on the future of the industry

I'm simply pointing out the dissonance between what AI companies have been telling us for the past 2+ years, and the results that we would expect to see if their claims were true. I'm not holding my breath that their promises will magically materialize given more time. If they were honest, they would acknowledge that the current tech simply won't get us there because of fundamental issues that still haven't been addressed (e.g. hallucination). But instead it's more profitable to promote software that "thinks", "reasons", and is "close to digital superintelligence".

That menu app is an example of vibe coding, not of increased productivity. He sidestepped the bulk of the work that a human still needs to do to ensure that the software works beyond the happy path scenario he tested, has no security issues, and so on. Clearly, the reason the DevOpsy tasks took him much longer is because he's not an ops person. The solution to this isn't to offload these tasks to AI as well, and ignore all the issues that this could cause on the operational side. It's to either offload them to a competent ops engineer, or become familiar with the tools and processes yourself so that it doesn't take you a week to do them next time.

If you want to use AI to assist you with mindless mechanical tasks, that's fine, I frequently do so too. But don't tell me it's making you more productive when you ignore fundamental processes of software engineering.

"We need to rewrite a lot of software," ok... why?

"AI is the new electricity" Really now... so I should expect a bill every month that always increases and to have my access cut off intermittently when there's a rolling AI power outage?

Interesting times indeed.

I'm sure he's a great guy, but let's face it: He's there speaking those things because he made some popular videos in the past and they needed to renew their PR strategy.

Speaking of PR strategy...

> [...] augments [...]

Iron Man does not buy his metal suit at Costco. He builds it, in a cave, with a bunch of scraps. That's the magic of building things.

> I really don't think people understand how great things could be.

That's not the issue. I think it could be great, but they're being greedy and underestimating their best audience.

Instead, they're focusing on complete beginners, in the hopes that those beginners will generate enough monkey-bashing to train a model that can churn good quality "software 1.0" (which is the real deal, so far irreplaceable). I believe that's a mistake.

That's what their PR material seems to imply.

> Have you tried building things with this stuff?

Yes.

> it's exciting, there's all kinds of new things to build

Yes, and no. I don't want to pay to build, or be dependent on an API or model delivery. It means I want to build the juicy stuff, the models themselves. Generate synthetic data, experiment with different approaches for training, etc. That is way outside the reach of common people (it can't be done now, you need lots of money).

I also don't want to buy a fancy GPU so early. Even the most expensive ones are currently very weak to do anything of real value. I could run some inference or adjust some weights, but I would still be at the mercy of some company delivering me base models.

The prices are all inflated because of the hype, and it seems AI companies are synchronizing this inflation with the hardware companies. I'm betting on this pricing bubble to pop.

Regarding the stuff I can do with already pre-trained models, there's not much to learn. In its core, it's basically the same good old stitching of APIs (I've been doing this for decades). I tried using the AI IDEs, but didn't found much value in them. I'm sure they're great for some scenarios, but it's more of a gimmick to use developers to generate training data than anything else.

Regarding "agents" and stuff, I have zero interest in competing in this "user of model" market. It's so easy to do, that any new idea is flooded with attempts and saturates in mere weeks.

When the opportunity to build something _really_ cool and novel appears, I'll give it a real chance. The race between big tech competitors is killing those opportunities for now, only people hired by big AI companies get to do it, and I'm not one of them.