Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy's talk on the future of the industry

donnamagi.com

Post reply on HN
Print view View original

61–70 of 193 posts

Re: Andrej Karpathy's talk on the future of the industry

#61

Post by alganet »

Henchman21 wrote: post #49
alganet wrote: post #42

> imagine changing it and programming the computer's life > imagine that the inputs for the car are on the bottom, and they're going through the software stack to produce the steering and acceleration > imagine inspecting them, and it's got an autonomy slider > imagine works as like this binary array of a different situation, of like what works and doesn't work -- Software 3.0 is imaginary. All in your head. I'm kidd…

If I’m going to be leaning on my imagination this much I am going to imagine a world where the tech industry considers at great length whether or not something should be built.

Let me be clear about what I think: I have zero fear of an AI apocalypse. I think the fear is part of the scam.

The danger I see is related to psychological effects caused by humans using LLMs on other humans. And I don't think that's a scenario anyone is giving much attention to, and it's not that bad (it's bad, but not world end bad).

I totally think we should all build it. To be trained from scratch on cheap commodity hardware, so that a lot of people can _really_ learn it and quickly be literate on it. The only true way of democratizing it. If it's not that way, it's a scam.

Re: Andrej Karpathy's talk on the future of the industry

#63

Post by autobodie »

arkj wrote: post #54

>Software 2.0 are the weights which program neural networks. >I think it's a fundamental change, is that neural networks became programmable with large libraries... And in my mind, it's worth giving it the designation of a Software 3.0. I think it's a bit early to change your mind here. We love your 2.0, let's wait for some more time till th e dust settles so we can see clearly and up the revision number. In fact I'm…

Objectivity is lacking throughout the entire talk, not only in the thesis. But objectivity isn't very good for building hype.

Re: Andrej Karpathy's talk on the future of the industry

#64

Post by pests »

arkj wrote: post #54

>Software 2.0 are the weights which program neural networks. >I think it's a fundamental change, is that neural networks became programmable with large libraries... And in my mind, it's worth giving it the designation of a Software 3.0. I think it's a bit early to change your mind here. We love your 2.0, let's wait for some more time till th e dust settles so we can see clearly and up the revision number. In fact I'm…

I think to understand how Andrej views 3.0 is hinted at with his later analogy at Tesla. He saw a ton of manually written Software 1.0 C++ replaced by the weights of the NN. What we used to write manually in explicit code is now incorporated into the NN itself, moving the implementation from 1.0 to 3.0.

Re: Andrej Karpathy's talk on the future of the industry

#65

Post by bcrosby95 »

I might be wrong, but it seems like some people are misinterpreting what is being said here.

Software 3.0 isn't about using AI to write code. It's about using AI instead of code.

So not Human -> AI -> Create Code -> Compile Code -> Code Runs -> The Magic Happens. Instead, it's Human -> AI -> The Magic Happens.

Re: Andrej Karpathy's talk on the future of the industry

#66

Post by bix6 »

no_wizard wrote: post #9

A large contention of this essay (which I’m assuming the talk is based on or is transcribed from depending on order) I do think that open source models will eventually catch up to closed source ones, or at least be “good enough” and I also think you can already see how LLMs are augmenting knowledge work. I don’t think it’s the 4th wave of pioneering a new dawn of civilization but it’s clear LLMs will remain useful wh…

Why would open source outpace? Isn’t there way more money in the closed source ones and therefore more incentive to work on them?

Re: Andrej Karpathy's talk on the future of the industry

#67

Post by no_wizard »

tayo42 wrote: post #58
no_wizard wrote: post #25

Earlier quoted context omitted.

This was how early browsers felt too, the open source browser engines were slower at adapting than the ones developed by Netscape and Microsoft, but eventually it all reversed and open source excelled past the closed source software. Another way to put it, is that over time you see this, it usually takes a little while for open source projects to catch up, but once they do they gain traction quite quickly over the cl…

Those were way simpler projects in the beginning when that happened. Like do you think a new browser would catch up today chrome now?

The tech behind LLMs has been open source for a very long time. Look at DeepSeek and LLAMA for example. They aren’t yet as capable as say Gemini but they aren’t “miles behind” either, especially if you know how to tune the models to be purpose built[0].

The time horizons will be different as they always are, but I believe it will happen eventually.

I’d also argue that browsers got complicated pretty fast, long cry from libhtml in a few short years.

[0]: of which I contend most useful applications of this technology will not be the generalized ChatGPT interface but specialized highly tuned models that don’t need the scope of a generalized querying

Post reply on HN