Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy: Software in the era of AI [video]

youtube.com

Post reply on HN
Print view View original

771–780 of 827 posts

Re: Andrej Karpathy: Software in the era of AI [video]

#771

Post by OJFord »

OJFord wrote: post #661

I'm not sure about the 1.0/2.0/3.0 classification, but it did lead me to think about LLMs as a programming paradigm: we've had imperative & declarative, procedural & functional languages, maybe we'll come to view deterministic vs. probabilistic (LLMs) similarly. def __main__: You are a calculator. Given an input expression, you compute the result and print it to stdout, exiting 0. Should you be unable to do this, you…

You know, the more I think about it, the more I like this model.

What we have today with ChatGPT and the like (and even IDE integrations and API use) is imperative right, it's like 'answer this question' or 'do this thing for me', it's a function invocation. Whereas the silly calculator program I presented above is (unintentionally) kind of a declarative probabilistic program - it's 'this is the behaviour I want, make it so' or 'I have these constraints and these unknowns, fill in the gaps'.

What if we had something like Prolog, but with the possibility of facts being kind of on-demand at runtime, powered by the LLM driving it?

Re: Andrej Karpathy: Software in the era of AI [video]

#772

Post by blobbers »

pelagicAustral wrote: post #119
blobbers wrote: post #114

Software 3.0 is the code generated by the machine, not the prompts that generated it. The prompts don't even yield the same output; there is randomness. The new software world is the massive amount of code that will be burped out by these agents, and it should quickly dwarf the human output.

I think that if you give the same task to three different developers you'll get three different implementations. It's not a random result if you do get the functionality that was expected, and at that, I do think the prompt plays an important role in offering a view of how the result was achieved.

Interestingly, I was generating some scraping code today. I prompted it fairly generically and it decided to spit out some Selenium code. I reported the stack trace failure, and it gave me a new script with playwright. That failed also and it gave me some suggestions to fix it. I asked it to update the whole script rather than snippets, and it responded with "Hey let's not use either of these and here we'll use the site's API." and proceeded to do that.

Kind of crazy, it basically found 3 different hammers to hit the nail I wanted. The API unfortunately seems to be timeing out (I had to add the timeout=10 to the post u_u)

Re: Andrej Karpathy: Software in the era of AI [video]

#773

Post by standeven »

llflw wrote: post #746
OJFord wrote: post #661

I'm not sure about the 1.0/2.0/3.0 classification, but it did lead me to think about LLMs as a programming paradigm: we've had imperative & declarative, procedural & functional languages, maybe we'll come to view deterministic vs. probabilistic (LLMs) similarly. def __main__: You are a calculator. Given an input expression, you compute the result and print it to stdout, exiting 0. Should you be unable to do this, you…

Why bother using human language to communicate with a computer? You interact with a computer using a programming language—code—which is more precise and effective. Specifically: → In 1.0, you communicate with computers using compiled code. → In 2.0, you communicate with compilers using high-level programming languages. → In 3.0, you interact with LLMs using prompts, which arguably should not be in natural human langu…

Why bother using higher-level programming languages to communicate with a computer? You interact with a computer using assembly - raw bit shifting and memory addresses - which is more precise and effective.

Re: Andrej Karpathy: Software in the era of AI [video]

#775

Post by iLoveOncall »

superfrank wrote: post #428
iLoveOncall wrote: post #197

Earlier quoted context omitted.

> incredibly impressed by the technology behind that demo An LLM generating some HTML?

At a speed that feels completely seamless to navigate through. Yeah, I'm pretty impressed by that.

Read the code that is actually being generated. It's only the content of the page, which itself is loaded progressively.

It takes 2 seconds to generate an extremely basic 300 characters page of content. Again, what is impressive here?

It's not fast, it gives the illusion of being fast.

Re: Andrej Karpathy: Software in the era of AI [video]

#776

Post by fnord77 »

manyaoman wrote: post #699
fnord77 wrote: post #47

Him claiming govts don't use AI or are behind the curve is not accurate. Modern military drones are very much AI agents

Governments obviously lead in military tech, but do you think they have access to better AI (in general) than consumers? Unless they do, I think it's fair to say that governments are behind the curve, since consumers tend to adopt things more quickly.

what consumer AI can send a vehicle a long distance, locate and track things of interest, and then decide to take actions against those things of interest?

Imagine a consumer AI that could go to the grocery store, find your favorite loaf of bread and bring it back.

Re: Andrej Karpathy: Software in the era of AI [video]

#777

Post by wing-_-nuts »

standeven wrote: post #773
llflw wrote: post #746

Earlier quoted context omitted.

Why bother using human language to communicate with a computer? You interact with a computer using a programming language—code—which is more precise and effective. Specifically: → In 1.0, you communicate with computers using compiled code. → In 2.0, you communicate with compilers using high-level programming languages. → In 3.0, you interact with LLMs using prompts, which arguably should not be in natural human langu…

Why bother using higher-level programming languages to communicate with a computer? You interact with a computer using assembly - raw bit shifting and memory addresses - which is more precise and effective.

You can express dang near anything you wish to express in assembly in a higher order programming language because it is designed to allow that level of clarity and specificity. In fact most have compile time checks to stop you if you have not properly specified certain behavior.

The English language is not comparable. It is a language designed to capture all the ambiguity of human thought, and as such is not appropriate for computation.

TLDR: There's a reason why programmers still exist after the dawn of 4GL / 'no code' frameworks. Otherwise we'd all be product managers typing specs into JIRA and getting fully formed applications out the other side.

Re: Andrej Karpathy: Software in the era of AI [video]

#778

Post by dustbunny »

standeven wrote: post #773
llflw wrote: post #746

Earlier quoted context omitted.

Why bother using human language to communicate with a computer? You interact with a computer using a programming language—code—which is more precise and effective. Specifically: → In 1.0, you communicate with computers using compiled code. → In 2.0, you communicate with compilers using high-level programming languages. → In 3.0, you interact with LLMs using prompts, which arguably should not be in natural human langu…

Why bother using higher-level programming languages to communicate with a computer? You interact with a computer using assembly - raw bit shifting and memory addresses - which is more precise and effective.

Using assembly is not really more precise in terms of solving the problem. You can definitely make an argument that using a higher level language is equally if not more precise. Especially since your low level assembly will be limited to which architectures it can run on, you can state that the c++ that generates that assembly is "more precisely defining a calculator program".

Re: Andrej Karpathy: Software in the era of AI [video]

#779

Post by kdrvr »

I honestly like his perspective around vibe coding. I feel like his original tweet has been taken misunderstood by the mainstream. (Proof-of-concepts churned out over the weekend will usually die or be mostly rewritten, anyways.) For programmers dipping their feet into new areas, I believe it can be useful.

Though, I do not see it being useful as a "gateway drug" (as he says) for kids learning to code. I have seen that children can understand langs and base programming concepts, given the right resources and encouragement. If kids in the 80s/early 90s learned BASIC and grew up to become software engineers; then what we have now (Scratch, Python, even Javascript + something like P5) are perfectly adequate to that task. Vibe coding really just teaches kids how to prompt LLMs properly.

Re: Andrej Karpathy: Software in the era of AI [video]

#780

Post by kewldev87 »

softfalcon wrote: post #762
OJFord wrote: post #661

I'm not sure about the 1.0/2.0/3.0 classification, but it did lead me to think about LLMs as a programming paradigm: we've had imperative & declarative, procedural & functional languages, maybe we'll come to view deterministic vs. probabilistic (LLMs) similarly. def __main__: You are a calculator. Given an input expression, you compute the result and print it to stdout, exiting 0. Should you be unable to do this, you…

If this is what it comes to, it would explain the many, many software malfunctions in Star Trek. If everything is an LLM/LRM (or whatever super advanced version they have in the 23rd century) then everything can evolve into weird emergent behaviours. stares at every weird holo-deck episode

[dead]
Post reply on HN