Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy: Software in the era of AI [video]

youtube.com

Post reply on HN
Print view View original

311–320 of 827 posts

Re: Andrej Karpathy: Software in the era of AI [video]

#311

Post by xphos »

koakuma-chan wrote: post #278
latexr wrote: post #254

Earlier quoted context omitted.

> Or generating 4 versions of PR for the same task so that the human could just pick the best one. That sounds awful. A truly terrible and demotivating way to work and produce anything of real quality. Why are we doing this to ourselves and embracing it? A few years ago, it would have been seen as a joke to say “the future of software development will be to have a million monkey interns banging on one million keyboar…

> That sounds awful. A truly terrible and demotivating way to work and produce anything of real quality This is the right way to work with generative AI, and it already is an extremely common and established practice when working with image generation.

"If the only tool you have is a hammer, you tend to see every problem as a nail."

I think the worlds leaning dangerously into LLMs expecting them to solve every problem under the sun. Sure AI can solve problems but I think that domain 1 they Karpathy shows if it is the body of new knowledge in the world doesn't grow with LLMs and agents maybe generation and selection is the best method for working with domain 2/3 but there is something fundamentally lost in the rapid embrace of these AI tools.

A true challenge question for people is would you give up 10 points of IQ for access to the next gen AI model? I don't ask this in the sense that AI makes people stupid but rather that it frames the value of intelligence is that you have it. Rather than, in how you can look up or generate an answer that may or may not be correct quickly. How we use our tools deeply shapes what we will do in the future. A cautionary tale is US manufacturing of precision tools where we give up on teaching people how to use Lathes, because they could simply run CNC machines instead. Now that industry has an extreme lack of programmers for CNC machines, making it impossible to keep up with other precision instrument producing countries. This of course is a normative statement and has more complex variables but I fear in this dead set charge for AI we will lose sight of what makes programming languages and programming in general valuable

Re: Andrej Karpathy: Software in the era of AI [video]

#312

Post by skydhash »

OtherShrezzing wrote: post #109
jonny_eh wrote: post #82

Earlier quoted context omitted.

An ever-shifting UI sounds unlearnable, and therefore unusable.

A mixed ever-shifting UI can be excellent though. So you've got some tools which consistently interact with UI components, but the UI itself is altered frequently. Take for example world-building video games like Cities Skylines / Sim City or procedural sandboxes like Minecraft. There are 20-30 consistent buttons (tools) in the game's UX, while the rest of the game is an unbounded ever-shifting UI.

The rest of the game is very deterministic where its state is controlled by the buttons. The slight variation is caused by the simulation engine and follows consistent patterns (you can’t have building on fire if there’s no building yet).

Re: Andrej Karpathy: Software in the era of AI [video]

#313

Post by imiric »

klabb3 wrote: post #255
pelagicAustral wrote: post #119

Earlier quoted context omitted.

I think that if you give the same task to three different developers you'll get three different implementations. It's not a random result if you do get the functionality that was expected, and at that, I do think the prompt plays an important role in offering a view of how the result was achieved.

> I think that if you give the same task to three different developers you'll get three different implementations. Yes, but if you want them to be compatible you need to define a protocol and conformance test suite. This is way more work than writing a single implementation. The code is the real spec. Every piece of unintentional non-determinism can be a hazard. That’s why you want the code to be the unit of maintena…

I know! Let's encode the spec into a format that doesn't have the ambiguities of natural language.

Re: Andrej Karpathy: Software in the era of AI [video]

#314

Post by diggan »

bsenftner wrote: post #273
diggan wrote: post #267

Earlier quoted context omitted.

But parent explicitly mentioned: > - yes I've tried giving it prompts so detailed a literal infant could follow along and accomplish the task Which you are saying that might have missed in the end regardless?

I'd like to see the prompt. I suspect that "literal infant" is expected to be a software developer without preamble. The initial sentence to an LLM carries far more relevance, it sets the context stage to understand what follows. If there is no introduction to the subject at hand, the response will be just like anyone fed a wall of words: confusion as to what all this is about.

You and me both :) But I always try to read the comments here with the most charitable interpretation I can come up with.

Re: Andrej Karpathy: Software in the era of AI [video]

#315

Post by beacon294 »

What is this "clerk" library he used at this timestamp to tell him what to do? https://youtu.be/LCEmiRjPEtQ?si=XaC-oOMUxXp0DRU0&t=1991

Gemini found it via screenshot or context: https://clerk.com/

This is what he used for login on MenuGen: https://karpathy.bearblog.dev/vibe-coding-menugen/

Re: Andrej Karpathy: Software in the era of AI [video]

#316

Post by sensanaty »

bsenftner wrote: post #260
diggan wrote: post #195

Earlier quoted context omitted.

You're not the first, nor the last person, to have a seemingly vastly different experience than me and others. So I'm curious, what am I doing differently from what you did/do when you try them out? This is maybe a bit out there, but would you be up for sending me like a screen recording of exactly what you're doing? Or maybe even a video call sharing your screen? I'm not working in the space, have no products or ser…

I'm a greybeard, 45+ years coding, including active in AI during the mid 80's and used it when it applied throughout my entire career. That career being media and animation production backends, where the work is both at the technical and creative edge. I currently have an AI integrated office suite, which has attorneys, professional writers, and political activists using the system. It is office software, word proces…

So AI is simultaneously going to take over everyone's job and do literally everything, including being used as application UI somehow... But you have to talk to it like a moody teenager at their first job lest you get nothing but garbage? I have to put just as much (and usually, more) effort talking to this non-deterministic black box as I would to an intern who joined a week ago to get anything usable out of it?

Yeah, I'd rather just type things out myself, and continue communicating with my fellow humans rather than expending my limited time on this earth appeasing a bullshit generator that's apparently going to make us all jobless Soon™

Re: Andrej Karpathy: Software in the era of AI [video]

#317

Post by jebarker »

elif wrote: post #308
osigurdson wrote: post #296

Earlier quoted context omitted.

I'm not sure that AI code has to be sloppy. I've had some success with hand coding some examples and then asking codex to rigorously adhere to prior conventions. This can end up with very self consistent code. Agree though on the "pick the best PR" workflow. This is pure model training work and you should be compensated for it.

Yep this is what Andrej talks about around 20 minutes into this talk. You have to be extremely verbose in describing all of your requirements. There is seemingly no such thing as too much detail. The second you start being vague, even if it WOULD be clear to a person with common sense, the LLM views that vagueness as a potential aspect of it's own creative liberty.

> the LLM views that vagueness as a potential aspect of it's own creative liberty.

I think that anthropomorphism actually clouds what’s going on here. There’s no creative choice inside an LLM. More description in the prompt just means more constraints on the latent space. You still have no certainty whether the LLM models the particular part of the world you’re constraining it to in the way you hope it does though.

Re: Andrej Karpathy: Software in the era of AI [video]

#318

Post by infecto »

NoOn3 wrote: post #310
infecto wrote: post #283

Earlier quoted context omitted.

LLM APIs are pretty darn cheap for most of the developed worlds income levels.

It's cheap now. But if you take into account all the training costs, then at such prices they cannot make a profit in any way. This is called dumping to capture the market.

No doubt the complete cost of training and to getting where we are today has been significant and I don’t know how the accounting will look years from now but you are just making up the rest based on feelings. We know operationally OpenAI is profitable on purely the runtime side, nobody knows how that will look when accounting for R&D but you have no qualification to say they cannot make a profit in any way.

Re: Andrej Karpathy: Software in the era of AI [video]

#319

Post by eddd-ddde »

bandoti wrote: post #285
abdullin wrote: post #102

Tight feedback loops are the key in working productively with software. I see that in codebases up to 700k lines of code (legacy 30yo 4GL ERP systems). The best part is that AI-driven systems are fine with running even more tight loops than what a sane human would tolerate. Eg. running full linting, testing and E2E/simulation suite after any minor change. Or generating 4 versions of PR for the same task so that the h…

Here’s a few problems I foresee: 1. People get lazy when presented with four choices they had no hand in creating, and they don’t look over the four and just click one, ignoring the others. Why? Because they have ten more of these on the go at once, diminishing their overall focus. 2. Automated tests, end-to-end sim., linting, etc—tools already exist and work at scale. They should be robust and THOROUGHLY reviewed by…

With lazy people the same applies for everything, code they do write, or code they review from peers. The issue is not the tooling, but the hands.

Re: Andrej Karpathy: Software in the era of AI [video]

#320

Post by infecto »

guappa wrote: post #301
infecto wrote: post #283

Earlier quoted context omitted.

LLM APIs are pretty darn cheap for most of the developed worlds income levels.

Yeah, because they're bleeding money like crazy now. You should consider how much it actually costs, not how much they charge. How do people fail to consider this?

>You should consider how much it actually costs, not how much they charge. How do people fail to consider this?

Sure, nobody can predict the long-term economics with certainty but companies like OpenAI already have compelling business fundamentals today. This isn’t some scooter startup praying for margins to appear; it’s a platform with real, scaled revenue and enterprise traction.

But yeah, tell me more about how my $200/mo plan is bankrupting them.

Post reply on HN