Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy: Software in the era of AI [video]

youtube.com

Post reply on HN
Print view View original

461–470 of 827 posts

Re: Andrej Karpathy: Software in the era of AI [video]

#461

Post by whilenot-dev »

pama wrote: post #448
tudorizer wrote: post #423

Earlier quoted context omitted.

I watched the entire talk, quite carefully. He explicitly states how excited he was about his tweet mentioning English. The disclaimer you mention was indeed mentioned, although it's "in one ear, out the other" with most of his audience. If I give you a glazed donut with a brief asterisk about how sugar can cause diabetes will it stop you from eating the donut? You also expect deterministic outcomes when making analo…

Your experience with fabs must be somewhat limited if you think that the state of the art in fabs produces deterministic results. Please lookup (or ask friends) for the typical yields and error mitigation features of modern chips and try to visualize if you think it is possible to have determinism when the density of circuits starts to approach levels that cannot be imspected with regular optical microscopes anymore.…

> Modern chip fabrication is closer to LLM code

As is, I don't quite understand what you're getting at here. Please just think that through and tell us what happens to the yield ratio when the software running on all those photolithography machines wouldn't be deterministic.

Re: Andrej Karpathy: Software in the era of AI [video]

#462

Post by diggan »

tudorizer wrote: post #456
diggan wrote: post #445

Earlier quoted context omitted.

I think this is the moment you're referring to? https://youtu.be/LCEmiRjPEtQ?si=QWkimLapX6oIqAjI&t=236 > maybe you've seen a lot of GitHub code is not just like code anymore there's a bunch of like English interspersed with code and so I think kind of there's a growing category of new kind of code so not only is it a new programming paradigm it's also remarkable to me that it's in our native language of English and s…

> Did you talk with them? Otherwise this is just creating an imaginary argument against some people you just assume they didn't listen. I have, unfortunately. Start-up founders, managers, investors who taunt the need for engineers because "AI can fix it". Don't get me wrong, there are plenty of "stochastic parrot" engineers even without AI, but still, not enough to make blanket statements.

That's a lot of people to talk to in a day more or less, since the talk happened. Were they all there and you too, or you all had a watch party or something?

Still, what's the outcome of our "glazed donut" argument, you got me curious what that would lead to. Did I die of diabetes?

Re: Andrej Karpathy: Software in the era of AI [video]

#463

Post by tudorizer »

pama wrote: post #448
tudorizer wrote: post #423

Earlier quoted context omitted.

I watched the entire talk, quite carefully. He explicitly states how excited he was about his tweet mentioning English. The disclaimer you mention was indeed mentioned, although it's "in one ear, out the other" with most of his audience. If I give you a glazed donut with a brief asterisk about how sugar can cause diabetes will it stop you from eating the donut? You also expect deterministic outcomes when making analo…

Your experience with fabs must be somewhat limited if you think that the state of the art in fabs produces deterministic results. Please lookup (or ask friends) for the typical yields and error mitigation features of modern chips and try to visualize if you think it is possible to have determinism when the density of circuits starts to approach levels that cannot be imspected with regular optical microscopes anymore.…

Fair. No process is 100% efficient and the depths of many topics become ambiguous to the point where margins of error need to be introduced.

Chip fabs are defo far into said depths.

Must we apply this at more shallow levels too?

Re: Andrej Karpathy: Software in the era of AI [video]

#464

Post by diggan »

tudorizer wrote: post #457
diggan wrote: post #450

Earlier quoted context omitted.

> Is that bad? It doesn't have to be. But it does get somewhat boring and trite after a while when you start noticing that certain subjects on HN tend to attract general and/or samey comments about $thing, rather than the submission topic within $thing, and I do think that is against the guidelines. > Please don't post shallow dismissals [...] Avoid generic tangents. Omit internet tropes. [...] The specific part of:…

Might sound like a generic tangent, but it's the conclusion people will leave from the talk.

But is it curious? Is it thoughtful and substantive? Maybe it could have been thoughtful, if it felt like it was in response to what was mentioned in the submission.

Re: Andrej Karpathy: Software in the era of AI [video]

#465

Post by mkleczek »

diggan wrote: post #460
mkleczek wrote: post #453

Earlier quoted context omitted.

This is why I call all this AI stuff BS. Using a formal language is a feature, not a bug. It is a cornerstone of all human engineering and scientific activity and is the _reason_ why these disciplines are successful. What you are describing (ie. ditching formal and using natural language) is moving humanity back towards magical thinking, shamanism and witchcraft.

> is the _reason_ why these disciplines Would you say that ML isn't a successful discipline? ML is basically balancing between "formal language" (papers/algorithms) and "non-deterministic outcomes" (weights/inference) yet it seems useful in a wide range of applications, even if you don't think about LLMs at all. > towards magical thinking, shamanism and witchcraft. I kind of feel like if you want to make a point abou…

> Would you say that ML isn't a successful discipline? ML is basically balancing between "formal language" (papers/algorithms) and "non-deterministic outcomes" (weights/inference) yet it seems useful in a wide range of applications

Usefulness of LLMs has yet to be proven. So far there is more marketing in it than actual, real world results. Especially comparing to civil and mechanical engineering, maths, electrical engineering and plethora of disciplines and methods that bring real world results.

Re: Andrej Karpathy: Software in the era of AI [video]

#466

Post by tudorizer »

diggan wrote: post #445
tudorizer wrote: post #423

Earlier quoted context omitted.

I watched the entire talk, quite carefully. He explicitly states how excited he was about his tweet mentioning English. The disclaimer you mention was indeed mentioned, although it's "in one ear, out the other" with most of his audience. If I give you a glazed donut with a brief asterisk about how sugar can cause diabetes will it stop you from eating the donut? You also expect deterministic outcomes when making analo…

I think this is the moment you're referring to? https://youtu.be/LCEmiRjPEtQ?si=QWkimLapX6oIqAjI&t=236 > maybe you've seen a lot of GitHub code is not just like code anymore there's a bunch of like English interspersed with code and so I think kind of there's a growing category of new kind of code so not only is it a new programming paradigm it's also remarkable to me that it's in our native language of English and s…

> That's a lot of people to talk to in a day more or less, since the talk happened. Were they all there and you too, or you all had a watch party or something?

hehe, I wish.

The topics in the talk are not new. They have been explored and pondered up for quite a while now.

As for the outcome of the donut experiment, I don't know. You tell me. Apply it repeatedly at a big scale and see if you should alter the initial offer for best outcomes (as relative as "best" might be).

Re: Andrej Karpathy: Software in the era of AI [video]

#467

Post by cobertos »

oc1 wrote: post #440
tudorizer wrote: post #405

95% terrible expression of the landscape, 5% neatly dumbed down analogies. English is a terrible language for deterministic outcomes in complex/complicated systems. Vibe coders won't understand this until they are 2 years into building the thing. LLMs have their merits and he sometimes aludes to them, although it almost feels accidental. Also, you don't spend years studying computer science to learn the language/synt…

AI is all about context window. If you figured out the context problem, you will see that all these "AI is bullshit, it doesn't work and can't produce working code" goes away. Same for everything else.

Untrue. I find problems with niche knowledge, heavy math, and/or lack of good online resources to be troublesome for AI. Examples so far I've found of consistent struggle points are shaders, parsers, and streams (in Nodejs at least)

Context window will solve a class of problems, but will not solve all problems with AI.

Re: Andrej Karpathy: Software in the era of AI [video]

#468

Post by NitpickLawyer »

Eggpants wrote: post #436
NitpickLawyer wrote: post #349

Earlier quoted context omitted.

No, there are 3rd party providers that run open-weights models and they are (most likely) not bleeding money. Their prices are kind of similar, and make sense in a napkin-math kind of way (we looked into this when ordering hardware). You are correct that some providers might reduce prices for market capture, but the alternatives are still cheap, and some are close to being competitive in quality to the API providers.

Starts with “No” then follows that up with “most likely”. So in other words you don’t know the real answer but posted anyways.

That most likely is for the case where they made their investment calculations wrong and they won't be able to recoup their hw costs. So I think it's safe to say there may be the outlier 3rd party provider that may lose money in the long run.

But the majority of them are serving at ~ the same price, and that matches to the raw cost + some profit if you actually look into serving those models. And those prices are still cheap.

So yeah, I stand by what I wrote, "most likely" included.

My main answer was "no, ..." because the gp post was only considering the closed providers only (oai, anthropic, goog, etc). But youc an get open-weight models pretty cheap, and they are pretty close to SotA, depending on your needs.

Re: Andrej Karpathy: Software in the era of AI [video]

#469

Post by diggan »

mkleczek wrote: post #465
diggan wrote: post #460

Earlier quoted context omitted.

> is the _reason_ why these disciplines Would you say that ML isn't a successful discipline? ML is basically balancing between "formal language" (papers/algorithms) and "non-deterministic outcomes" (weights/inference) yet it seems useful in a wide range of applications, even if you don't think about LLMs at all. > towards magical thinking, shamanism and witchcraft. I kind of feel like if you want to make a point abou…

> Would you say that ML isn't a successful discipline? ML is basically balancing between "formal language" (papers/algorithms) and "non-deterministic outcomes" (weights/inference) yet it seems useful in a wide range of applications Usefulness of LLMs has yet to be proven. So far there is more marketing in it than actual, real world results. Especially comparing to civil and mechanical engineering, maths, electrical e…

> Usefulness of LLMs has yet to be proven.

What about ML (Machine Learning) as a whole? I kind of wrote ML instead of LLMs just to avoid this specific tangent. Are you feelings about that field the same?

Re: Andrej Karpathy: Software in the era of AI [video]

#470

Post by strangescript »

tudorizer wrote: post #405

95% terrible expression of the landscape, 5% neatly dumbed down analogies. English is a terrible language for deterministic outcomes in complex/complicated systems. Vibe coders won't understand this until they are 2 years into building the thing. LLMs have their merits and he sometimes aludes to them, although it almost feels accidental. Also, you don't spend years studying computer science to learn the language/synt…

Who said I wanted my outcomes to be deterministic. Why is it that the only way we accept programming is for completely deterministic outcomes, when the reality is that is an implementation detail.

I am a real user and I am on a general purpose e-commerce site and my ask is "I want a TV that is not that expensive", then by definition the user request is barely deterministic. User requests are normally like this for any application. High level and vague at best. Then developers spend all their time on edge cases, user QA, in the weeds junk that the User does not care about at all. People dont want to click filters and fill out forms for your app. They want it to be easy.

Post reply on HN