Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy – It will take a decade to work through the issues with agents

dwarkesh.com

Post reply on HN
Print view View original

741–750 of 1001 posts

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#741

Post by hvb2 »

roody15 wrote: post #487

One inherent limitation of current LLM/AI is that they are primarily trained on abstracted data that focuses primarily on mimicking our logical and reasoning prefrontal cortex portion of the mind. However most humans make decisions based on activity in the limbic regions of the brain which are essentially emotional and intuition based. So we often will do something before we actually know why we did it, however to ma…

In the same sense that split brain patients [1] will make up a reasonable explanation for what the other half did.

And why witnesses are preferably interviewed very shortly after they witnessed a crime. Before their brains start to 'fill in the blanks'

1: https://en.wikipedia.org/wiki/Split-brain

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#742

Post by vharish »

hyperadvanced wrote: post #579
kllrnohj wrote: post #575

Earlier quoted context omitted.

> Eg. At my company we have 100x more demand than we can get capacity for, and we’re barely getting started. We have a roadmap with 1000x+ the current demand and we’re a relatively small company. OpenAI's revenue is $13bn with 70% of that coming from people just spending $20/mo to talk to ChatGPT. Anthropic is projecting $9bn in revenue in 2025. For nice cold splash of reality, fucking Arizona Iced Tea has $3bn in re…

This is correct, it should burn the retinas of anyone thinking that OAI or Anthropic are in any way worth their multi-billion dollar valuations. I liked AK’s analysis of AI for coding here (it’s overly defensive, lacks style and functionality awareness, is a cargo cultist, and/or just does it wrong a lot) but autocomplete itself is super valuable, as is the ability to generate simple frontend code and let you solve t…

There are many more use cases that aren't fully realised yet. With regards to coding, LLMs have shortcomings. However, there's a lot of work that can be automated. Any work that requires interaction with a computer can eventually be automated to some extent. To what extent is something only time can tell.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#743

Post by amelius »

ziofill wrote: post #545
dreambuffer wrote: post #509

Earlier quoted context omitted.

Photons hit a human eye and then the human came up with language to describe that and then encoded the language into the LLM. The LLM can capture some of this relationship, but the LLM is not sensing actual photons, nor experiencing actual light cone stimulation, nor generating thoughts. Its "world model" is several degrees removed from the real world. So whatever fragment of a model it gains through learning to comp…

I agree with this. A metaphor I like is that the reason why humans say the night sky is beautiful is because they see that it is, whereas an LLM says it because it’s been said enough times in its training data.

Humans evolved to think the night sky is beautiful. That's also training. If humans were zapped by lightning every time they went outside at night, they would not think that a night sky is beautiful.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#744

Post by amelius »

beowulfey wrote: post #726
stouset wrote: post #652

Earlier quoted context omitted.

To play devil’s advocate, you have never seen the night sky. Photoreceptors in your eye have been excited in the presence of photons. Those photoreceptors have relayed this information across a nerve to neurons in your brain which receive this encoded information and splay it out to an array of other neurons. Each cell in this chain can rightfully claim to be a living organism in and of itself. “You” haven’t directly…

while true, that doesnt change the fact that every one of those independent units of transmission are within a single system (being trained on raw inputs), whereas the language model is derived from structured external data from outside the system. it's "skipping ahead" through a few layers of modeling, so to speak.

But where you place the boundaries of a system is subjective.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#745

Post by ecocentrik »

password54321 wrote: post #209
ecocentrik wrote: post #127

Earlier quoted context omitted.

LLMs are close enough to pass the Turing Test. That was a huge milestone. They are capable of abstract reasoning and can perform many tasks very well but they aren't AGI. They can't teach themselves to play chess at the level of a dedicated chess engine or fly an airplane using the same model they use to copypasta a React UI. They can only fool non-proficient humans into believing that they might be capable of doing…

Turing Test was a thought experiment not a real benchmark for intelligence. If you read the paper the idea originated from it is largely philosophical. As for abstract reasoning, if you look at ARC-2 it is barely capable though at least some progress has been made with the ARC-1 benchmark.

I wasn't claiming the Turing Test was a benchmark for intelligence but the ability to fool a human into thinking a machine is intelligent in conversation is still a significant milestone. I should have said "some abstract reasoning". ARC-2 looks promising.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#746

Post by DanielHB »

ekjhgkejhgk wrote: post #134
Imnimo wrote: post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…

Problem is that these models feels like they are 8 and getting more 8's

(maybe 7)

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#747

Post by spuz »

amelius wrote: post #743
ziofill wrote: post #545

Earlier quoted context omitted.

I agree with this. A metaphor I like is that the reason why humans say the night sky is beautiful is because they see that it is, whereas an LLM says it because it’s been said enough times in its training data.

Humans evolved to think the night sky is beautiful. That's also training. If humans were zapped by lightning every time they went outside at night, they would not think that a night sky is beautiful.

Interestingly this is a question I've had for a while. Night brings potentially deadly cold, predators, a drastic limit in vision so why do we find the sunset and night sky beautiful. Why do we stop and watch the sun set - something that happens every day - rather than prepare for the food and warmth we need to survive the night?

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#748

Post by adastra22 »

foogazi wrote: post #583
del82 wrote: post #569

Earlier quoted context omitted.

I mean, I think the reason I would say the night sky is “beautiful” is because the meaning of the word for me is constructed from the experiences I’ve had in which I’ve heard other people use the word. So I’d agree that the night sky is “beautiful”, but not because I somehow have access to a deeper meaning of the word or the sky than an LLM does. As someone who (long ago) studied philosophy of mind and (Chomskian) li…

> I think the reason I would say the night sky is “beautiful” is because the meaning of the word for me is constructed from the experiences I’ve had in which I’ve heard other people use the word. Ok but you don’t look at every night sky or every sunset and say “wow that’s beautiful” There’s a quality to it - not because you heard someone say it but because you experience it

Because words are much lower bandwidth than speech. But if you were “told” about a sunset by means of a Matrix style direct mind uploading of an experience, it would seem just as real and vivid. That’s a quantitative difference in bandwidth, not a qualitative difference in character.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#750

Post by foofoo12 »

jstummbillig wrote: post #727
willyxdjazz wrote: post #715

Maybe I'm being too simplistic, but I think we're mixing two distinct debates. Today we have an extraordinary invention—comparable to the wheel in its time. That invention is: predictive inference over all human knowledge. Period. I don't like calling it "Artificial Intelligence" because it's not intelligence; it's a prediction system that can project responses by illuminating patterns across all human knowledge enca…

> I don't like calling it "Artificial Intelligence" because it's not intelligence A pattern I noticed in a AI[sic] discussions: Handwavily declaring what intelligence is not, while not explaining what is.

> Handwavily declaring what intelligence is not, while not explaining what is.

That goes in the other direction too. Declaring it intelligent without explaining what it is. Or even worse, if any explanations are offered, they are often half truths or exaggerated.

Post reply on HN