Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is my toddler a stochastic parrot?

newyorker.com

Post reply on HN
Print view View original

61–70 of 363 posts

Re: Is my toddler a stochastic parrot?

#61

Post by meindnoch »

LASR wrote: post #5

Often the "stochastic parrot" line is used as a reduction on what an LLM truly is. I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Where the industry is headed - multi-modal models. This really I think is the remaining frontier of LLM Human parity. I also have a 15 month old son. It's totally obvio…

That's a pretty bold statement, coming from someone with the subjective experience of consciousness.

Re: Is my toddler a stochastic parrot?

#62

Post by dekhn »

atleastoptimal wrote: post #4
carlossouza wrote: post #3

Great essay; impressive content. The fact that it's very unlikely for any of the current models to create something that even remotely resembles this article tells me we are very far away from AGI.

don't underestimate exponentials

or the power of sigmoids to replace exponentials when the exponential peters out.

Re: Is my toddler a stochastic parrot?

#63

Post by Retr0id »

This is a really beautiful article, and while there are certainly fundamental differences between how a toddler thinks and learns, and how an LLM "thinks", I don't think we should get too comfortable with those differences.

Every time I say to myself "AI is no big deal because it can't do X", some time later someone comes along and makes an AI that does X.

Re: Is my toddler a stochastic parrot?

#64

Post by dale_glass »

jocaal wrote: post #37
function_seven wrote: post #12

Earlier quoted context omitted.

I'm in the same boat. It feels wrong to contemplate that our consciousness might not be a magical independent agent with supernatural powers, but is rather an emergent property of complex-but-deterministic actions and reactions. Like it somehow diminishes us. Reduces us to cogs and levers and such. But I can't imagine how it could be otherwise, though. I'm still baffled by the existence of qualia, phenomenology, etc.…

> emergent property of complex-but-deterministic actions and reactions I think you mean non-deterministic. The last century of physics was dominated by work showing how deterministic systems emerged from non-deterministic foundations. It seems that probability and statistics were the branches of maths behind everything. Who would have thought.

I think "deterministic" is the more correct option.

Yes, of course deep down there's unimaginable numbers of atoms randomly bumping around. But so far everything suggests that biological organisms do their damnedest to abstract that away and operate on a deterministic basis.

Think like how you keep changing -- gaining and losing cells, changing chemical balance, and on the whole it takes a lot of damage, even brain damage, to produce measurable results.

Re: Is my toddler a stochastic parrot?

#66

Post by mo_42 »

LASR wrote: post #5

Often the "stochastic parrot" line is used as a reduction on what an LLM truly is. I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Where the industry is headed - multi-modal models. This really I think is the remaining frontier of LLM Human parity. I also have a 15 month old son. It's totally obvio…

> I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor.

I agree with the first sentence but not with the second one. Consciousness most probably does not arise from just a next-token predictor. At least not from an architecture similar to current LLMs.

Both humans and LLMs basically learn to predict what happens next. However, LLMs only predict when we ask them. In contrast, humans predict something all the time. Even when we don't have any sensory input, our brain plays scenarios. Maybe consciousness arises because the result of our thinking is fed back as input. In that sense, we simulate a world that includes us acting and communicating in that world.

Also noteworthy, the human brain handles a variety of sensory information and it's output is not only language. LLMs are restricted to only language. But to me it seems like it's enough for consciousness if we can give it the self-referential property.

Re: Is my toddler a stochastic parrot?

#67

Post by Terr_ »

TaylorAlexander wrote: post #48
ImHereToVote wrote: post #46

Earlier quoted context omitted.

Except LLM's are built on neural networks. That are based on how neurons work. The first tech that actually copies aspects of us.

sigh Neural networks are not based on how neurons work. They do not copy aspects of us. They call them neural networks because they are sort of conceptually like networks of neurons in the brain but they’re so different as to make false the statement that they are based on neurons.

*brandishes crutches*

"Behold! The Mechanical Leg! The first technology that actually copies aspects of our very selves! Think of what wonders of self-discovery it shall reveal!" :p

P.S.: "My god, it is stronger in compression rather than shear-stresses, how eerily similar to real legs! We're on to something here!"

Re: Is my toddler a stochastic parrot?

#68

Post by User23 »

nix0n wrote: post #53
Probiotic6081 wrote: post #10

Earlier quoted context omitted.

> I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Almost every time I'm on hackernews I end up baffled by software engineers feeling entitled to have an unfunded opinion on scientific disciplines outside of their own field of expertise. I've literally never encountered that level of hubris from any…

> software engineers feeling entitled to have an unfunded opinion on scientific disciplines outside of their own field of expertise There's an XKCD about this behavior[0]. The title is actually "Physicists", but I also have seen it on HN (especially with psychology). [0] https://xkcd.com/793/

Well with psychology it’s more fair. Thanks to the replication crisis we can say with a straight face that psychologists aren’t even experts on psychology. As usual the demonstrable expertise in the field lies with the pragmatic types. For psychology that means salesmen, pickup artists, advertisers, conmen, propagandists, high school queen bees, and so on.

Re: Is my toddler a stochastic parrot?

#69

Post by idle_zealot »

dekhn wrote: post #41
xianshou wrote: post #33

Pour one out for the decline of human exceptionalism! Once you get material-reductionist enough and accept yourself as a pure function of genetics, environment, past experience, and chance, this conclusion becomes inevitable. I also expect it to be the standard within a decade, with AI-human parity of capabilities as the key driver.

We'll see! I came to this conclusion a long time ago but at the same time I do subjectively experience consciousness, which in itself is something a mystery in the material-reductionist philosophy.

I hear this a lot, but is it really a mystery/incompatible with materialism? Is there a reason consciousness couldn't be what it feels like to be a certain type of computation? I don't see why we would need something immaterial or some undiscovered material component to explain it.

Re: Is my toddler a stochastic parrot?

#70

Post by somewhereoutth »

function_seven wrote: post #12
LASR wrote: post #5

Often the "stochastic parrot" line is used as a reduction on what an LLM truly is. I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Where the industry is headed - multi-modal models. This really I think is the remaining frontier of LLM Human parity. I also have a 15 month old son. It's totally obvio…

I'm in the same boat. It feels wrong to contemplate that our consciousness might not be a magical independent agent with supernatural powers, but is rather an emergent property of complex-but-deterministic actions and reactions. Like it somehow diminishes us. Reduces us to cogs and levers and such. But I can't imagine how it could be otherwise, though. I'm still baffled by the existence of qualia, phenomenology, etc.…

Indeed our consciousness is an emergent property from a complex system, however the complexity of that system - the human brain - is almost unfathomably beyond the complexity of anything we might make in silicon now or in the foreseeable future.

For example, it is possible to write down a (natural/whole) number that completely describes the state of an LLM - its connections and weights etc - for example by simply taking the memory space and converting into a number as a very long sequence of 1s and 0s. The number will be very big, but still indescribably smaller than a number that could perfectly describe the state of a human brain - not least as that number is likely to lie on the real line, or beyond, even if it was calculable. See Cantor for a discussion on the various infinities and their respective cardinalities.

Post reply on HN