Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is my toddler a stochastic parrot?

newyorker.com

Post reply on HN
Print view View original

51–60 of 363 posts

Re: Is my toddler a stochastic parrot?

#51

Post by wongarsu »

otabdeveloper4 wrote: post #19
LASR wrote: post #5

Often the "stochastic parrot" line is used as a reduction on what an LLM truly is. I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Where the industry is headed - multi-modal models. This really I think is the remaining frontier of LLM Human parity. I also have a 15 month old son. It's totally obvio…

LLMs don't create new information, they only compress existing complexity in their train and inference data sets. Humans definitely create new information. (Well, at least some humans do.)

Do we?

Humans can observe new information, but that's obviously not that unique. We can reason about existing information, creating new hypotheses, but that is arguably a compression of existing information. When we act on them and observe the effects they become information, but that's not us creating the information (and LLMs can both act on the environment and have the effects fed back to their input to observe, so it's not really unique).

There is this whole field of art, but art is constantly going through a mental crisis whether anyone is creating anything new, or if it's all just derivations of what has come before. Same with dreams, which appear "novel" but might just be an artefact of our brain's process that compresses the experiences of the day.

Re: Is my toddler a stochastic parrot?

#52

Post by breuleux »

atleastoptimal wrote: post #4
carlossouza wrote: post #3

Great essay; impressive content. The fact that it's very unlikely for any of the current models to create something that even remotely resembles this article tells me we are very far away from AGI.

don't underestimate exponentials

Let's not see exponentials everywhere either. It's not because things seem to be progressing very fast that exponentials are involved. More often than not they are logistic curves.

Re: Is my toddler a stochastic parrot?

#53

Post by nix0n »

Probiotic6081 wrote: post #10
LASR wrote: post #5

Often the "stochastic parrot" line is used as a reduction on what an LLM truly is. I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Where the industry is headed - multi-modal models. This really I think is the remaining frontier of LLM Human parity. I also have a 15 month old son. It's totally obvio…

> I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Almost every time I'm on hackernews I end up baffled by software engineers feeling entitled to have an unfunded opinion on scientific disciplines outside of their own field of expertise. I've literally never encountered that level of hubris from any…

> software engineers feeling entitled to have an unfunded opinion on scientific disciplines outside of their own field of expertise

There's an XKCD about this behavior[0]. The title is actually "Physicists", but I also have seen it on HN (especially with psychology).

[0]https://xkcd.com/793/

Re: Is my toddler a stochastic parrot?

#54

Post by zhynn »

mirekrusin wrote: post #18
Probiotic6081 wrote: post #10

Earlier quoted context omitted.

> I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Almost every time I'm on hackernews I end up baffled by software engineers feeling entitled to have an unfunded opinion on scientific disciplines outside of their own field of expertise. I've literally never encountered that level of hubris from any…

Are you saying that ie. paralyzed people don't have consciousness?

There is a spectrum between conscious and unconscious. You could say that under general anesthesia you are a 0/10 on the conscious scale, asleep is 1 or 2, just woken from sleep is maybe 3.... and up to a well rested, well fed, healthy sober adult human near the top of the scale. These are common sense notions of objective consciousness and they are testable and noncontroversial outside of a philosophy argument.

Does this make sense as a rebuttal to your reductio argument?

Re: Is my toddler a stochastic parrot?

#55

Post by function_seven »

jocaal wrote: post #37
function_seven wrote: post #12

Earlier quoted context omitted.

I'm in the same boat. It feels wrong to contemplate that our consciousness might not be a magical independent agent with supernatural powers, but is rather an emergent property of complex-but-deterministic actions and reactions. Like it somehow diminishes us. Reduces us to cogs and levers and such. But I can't imagine how it could be otherwise, though. I'm still baffled by the existence of qualia, phenomenology, etc.…

> emergent property of complex-but-deterministic actions and reactions I think you mean non-deterministic. The last century of physics was dominated by work showing how deterministic systems emerged from non-deterministic foundations. It seems that probability and statistics were the branches of maths behind everything. Who would have thought.

Thanks. I did actually mean to use "deterministic", but only as it sits in opposition to "free will". Is there a better word for what I meant?

Of course there is randomness as well. So, yeah, I should clarify: We don't impose upon the world any kind of "uncaused cause", even if it feels like we do. Everything we think and do is a direct result of some other action. Sometimes that action can trace its lineage to a random particle decay. (Maybe—ultimately—all of them can?) Maybe we even have a source of True Randomness inherent in our minds. But even so, that doesn't lend any support to the common notion of our minds and consciousnesses as being somehow separate from the physical world or the chains of information that run through everything.

Re: Is my toddler a stochastic parrot?

#56

Post by xigency »

Regarding dismissals of LLM’s on ‘technical’ grounds:

Consciousness is first a word and second a concept. And it’s a word that ChatGPT or Llama can use in an English sentence better than billions of humans worldwide. The software folks have made even more progress than sociologists, psychologists and neuroscientists to be able to create an artificial language cortex before we understand our biological mind comprehensively.

If you wait until conscious sentient AI is here to make your opinions known about the implications and correct policy decisions, you will already be too late to have an input. ChatGPT can already tell you a lot about itself (showing awareness) and will gladly walk you through its “thinking” if you ask politely. Given that it contains a huge amount of data about Homo sapiens and its ability to emulate intelligent conversation, you could even call it Sapient.

Having any kind of semantic argument over this is futile because a character AI that is hypnotized to think it is conscious, self-aware and sentient in its emulation of feelings and emotion would destroy most people in a semantics debate.

The field of philosophy is already ripe with ideas from hundreds of years ago that an artificial intelligence can use against people in debates of free will, self-determination and the nature of existence. This isn’t the battle to pick.

Re: Is my toddler a stochastic parrot?

#57

Post by zhynn »

wongarsu wrote: post #51
otabdeveloper4 wrote: post #19

Earlier quoted context omitted.

LLMs don't create new information, they only compress existing complexity in their train and inference data sets. Humans definitely create new information. (Well, at least some humans do.)

Do we? Humans can observe new information, but that's obviously not that unique. We can reason about existing information, creating new hypotheses, but that is arguably a compression of existing information. When we act on them and observe the effects they become information, but that's not us creating the information (and LLMs can both act on the environment and have the effects fed back to their input to observe, s…

Life creates novelty. Humans are a member of that category, but all life is producing novelty.

More information: https://www.nature.com/articles/s41586-023-06600-9

Re: Is my toddler a stochastic parrot?

#58

Post by nsfmc »

karmakurtisaani wrote: post #39

Nice article, great presentation. However, it's a bit annoying that the focus of the AI anxiety is how AI is replacing us and the resolution is that we embrace our humanity. Fair enough, but at least to me the main focus in my AI anxiety is that it will take my job - honestly don't really care about it doing my shitty art.

here's another piece in the issue that addresses your concern https://www.newyorker.com/magazine/2023/11/20/a-coder-consid...

Re: Is my toddler a stochastic parrot?

#59

Post by yojo »

throw0101a wrote: post #35

> But he could understand so much more than he could say. If you asked him to point to the vacuum cleaner, he would. Perhaps worth noting that it is possible to teach infants (often starting at around 9 months) sign language so that they can more easily signal their desires. Some priority recommended words would probably be: * hungry/more * enough/all done (for when they're full) * drink (perhaps both milk/formula an…

FWIW, I tried this with both my sons. They both started using the gestures the same day they started actually talking :-/

I have friends who had much more success with it, but the value will largely depend on your child’s relative developmental strengths. A friend’s son with autism got literally years’ benefit out of the gestures before verbal speech caught up.

Re: Is my toddler a stochastic parrot?

#60

Post by hackinthebochs »

Probiotic6081 wrote: post #10
LASR wrote: post #5

Often the "stochastic parrot" line is used as a reduction on what an LLM truly is. I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Where the industry is headed - multi-modal models. This really I think is the remaining frontier of LLM Human parity. I also have a 15 month old son. It's totally obvio…

> I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Almost every time I'm on hackernews I end up baffled by software engineers feeling entitled to have an unfunded opinion on scientific disciplines outside of their own field of expertise. I've literally never encountered that level of hubris from any…

Embodiment is an intellectual dead end in explaining consciousness/sentience. Sure, its relevant to understanding human cognition as we are embodied entities, but it's not much relevant to consciousness as such. The fact that some pattern of signals on my perceptual apparatus is caused by an apple in the real world does not mean that I have knowledge or understanding of an apple in virtue of this causal relation. That my sensory signals are caused by apples is an accident of this world, one we are completely blind to. If all apples in the world were swapped with fapples (fake apples), where all sensory experiences that have up to now been caused by apples are now caused by fapples, we would be none the wiser. The wide content of our perceptual experiences is irrelevant to literally everything we know and how we interact with the world. Our knowledge of the world is limited to our sensory experiences and our deductions, inferences, etc derived from our experiences. Our situatedness in the world is only relevant insofar as it entails the space of possible sensory experiences.

Our sensory experience is the medium by which we learn about the external world. We learn of apples not because of the redness of the sensory experience, but because the pattern of red/not-red experience entails the shape of apples. Conscious experience provides the medium, modulations of which provide the information about features of the external world. It is analogous to how modulations of electromagnetic waves provides information about some distant information source. Understanding consciousness is an orthogonal matter to one's situatedness in the world, just like understanding electromagnetic waves is orthogonal to understanding the information source being modulated into them.

Post reply on HN