Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

ranpara.net

Post reply on HN
Print view View original

91–100 of 204 posts

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#91

Post by amalcon »

estearum wrote: post #55
vitamark wrote: post #38

Earlier quoted context omitted.

Well, language is not really required for thinking and conscious experiences, a huge chunk of people don't even have an internal monologue, and some (that'd be me) don't have any "byproducts" of thought process. Though there are recent experiments showing that unconscious brain retains language processing abilities [1], so the two might be as well independent systems [1]: https://www.nature.com/articles/s41586-026-10…

IMO it's far more likely that people don't agree on what it means to "have an internal monologue" than that people actually bifurcate that dramatically on cognition. Surely there are some people with an actual difference from baseline, but I don't think it's the near-equal split that seems to appear when you just ask people about internal monologue This is in contrast with aphantasia (complete lack of mental imagery)…

I don't think a bifurcation of cognition is necessary to explain this. You get both people who claim to always think with words (to the point where they don't understand how it's possible to think without one), and people who claim never to outside of linguistic tasks like writing (to the point where they need "internal monologue" explained to them). In my experience, if you get down to it -- most people will have experienced cases of the thought preceding the word (you can skip the word, or not, but either way it demonstrates a thought without a word) and of the word preceding the thought (people who claim not to have an internal monologue will usually still do this sometimes when writing).

It's just that subvocalization is one powerful technique for organizing thoughts, but not the only one. If you had a swimmer who only ever learned the crawl stroke (freestyle), they might wonder how someone can swim without it. The difference, of course, is that the breaststroke is a demonstrable physical act whereas cognitive techniques are not. The monologuers are just more practiced at subvocalization-based reasoning, and the non-monologuers are more practiced at other types.

It's not that hard to exert limited control over which technique you're using -- and I think there are multiple nonverbal techniques. I'd encourage people to explore the space a bit. Subvocalization-first approaches are very good for linear progressions, whereas subvocalization-last (or never) approaches are usually better for big-picture reasoning.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#92

Post by sesteel »

I am pretty sure human level intelligence is emergent from language use. The more complex the language the higher the intelligence since language is itself a tool used to not only to communicate, but also classify the human experience. Without language, we are fairly primitive relying on instincts and more primitive communication. Feral children are an example of what a base state might be.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#93

Post by SpicyLemonZest »

kibwen wrote: post #60
jacobr1 wrote: post #20

LLMs as clearly doing more than just modeling words. In order to predict word placement, they need to build some kind of model, their latent space, of the types of things they are able to predict. Not really full world models yet - but they have decent "blog space" or "github project" models. And you can see this with multimodality, or non-text modality modals such as images and audio. They map from their latent spac…

Except that the Chinese Room shows that the existence of a mapping from input to output, however emergently it might have been devised, is not alone sufficient to demonstrate understanding.

"Understanding" is your word, it doesn't appear in the source article nor the comment you're responding to. The Chinese Room argument does not attempt to show that a mapping from input to output can't implement a latent space, or that it can't implement complex models of what the language is describing. If a person can express those things explicitly in the output, or if you have to do them in order to correctly respond how a person would, then the room by definition has those capabilities.

(What's the point of the argument if it doesn't tell us anything about the capabilities or internals of an AI? I'm not sure.)

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#94

Post by palmotea »

abound wrote: post #88
palmotea wrote: post #75

Earlier quoted context omitted.

> Similarly, I'm reading "Supernormal Stimuli" now, and there's a whole section on infants raised by various animals (chimps, wolves, chickens, leopards, etc, etc). When they are found and brought back to human societies, basically none of them develop full language and integrate fully. Infants raised by chickens? Chickens!? That doesn't sound real.

Reading it again, the book is referencing this case: https://www.theguardian.com/world/2004/jul/11/fiji.jennyfors...

So not "raised by chickens," but rather abused and isolated in the presence of chickens. That's more plausible.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#95

Post by GuB-42 »

fasteddie31003 wrote: post #59

Maybe consciousness just does not exist. Could it be that humans invented it to feel superior to animals, but it does not really exist? I wonder if we will think of the word "consciousness" in the same way we think of the word "aether" in the future.

Consciousness exists by definition. It is a subjective experience and you are experiencing it.

Or more precisely I am experiencing it, but since you are, I think, a human just like me, with a brain and biology very similar to mine, I assume that you must experience consciousness too.

And that is the big question, and one that science may never have an answer to. What is conscious and what is not? The only consciousness I know exists is mine, the rest is just guessing based on similarities. Other humans: almost certainly conscious, higher intelligence life forms: probably conscious, plants: probably not conscious, rocks: almost certainly not conscious, LLMs: ???

LLMs are unusual in that they have a very human-like behavior, more than anything non human, which would make them a good candidate for having consciousness, but on a material level, they are more like rocks, which point them as not being conscious.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#96

Post by IAmGraydon »

t43562 wrote: post #43
fedeb95 wrote: post #11

That words are a byproduct of consciousness is a bold claim and requires some justification. I'm not implying it's backwards also for humans, but that the statement might not be true, and something else could be. Also, defining consciousness is hard. For instance, you can dream words.

Have you ever had impressions about something complicated, e.g. someone's personality which you cannot put into words? I'm thinking of my kid - her personality when she was tiny. I could see what it was but not explain it. Hence I think ideas are unquestionably first and words are the inadequate and inaccurate description of thoughts that exist without them.

I agree, and I think that's a good example. Sometimes we can hold a concept in our minds of something and cannot find the words to attach to it. The word we eventually give that concept when it's human, as in your example, is a name. This is because the concept of a personality is too complex to describe with other, preexisting words.

Still, even though I think the author is right about consciousness/concepts preceding language, I think he's wrong about the other part. The statistical models that underlie AI very much remind me of what we call a human concept. It's a statistical model of that thing, person, or idea. However, we humans seem to have this ability to create new concepts that are wildly different than our training data (what we have experienced in life), and we call this creativity. It's missing from AI for some reason. That is the part that makes me question if AI will ever match humans.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#97

Post by epihelix »

t43562 wrote: post #43
fedeb95 wrote: post #11

That words are a byproduct of consciousness is a bold claim and requires some justification. I'm not implying it's backwards also for humans, but that the statement might not be true, and something else could be. Also, defining consciousness is hard. For instance, you can dream words.

Have you ever had impressions about something complicated, e.g. someone's personality which you cannot put into words? I'm thinking of my kid - her personality when she was tiny. I could see what it was but not explain it. Hence I think ideas are unquestionably first and words are the inadequate and inaccurate description of thoughts that exist without them.

Have you ever reasoned through a complex coding or maths problem? Tell me, how did you do that without words?

You (and TFA) are making a false dichotomy here. Yes, of course we think in images. But we also have an inner monologue that is critically important for much of our higher-level thought. How do you even write a HN comment without thinking through it in words?

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#98

Post by enaaem »

yieldcrv wrote: post #5
DevarshRanpara wrote: post #2

Stop for one second and ask yourself a simple question. Where do your words come from? When you speak, what comes first, the idea or the word? Do you first feel a thought inside you, and only after that go searching for the right word to wrap around it? I think we all do. The word is never the start. The word is just the skin. The idea, the consciousness, is the thing sitting under it.

We’re stochastic parrots most of the time, with a varying ability to reason but opting for pattern conformity usually

In some meditation practices, the task is to observe the stochastic parrot in your head. You will soon realise that you cannot control this parrot. If you cannot control your thoughts, could it be that you are not your thoughts? And if you are not your thoughts--the word stream in your head--what are you then?

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#99

Post by sometimelurker »

As I see it, LLMs will/do develop 'introspectiveness' if it is selected for during RL. If its not delibrativly selected for (this is probably done because it helps with alignment; if the machine can align itself that's great) its probably a generally useful skill to have when doing normal tasks, https://www.anthropic.com/research/introspection

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#100

Post by jvanderbot »

sometimelurker wrote: post #99

As I see it, LLMs will/do develop 'introspectiveness' if it is selected for during RL. If its not delibrativly selected for (this is probably done because it helps with alignment; if the machine can align itself that's great) its probably a generally useful skill to have when doing normal tasks, https://www.anthropic.com/research/introspection

Their introspectiveness doesn't strike me as indicative of anything, since we might be just reinforcing a "plausible word salad generator for input context that leads to better solutions" or just a "post hoc rationalizer", both of which may or may not be the key mechanism behind verbal consciousness.
Post reply on HN