LLMs as clearly doing more than just modeling words. In order to predict word placement, they need to build some kind of model, their latent space, of the types of things they are able to predict. Not really full world models yet - but they have decent "blog space" or "github project" models. And you can see this with multimodality, or non-text modality modals such as images and audio. They map from their latent spac…
Except that the Chinese Room shows that the existence of a mapping from input to output, however emergently it might have been devised, is not alone sufficient to demonstrate understanding.
Words Are a Byproduct of Consciousness. For LLMs, It's Backwards
81–90 of 204 posts
Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards
#82LLMs as clearly doing more than just modeling words. In order to predict word placement, they need to build some kind of model, their latent space, of the types of things they are able to predict. Not really full world models yet - but they have decent "blog space" or "github project" models. And you can see this with multimodality, or non-text modality modals such as images and audio. They map from their latent spac…
Except that the Chinese Room shows that the existence of a mapping from input to output, however emergently it might have been devised, is not alone sufficient to demonstrate understanding.
... we now have LLMs to even more pointedly show the deficiencies of the argument.
Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards
#83Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards
#84That words are a byproduct of consciousness is a bold claim and requires some justification. I'm not implying it's backwards also for humans, but that the statement might not be true, and something else could be. Also, defining consciousness is hard. For instance, you can dream words.
Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards
#85It is not all obvious that consciousness precedes language. Hellen Keller provides a vivid description of her pre-verbal mental state as something less than reflective conscious. > Before my teacher came to me, I did not know that I am. I lived in a world that was a no-world. I cannot hope to describe adequately that unconscious, yet conscious time of nothingness. I did not know that I knew aught, or that I lived or…
There are really three main observations that give pause. First, after she started working with Sullivan, Keller apparently developed a writing style and nuanced opinions reminiscent of a college-educated person virtually overnight. That’s pretty surprising for any child, let alone one who had only recently acquired language. Second, all of that disappeared overnight when Sullivan died - Keller’s writing style changed drastically, becoming much simpler, and all that eloquent insight also disappeared along with Sullivan. Finally, we’ve never since seen another Helen Keller. On the contrary, later observation of deaf-blind people who don’t acquire language until later in life is part of what inspired a variant of the critical period hypothesis that posits that people who don’t learn their first language (doesn’t matter whether it’s spoken or signed) in early childhood are never able to acquire any language to a particularly high level of proficiency.
That said, even if that Keller quote is now considered dubious, it may not be entirely off base. Supposedly people who don’t acquire language in early childhood also tend to show less capacity for abstract reasoning tasks. Which is still a far cry from the “no-consciousness” that Keller-maybe-Sullivan describes, but does still suggest that language and reasoning are mutually supportive.
Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards
#86Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards
#87Earlier quoted context omitted.
Well, language is not really required for thinking and conscious experiences, a huge chunk of people don't even have an internal monologue, and some (that'd be me) don't have any "byproducts" of thought process. Though there are recent experiments showing that unconscious brain retains language processing abilities [1], so the two might be as well independent systems [1]: https://www.nature.com/articles/s41586-026-10…
IMO it's far more likely that people don't agree on what it means to "have an internal monologue" than that people actually bifurcate that dramatically on cognition. Surely there are some people with an actual difference from baseline, but I don't think it's the near-equal split that seems to appear when you just ask people about internal monologue This is in contrast with aphantasia (complete lack of mental imagery)…
Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards
#88Earlier quoted context omitted.
Similarly, I'm reading "Supernormal Stimuli" now, and there's a whole section on infants raised by various animals (chimps, wolves, chickens, leopards, etc, etc). When they are found and brought back to human societies, basically none of them develop full language and integrate fully. There's also the grim history of language deprivation experiments: https://en.wikipedia.org/wiki/Language_deprivation_experimen... It…
> Similarly, I'm reading "Supernormal Stimuli" now, and there's a whole section on infants raised by various animals (chimps, wolves, chickens, leopards, etc, etc). When they are found and brought back to human societies, basically none of them develop full language and integrate fully. Infants raised by chickens? Chickens!? That doesn't sound real.
Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards
#89Maybe consciousness just does not exist. Could it be that humans invented it to feel superior to animals, but it does not really exist? I wonder if we will think of the word "consciousness" in the same way we think of the word "aether" in the future.
Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards
#90Maybe consciousness just does not exist. Could it be that humans invented it to feel superior to animals, but it does not really exist? I wonder if we will think of the word "consciousness" in the same way we think of the word "aether" in the future.