121–130 of 204 posts
Writing is a shadow of thought. The better the writing, the more clearly the shadow represents the shape of the original thought. Even the best writing can never perfectly capture the original thought. Writing is one of the best tools we have to share thoughts across space and time.
It is not all obvious that consciousness precedes language. Hellen Keller provides a vivid description of her pre-verbal mental state as something less than reflective conscious. > Before my teacher came to me, I did not know that I am. I lived in a world that was a no-world. I cannot hope to describe adequately that unconscious, yet conscious time of nothingness. I did not know that I knew aught, or that I lived or…
It is likely that language forms a sort of quanta of reasoning. It is far easier to posit a verbal hypothesis pertaining to verbal subjects and test it than to try and do it nonverbally. Maths is probably a great example of this. Try and describe Pythagoras' Theorem without using maths notation or words. Difficult, right? Reasoning is a House of Cards, and without understanding what the card is that exercise becomes…
LLMs as clearly doing more than just modeling words. In order to predict word placement, they need to build some kind of model, their latent space, of the types of things they are able to predict. Not really full world models yet - but they have decent "blog space" or "github project" models. And you can see this with multimodality, or non-text modality modals such as images and audio. They map from their latent spac…
Except that the Chinese Room shows that the existence of a mapping from input to output, however emergently it might have been devised, is not alone sufficient to demonstrate understanding.
No!! This is misinformation.
An LLM predicts the next words (tokens) based on the words before it *and* on the internal representations it has learned from training.
Processing language can lead to internal representations that capture patterns and relationships; LLMs can exhibit emergent abilities, including reasoning-like (stress on "-like") behavior, even when they weren't explicitly trained for that.
Making reductionist claims about LLMs, based on next-word prediction, is similar to making such claims about biology, based on amino acids.
LLMs as clearly doing more than just modeling words. In order to predict word placement, they need to build some kind of model, their latent space, of the types of things they are able to predict. Not really full world models yet - but they have decent "blog space" or "github project" models. And you can see this with multimodality, or non-text modality modals such as images and audio. They map from their latent spac…
The assumption of this sort of argument, if I understand it correctly, is that consciousness is just an ordinary byproduct that appears (or grows gradually) somewhere on the spectrum of complexity. If that’s true, mastering language is basically orthogonal to being conscious (as you’d maybe expect, GPT-2 was pretty good at language, but had a relatively tiny amount of “neurons”). The same is arguably true for world m…
It is not all obvious that consciousness precedes language. Hellen Keller provides a vivid description of her pre-verbal mental state as something less than reflective conscious. > Before my teacher came to me, I did not know that I am. I lived in a world that was a no-world. I cannot hope to describe adequately that unconscious, yet conscious time of nothingness. I did not know that I knew aught, or that I lived or…
I think this is different if you have vision though. I struggle to understand how someone can look at intelligent animals that don’t have language and not intuitively assume that they’re conscious. Yes this is a tricky problem without an answer but IMO the elements of consciousness can exist without language if visual abstractions are present.
If for no other reason, I believe it's quite reasonable to believe that cognition must be Turing-complete (okay, linear bounded automaton), and there must be some kind of support for recursive structures, no matter what the "primitives" of such consciousness is (either visual or linguistic).
I believe in the case of animal intelligence, they (we) have a pretty good hard-coded 3d/4d world model that ought to encode some level of recursion intrinsically (in thought, we very very often use spatial and temporal words), e.g. understanding the motion of another animal, e.g. a rodent running through a tree trunk requires imagination and future prediction already over an abstract agent. Humans just probably have more "stacks" available, and can offload them into either limited "brain memory", or very importantly drawings, text, etc, for unbounded memory.
But I'm just thinking out loud, I'm absolutely no expert on any of these topics.
Earlier quoted context omitted.
IMO it's far more likely that people don't agree on what it means to "have an internal monologue" than that people actually bifurcate that dramatically on cognition. Surely there are some people with an actual difference from baseline, but I don't think it's the near-equal split that seems to appear when you just ask people about internal monologue This is in contrast with aphantasia (complete lack of mental imagery)…
My mom has aphantasia and we were both shocked to discover it even exists. She saw something about it online and then started asking everyone in the family if they could picture an apple in their mind. We all said - yes of course, what are you talking about? And she said, "When you say the word apple, I think of the concept of an apple. It's juicy. It's crisp. It's red, etc, but I don't SEE an apple. I just know what…
It's unfortunately very insulting when someone says, "hey remember when we did that very important thing together?" and my answer has always been "uhh... I remember that we did it?"
It is not all obvious that consciousness precedes language. Hellen Keller provides a vivid description of her pre-verbal mental state as something less than reflective conscious. > Before my teacher came to me, I did not know that I am. I lived in a world that was a no-world. I cannot hope to describe adequately that unconscious, yet conscious time of nothingness. I did not know that I knew aught, or that I lived or…
Super interesting quote and link though!
[0] Fedorenko et al, https://www.nature.com/articles/s41586-024-07522-w
Earlier quoted context omitted.
Except that the Chinese Room shows that the existence of a mapping from input to output, however emergently it might have been devised, is not alone sufficient to demonstrate understanding.
The Chinese Room as a whole understands.
This in the sense that we can easily retain human "understanding" by stripping away almost all parts of the human body or replacing them with fairly trivially made replacements, except for the brain.
In the Chinese Room the equivalent is the magic rulebook: We have no idea how to construct/replace it, yet people somehow handwave that away whilst simultaneously confidently asserting it does not understand anything.
Earlier quoted context omitted.
Similarly, I'm reading "Supernormal Stimuli" now, and there's a whole section on infants raised by various animals (chimps, wolves, chickens, leopards, etc, etc). When they are found and brought back to human societies, basically none of them develop full language and integrate fully. There's also the grim history of language deprivation experiments: https://en.wikipedia.org/wiki/Language_deprivation_experimen... It…
That humans who have grown out of their optimal brain plasticity and hence never acquire the capacity for ordinary human language are not somehow "conscious" is not supported. Rather, it means that, by its very definition, we do not have the means to communicate with them in order to even ask the sorts of questions necessary to determine such a thing. The premise is unfalsifiable.