Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

ranpara.net

Post reply on HN
Print view View original

71–80 of 204 posts

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#71

Post by encyclopedism »

Festro wrote: post #34

"At its core, it is a big pile of words that predicts the next word, using some maths the computers figured out." "An LLM predicts the next word based on all the words before it. That is the whole story. There is no idea sitting underneath. The words are everything." This is just flat out wrong. Words are used as training data, to build a system of vector embeddings. The LLM contains no words. That was the training d…

Defining consciousness is one of the most difficult outstanding problems in the whole of science please see: https://en.wikipedia.org/wiki/Hard_problem_of_consciousness

An approach that might shed some light is instead to define what consciousness ISN'T. Naively let us say consciousness is NOT a large list of weights (i.e. an LLM).

The uncanny emergent ability depends entirely on training data. A mathematical model is used to match output against training data (via loss functions etc). The training data contains all the human ingenuity, logic, rational, patterns and features.

Try giving an LLM model the alphabet ALONE and see what it comes up with?

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#72

Post by [deleted] »

ohthehugemanate wrote: post #68

This person is just making stuff up that feels true to them. Caveat lector. There are reams of grown up scientific research and philosophy on the relationship between language, thought, neurobiology, and inference. It's nice to hear how one person feels they think, but there is no weight to it if they don't know page one of the syllabus.

[deleted]

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#73

Post by IAmGraydon »

>An LLM predicts the next word based on all the words before it. That is the whole story. There is no idea sitting underneath. The words are everything. For an LLM, words are the source, and any meaning is just a byproduct that falls out by accident.

I don't think that's right. What makes LLMs (and other forms of "AI") work is the very fact that they hold a statistical model underneath and put words on top of it. That statistical model, at least to me, appears to be somewhat akin to what a human would call a "concept".

Like a stable diffusion model has a statistical model of what a "chair" is. Not one exact chair, but all chairs. They all have backs, 4 legs, a flat area to sit on. It can then carve an image of a chair out of noise using this concept. Take a music model like Suno as another example. If you tell it to make a rock song, it has a statistical idea of what this would looks like. The harmonic progression, the tempo, the types of instruments used. It then attaches sounds to this statistical model and you get a "song". LLMs also appear to do this. If you ask it for a paragraph of a specific flavor of prose, it has that as a statistical model, and it attaches words to it.

I am not a big believer in LLMs delivering on even half the hype they've generated. However, this very idea of human concepts vs AI's statistical models is the one thing makes me wonder, sometimes, if they're on the right track, even if we are a long way off from AGI. It's kind of funny that the author of the above article landed on the opposite conclusion.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#74

Post by PaulDavisThe1st »

kibwen wrote: post #60
jacobr1 wrote: post #20

LLMs as clearly doing more than just modeling words. In order to predict word placement, they need to build some kind of model, their latent space, of the types of things they are able to predict. Not really full world models yet - but they have decent "blog space" or "github project" models. And you can see this with multimodality, or non-text modality modals such as images and audio. They map from their latent spac…

Except that the Chinese Room shows that the existence of a mapping from input to output, however emergently it might have been devised, is not alone sufficient to demonstrate understanding.

It does so only in the claims of its creator. Plenty of other people have pointed out fallacies in the claim. My favorite is the Dennett/Hofstadter observation that while the man in the room may not understand chinese, the room/system certainly does.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#75

Post by palmotea »

abound wrote: post #37
michael1999 wrote: post #10

It is not all obvious that consciousness precedes language. Hellen Keller provides a vivid description of her pre-verbal mental state as something less than reflective conscious. > Before my teacher came to me, I did not know that I am. I lived in a world that was a no-world. I cannot hope to describe adequately that unconscious, yet conscious time of nothingness. I did not know that I knew aught, or that I lived or…

Similarly, I'm reading "Supernormal Stimuli" now, and there's a whole section on infants raised by various animals (chimps, wolves, chickens, leopards, etc, etc). When they are found and brought back to human societies, basically none of them develop full language and integrate fully. There's also the grim history of language deprivation experiments: https://en.wikipedia.org/wiki/Language_deprivation_experimen... It…

> Similarly, I'm reading "Supernormal Stimuli" now, and there's a whole section on infants raised by various animals (chimps, wolves, chickens, leopards, etc, etc). When they are found and brought back to human societies, basically none of them develop full language and integrate fully.

Infants raised by chickens? Chickens!? That doesn't sound real.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#76

Post by kaonwarb »

micoti wrote: post #39
throwaway260124 wrote: post #27

"In the beginning was the Word, and the Word was with God, and the Word was God." And Islam the first verse was ‘read’ The first time I heard about emergent behavior in llms I thought about that. Maybe thought is too much because I don’t have anything coherent in mind but it feels so captivating like the premise of sci-fi story.

Word in the Bible is not meaning the actual word (like sematic array of characters) but more like greek word logos

It's a hard word to translate. https://en.wikipedia.org/wiki/Logos has a good overview.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#77

Post by throw310822 »

ohthehugemanate wrote: post #68

This person is just making stuff up that feels true to them. Caveat lector. There are reams of grown up scientific research and philosophy on the relationship between language, thought, neurobiology, and inference. It's nice to hear how one person feels they think, but there is no weight to it if they don't know page one of the syllabus.

> just making stuff up that feels true to them

Hallucinating it?

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#78

Post by ordersofmag »

cowlby wrote: post #30

Isn't it interesting then how much of our intelligence is captured by our words/language? I would've thought to create AI you needed to replicate neurons and synapses and learning. So it's still amazing to me that statistical modelling of our words creates a Claude Opus.

Perhaps LLM's do replicate the important features of "neurons and synapses and learning" and perhaps a "statistical modelling of our words" is pretty much what our brains are doing? Most of the counter-arguments I've seen boil down to "but humans are special" which I'm not sure I find compelling.

Not saying LLM's are conscious. Just that much of our amazement about their behavior seems to say more about us realizing the things we can do are not magic, rather than them being so amazing.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#79

Post by lebuffon »

I look at culture, under which I put language as a component, as the O/S of the human brain. So yes a brain with no O/S can think but it probably will never replicate, in one life time, the accumulated thinking skills of a brain given a good O/S.

How many of us would independently reinvent language with specific tenses, the diatonic scale, the number zero, perspective drawing, geometry, calculus, atomic theory, quantum mechanics etc. if left on our own from childhood?

So to me, the question becomes, can human language encode the sum of all human thought? The answer might be no. But it might be good enough to get close.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#80

Post by PaulDavisThe1st »

sourdecor wrote: post #46

Reminds me of this from "Gödel, Escher, Bach: an Eternal Golden Braid": "Let me rephrase these last couple of sentences without using the slightly technical term "isomorphism". When a system of "meaningless" symbols has patterns in it that accurately track, or mirror, various phenomena in the world, then that tracking or mirroring imbues the symbols with some degree of meaning - indeed, such tracking or mirroring is…

While I know what Douglas Hofstadter's book "Godel Escher Bach - an eternal golden braid" is, it might be worth revisiting your apparent assumption that more or less everybody on HN knows about a book from 47 years ago (regardless of its significance to those of us who have encountered it already).
Post reply on HN