Earlier quoted context omitted.
I'm definitely a stream of words. My "abstract thoughts" are a stream of words too, they just don't get sounded out. Tbf I'd rather they weren't there in the first place. But bodies which refuse to harbor an "interiority" are fast-tracked to destruction because they can't suf^W^W^W be productive. Funny movie scene from somewhere. The sergeant is drilling the troops: "You, private! What do you live for!", and expects…
Then what are non-human animals doing?
Bag of words, have mercy on us
321–330 of 362 posts
Re: Bag of words, have mercy on us
#322Earlier quoted context omitted.
If you want actionable intuition, try "a human with almost zero self-awareness". "Self-awareness" used in a purely mechanical sense here: having actionable information about itself and its own capabilities. If you ask an old LLM whether it's able to count the Rs in "strawberry" successfully, it'll say "yes". And then you ask it to do so, and it'll say "2 Rs". It doesn't have the self-awareness to know the practical l…
Agree that's a better intuition, with pretraining pushing the model towards saying "I don't know" in the kinds of situations where people write that as opposed to by introspection of its own confidence.
Anthropic has discovered that this is definitely the case for name recognition, and I suspect that names aren't the only things subject to a process like that.
Re: Bag of words, have mercy on us
#323Every day I see people treat gen AI like a thinking human, Dijkstra's attitudes about anthropomorphizing computers is vindicated even more. That said, I think the author's use of "bag of words" here is a mistake. Not only does it have a real meaning in a similar area as LLMs, but I don't think the metaphor explains anything. Gen AI tricks laypeople into treating its token inferences as "thinking" because it is traine…
I'll make the following observation: The contra-positive of "All LLMs are not thinking like humans" is "No humans are thinking like LLMs" And I do not believe we actually understand human thinking well enough to make that assertion. Indeed, it is my deep suspicion that we will eventually achieve AGI not by totally abandoning today's LLMs for some other paradigm, but rather embedding them in a loop with the right pers…
Re: Bag of words, have mercy on us
#324Every day I see people treat gen AI like a thinking human, Dijkstra's attitudes about anthropomorphizing computers is vindicated even more. That said, I think the author's use of "bag of words" here is a mistake. Not only does it have a real meaning in a similar area as LLMs, but I don't think the metaphor explains anything. Gen AI tricks laypeople into treating its token inferences as "thinking" because it is traine…
Bag of words is actually the perfect metaphor. The data structure is a bag. The output is a word. The selection strategy is opaquely undefined. > Gen AI tricks laypeople into treating its token inferences as "thinking" because it is trained to replicate the semiotic appearance of doing so. A "bag of words" doesn't sufficiently explain this behavior. Something about there being significant overlap between the smartest…
Re: Bag of words, have mercy on us
#325Earlier quoted context omitted.
I'm definitely a stream of words. My "abstract thoughts" are a stream of words too, they just don't get sounded out. Tbf I'd rather they weren't there in the first place. But bodies which refuse to harbor an "interiority" are fast-tracked to destruction because they can't suf^W^W^W be productive. Funny movie scene from somewhere. The sergeant is drilling the troops: "You, private! What do you live for!", and expects…
I doubt words are involved when we e.g. solve a mathematical problem. To me, solving problems happens in a logico/aesthetical space which may be the same as when you are intellectually affected by a work of art. I don't remember myself being able to translate directly into words what I feel for a great movie or piece of music, even if in the late I can translate this "complex mental entity" into words, exactly like I…
The twist about words in particular is they are distinctly articulable symbols, i.e. you can sound 'em out - and thus, presumably, have a reasonable expectation for bearers of the same language to comprehend if not what you meant then at least some vaguely predictable meaning-cloud associated with the given speech act.
That's unlike e.g. the numbers (which are more compressed, and thus easier to get wrong), or the syntagms of a programming language (which don't even have a canonical sonic representation).
Therefore, it's usually words that are taught to a mind during the formative stages of its emergence. That is, the words that you are taught, your means of inner reflection, are still sort of an imposition from the outside.
Just consider what you life trajectory would've been if in your childhood you had refused to learn any words, or learned them and then refused to mistake them for the things they represent!
Infants and even some animals recognize their reflection in a mirror; however, practically speaking, introspection is something that one needs to be taught: after recognizing your reflection you still need to be instructed what is to be done about it.
Unfortunately, introspection needing to be taught means that introspection can be taught wrongly.
As you can see with the archetypical case of "old and wise person does something completely stupid in response to communication via digital device", a common failure mode of how people are taught introspection (and, I figure, an intentional one!) is not being able to tell apart yourself from your self, i.e. not having an intuitive sense of where the boundary lies between perception and cognition, i.e. going through life without ever learning the difference between the "you" and the "words about you".
It's extremely common, and IMO an extremely factory-farming kind of tragic.
I say it must be extremely intentional as well, because the well-known practice of using "introspection modulators" to establish some sort of perceptual point of reference (such as where the interior logicoaeshtetical space ends and exterior causalityspace begins) very often ends up with the user in, well, a cage of some sort.
Re: Bag of words, have mercy on us
#326They are search engines that can remix results.
I like this one because I think most modern folks have a usefully accurate model of what a search engine is in their heads, and also what "remixing" is, which adds up to a better metaphor than "human machine" or whatever.
Re: Bag of words, have mercy on us
#327Earlier quoted context omitted.
Are you a stream of words or are your words the “simplistic” projection of your abstract thoughts? I don’t at all discount the importance of language in so many things, but the question that matters is whether statistical models of language can ever “learn” abstract thought, or become part of a system which uses them as a tool. My personal assessment is that LLMs can do neither.
Words are the "simplistic" projection of an LLM's abstract thoughts. An LLM has: words in its input plane, words in its output plane, and A LOT of cross-linked internals between the two. Those internals aren't "words" at all - and it's where most of the "action" happens. It's how LLMs can do things like translate from language to language, or recall knowledge they only encountered in English in the training data whil…
Re: Bag of words, have mercy on us
#328Everyone is out here acting like "predicting the next thing" is somehow fundamentally irrelevant to "human thinking" and it is simply not the case. What does it mean to say that we humans act with intent? It means that we have some expectation or prediction about how our actions will effect the next thing, and choose our actions based on how much we like that effect. The ability to predict is fundamental to our abili…
Re: Bag of words, have mercy on us
#329The quantitative and qualitative difference between (a) "all words ever written" and (b) "ones that could be scraped off the internet or scanned out of book" easily exceeds the size of any LLM
Compared to (a), (b) is a tiny pouch, not even a bag
Opinions may differ on whether (b) is a representative sample of (a)
The words "scanned out of a book" would seem to be the most useful IMHO but the AI companies do not have enough words from those sources to produce useful general purpose LLMs
They have to add words "that could be scraped off the internet" which, let's be honest, is mostly garbage
Re: Bag of words, have mercy on us
#330As a consequence of my profession, I understand how LLMs work under the hood. I also know that we data and tech folks will probably never win the battle over anthropomorphization. The average user of AI, nevermind folks who should know better, is so easily convinced that AI "knows," "thinks," "lies," "wants," "understands," etc. Add to this that all AI hosts push this perspective (and why not, it's the easiest white…