It still boggles my mind why an amazing text autocompletion system trained on millions of books and other texts is forced to be squeezed through the shape of a prompt/chat interface, which is obviously not the shape of most of its training data. Using it as chat reduces the quality of the output significantly already.
A non-anthropomorphized view of LLMs
251–260 of 432 posts
Re: A non-anthropomorphized view of LLMs
#252Earlier quoted context omitted.
Well, that's a strong claim of equivalence between computationable models and realty. The consensual view is rather that no map is matching fully the territory, or said otherwise the territory includes ontological components that exceeds even the most sophisticated map that can be ever built.
I believe the consensus view is that physics is computable.
Thanks. I think the original point about the word 'trained' being contentious still stands, as evidenced by this thread :)
Re: A non-anthropomorphized view of LLMs
#253It still boggles my mind why an amazing text autocompletion system trained on millions of books and other texts is forced to be squeezed through the shape of a prompt/chat interface, which is obviously not the shape of most of its training data. Using it as chat reduces the quality of the output significantly already.
What's your suggested alternative?
Re: A non-anthropomorphized view of LLMs
#254Earlier quoted context omitted.
> It seems like, we can at best, claim that we have modeled the human thought process for reasoning/analytic/quantitative through Linear Algebra....I don't understand is what about linear algebra that makes it so special that it creates a fully functioning life or aspects of a life? Not linear algebra. Artificial neural networks create arbitrarily non-linear functions. That's the point of non-linear activation functi…
ANNs are just mathematical transformations, powered by linear algebra + non-linear functions. They simulate certain cognitive processes — but they are fundamentally math, not magic.
I think the point of mine that you're missing (or perhaps disagreeing with implicitly) is that *everything* is fundamentally math. Or, if you like, everything is fundamentally physics, and physics is fundamentally math.
So classes of functions (ANNs) that can approximate our desired function to arbitrary precision are what we should be expecting to be working with.
Re: A non-anthropomorphized view of LLMs
#255A person’s anthropomorphization of LLMs is directly related to how well they understand LLMs.
Once you dispel the magic, it naturally becomes hard to use words related to consciousness, or thinking. You will probably think of LLMs more like a search engine: you give an input and get some probable output. Maybe LLMs should be rebranded as “word engines”?
Regardless, anthropomorphization is not helpful, and by using human terms to describe LLMs you are harming the layperson’s ability to truly understand what an LLM is while also cheapening what it means to be human by suggesting we’ve solved consciousness. Just stop it. LLMs do not think, given enough time and patience you could compute their output by hand if you used their weights and embeddings to manually do all the math, a hellish task but not an impossible one technically. There is no other secret hidden away, that’s it.
Re: A non-anthropomorphized view of LLMs
#256The problem with viewing LLMs as just sequence generators, and malbehaviour as bad sequences, is that it simplifies too much. LLMs have hidden state not necessarily directly reflected in the tokens being produced and it is possible for LLMs to output tokens in opposition to this hidden state to achieve longer term outcomes (or predictions, if you prefer). Is it too anthropomorphic to say that this is a lie? To say th…
Maybe it's just because so much of my work for so long has focused on models with hidden states but this is a fairly classical feature of some statistical models. One of the widely used LLM textbooks even started with latent variable models; LLMs are just latent variable models just on a totally different scale, both in terms of number of parameters but also model complexity. The scale is apparently important, but seeing them as another type of latent variable model sort of dehumanizes them for me.
Latent variable or hidden state models have their own history of being seen as spooky or mysterious though; in some ways the way LLMs are anthropomorphized is an extension of that.
I guess I don't have a problem with anthropomorphizing LLMs at some level, because some features of them find natural analogies in cognitive science and other areas of psychology, and abstraction is useful or even necessary in communicating and modeling complex systems. However, I do think anthropomorphizing leads to a lot of hype and tends to implicitly shut down thinking of them mechanistically, as a mathematical object that can be probed and characterized — it can lead to a kind of "ghost in the machine" discourse and an exaggeration of their utility, even if it is impressive at times.
Re: A non-anthropomorphized view of LLMs
#257> I am baffled that the AI discussions seem to never move away from treating a function to generate sequences of words as something that resembles a human. This is such a bizarre take. The relation associating each human to the list of all words they will ever say is obviously a function. > almost magical human-like powers to something that - in my mind - is just MatMul with interspersed nonlinearities. There's a ric…
>There's a rich family of universal approximation theorems Wow, look-up tables can get increasingly good at approximating a function!
A function is by definition a lookup table.
The lookup table is just (x, f(x)).
So, yes, trivially if you could construct the lookup table for f then you'd approximate f. But to construct it you have to know f. And to approximate it you need to know f at a dense set of points.
Re: A non-anthropomorphized view of LLMs
#258Re: A non-anthropomorphized view of LLMs
#259Earlier quoted context omitted.
I kinda agree with both of you. It might be a required abstraction, but it's a leaky one. Long before LLMs, I would talk about classes / functions / modules like "it then does this, decides the epsilon is too low, chops it up and adds it to the list". The difference I guess it was only to a technical crowd and nobody would mistake this for anything it wasn't. Everybody know that "it" didn't "decide" anything. With AI…
Agreeing with you, this is a "can a submarine swim" problem IMO. We need a new word for what LLMs are doing. Calling it "thinking" is stretching the word to breaking point, but "selecting the next word based on a complex statistical model" doesn't begin to capture what they're capable of. Maybe it's cog-nition (emphasis on the cog).
It will help significantly, to realize that the only thinking happening is when the human looks at the output and attempts to verify if it is congruent with reality.
The rest of the time it’s generating content.
Re: A non-anthropomorphized view of LLMs
#260> I am baffled that the AI discussions seem to never move away from treating a function to generate sequences of words as something that resembles a human. This is such a bizarre take. The relation associating each human to the list of all words they will ever say is obviously a function. > almost magical human-like powers to something that - in my mind - is just MatMul with interspersed nonlinearities. There's a ric…
Not that this is your main point, but I find this take representative, “do you believe there's anything about humans that exists outside the mathematical laws of physics?”There are things “about humans”, or at least things that our words denote, that are outside physic’s explanatory scope. For example, the experience of the colour red cannot be known, as an experience, by a person who only sees black and white. This…
This idea is called qualia [0] for those unfamiliar.
I don't have any opinion on the qualia debates honestly. I suppose I don't know what it feels like for an ant to find a tasty bit of sugar syrup, but I believe it's something that can be described with physics (and by extension, things like chemistry).
But we do know some things about some qualia. Like we know how red light works, we have a good idea about how photoreceptors work, etc. We know some people are red-green colorblind, so their experience of red and green are mushed together. We can also have people make qualia judgments and watch their brains with fMRI or other tools.
I think maybe an interesting question here is: obviously it's pleasurable to animals to have their reward centers activated. Is it pleasurable or desirable for AIs to be rewarded? Especially if we tell them (as some prompters do) that they feel pleasure if they do things well and pain if they don't? You can ask this sort of question for both the current generation of AIs and future generations.