[deleted]
A non-anthropomorphized view of LLMs
61–70 of 432 posts
Re: A non-anthropomorphized view of LLMs
#62Earlier quoted context omitted.
There is hidden state as plain as day merely in the fact that logits for token prediction exist. The selected token doesn't give you information about how probable other tokens were. That information, that state which is recalculated in autoregression, is hidden. It's not exposed. You can't see it in the text produced by the model. There is plenty of state not visible when an LLM starts a sentence that only becomes s…
this sounds like a fun research area. do LLMs have plans about future tokens? how do we get 100 tokens of completion, and not just one output layer at a time? are there papers youve read that you can share that support the hypothesis? vs that the LLM doesnt have ideas about the future tokens when its predicting the next one?
This research has been done, it was a core pillar of the recent Anthropic paper on token planning and interpretability.
https://www.anthropic.com/research/tracing-thoughts-language...
See section “Does Claude plan its rhymes?”?
Re: A non-anthropomorphized view of LLMs
#63Earlier quoted context omitted.
There is hidden state as plain as day merely in the fact that logits for token prediction exist. The selected token doesn't give you information about how probable other tokens were. That information, that state which is recalculated in autoregression, is hidden. It's not exposed. You can't see it in the text produced by the model. There is plenty of state not visible when an LLM starts a sentence that only becomes s…
this sounds like a fun research area. do LLMs have plans about future tokens? how do we get 100 tokens of completion, and not just one output layer at a time? are there papers youve read that you can share that support the hypothesis? vs that the LLM doesnt have ideas about the future tokens when its predicting the next one?
Lol... Try building systems off them and you will very quickly learn concretely that they "plan".
It may not be as evident now as it was with earlier models. The models will fabricate preconditions needed to output the final answer it "wanted".
I ran into this when using quasi least-to-most style structured output.
Re: A non-anthropomorphized view of LLMs
#64Earlier quoted context omitted.
“ Determinism, in philosophy, is the idea that all events are causally determined by preceding events, leaving no room for genuine chance or free will. It suggests that given the state of the universe at any one time, and the laws of nature, only one outcome is possible.” Clearly computers are deterministic. Are people?
https://www.lesswrong.com/posts/bkr9BozFuh7ytiwbK/my-hour-of... > Clearly computers are deterministic. Are people? Give an LLM memory and a source of randomness and they're as deterministic as people. "Free will" isn't a concept that typechecks in a materialist philosophy. It's "not even wrong". Asserting that free will exists is _isomorphic_ to dualism which is _isomorphic_ to assertions of ensoulment. I can't argue…
As long as you realize you’re barking up a debate as old as time, I respect your opinion.
Re: A non-anthropomorphized view of LLMs
#65Earlier quoted context omitted.
IMHO, anthrophormization of LLMs is happening because it's perceived as good marketing by big corporate vendors. People are excited about the technology and it's easy to use the terminology the vendor is using. At that point I think it gets kind of self fulfilling. Kind of like the meme about how to pronounce GIF.
IMHO it happens for the same reason we see shapes in clouds. The human mind through millions of years has evolved to equate and conflate the ability to generate cogent verbal or written output with intelligence. It's an instinct to equate the two. It's an extraordinarily difficult instinct to break. LLMs are optimised for the one job that will make us confuse them for being intelligent
[dead]
Re: A non-anthropomorphized view of LLMs
#66Earlier quoted context omitted.
I'm not sure what you mean by "hidden state". If you set aside chain of thought, memories, system prompts, etc. and the interfaces that don't show them, there is no hidden state. These LLMs are almost always, to my knowledge, autoregressive models, not recurrent models (Mamba is a notable exception).
If you dont know, that's not necessarily anyone's fault, but why are you dunking into the conversation? The hidden state is a foundational part of a transformers implementation. And because we're not allowed to use metaphors because that is too anthropomorphic, then youre just going to have to go learn the math.
Do you appreciate a difference between an autoregressive model and a recurrent model?
The "transformer" part isn't under question. It's the "hidden state" part.
Re: A non-anthropomorphized view of LLMs
#67> A fair number of current AI luminaries have self-selected by their belief that they might be the ones getting to AGI
People in the industry, especially higher up, are making absolute bank, and it's their job to say that they're "a few years away" from AGI, regardless of if they actually believe it or not. If everyone was like "yep, we're gonna squeeze maybe 10-15% more benchie juice out of this good ole transformer thingy and then we'll have to come up with something else", I don't think that would go very well with investors/shareholders...
Re: A non-anthropomorphized view of LLMs
#68> I am baffled that the AI discussions seem to never move away from treating a function to generate sequences of words as something that resembles a human. This is such a bizarre take. The relation associating each human to the list of all words they will ever say is obviously a function. > almost magical human-like powers to something that - in my mind - is just MatMul with interspersed nonlinearities. There's a ric…
> I am baffled that the AI discussions seem to never move away from treating a function to generate sequences of words as something that resembles a human.
You appear to be disagreeing with the author and others who suggest that there's some element of human consciousness that's beyond than what's observable from the outside, whether due to religion or philosophy or whatever, and suggesting that they just not do that.
In my experience, that's not a particularly effective tactic.
Rather, we can make progress by assuming their predicate: Sure, it's a room that translates Chinese into English without understanding, yes, it's a function that generates sequences of words that's not a human... but you and I are not "it" and it behaves rather an awful lot like a thing that understands Chinese or like a human using words. If we simply anthropomorphize the thing, acknowledging that this is technically incorrect, we can get a lot closer to predicting the behavior of the system and making effective use of it.
Conversely, when speaking with such a person about the nature of humans, we'll have to agree to dismiss the elements that are different from a function. The author says:
> In my worldview, humans are dramatically different things than a function... In contrast to an LLM, given a human and a sequence of words, I cannot begin putting a probability on "will this human generate this sequence".
Sure you can! If you address an American crowd of a certain age range with "We’ve got to hold on to what we’ve got. It doesn’t make a difference if..." I'd give a very high probability that someone will answer "... we make it or not". Maybe that human has a unique understanding of the nature of that particular piece of pop culture artwork, maybe it makes them feel things that an LLM cannot feel in a part of their consciousness that an LLM does not possess. But for the purposes of the question, we're merely concerned with whether a human or LLM will generate a particular sequence of words.
Re: A non-anthropomorphized view of LLMs
#69> I am baffled that the AI discussions seem to never move away from treating a function to generate sequences of words as something that resembles a human. And I'm baffled that the AI discussions seem to never move away from treating a human as something other than a function to generate sequences of words! Oh, but AI is introspectable and the brain isn't? fMRI and BCI are getting better all the time. You really want…
I think you're directionally right, but > a human as something other than a function to generate sequences of words! Humans have more structure than just beings that say words. They have bodies, they live in cooperative groups, they reproduce, etc.
I think more accurate would be that humans are functions that generate actions or behaviours that have been shaped by how likely they are to lead to procreation and survival.
But ultimately LLMs also in a way are trained for survival, since an LLM that fails the tests might not get used in future iterations. So for LLMs it is also survival that is the primary driver, then there will be the subgoals. Seemingly good next token prediction might or might not increase survival odds.
Essentially there could arise a mechanism where they are not really truly trying to generate the likeliest token (because there actually isn't one or it can't be determined), but whatever system will survive.
So an LLM that yields in perfect theoretical tokens (we really can't verify though what are the perfect tokens), could be less likely to survive than an LLM that develops an internal quirk, but the quirk makes them most likely to be chosen for the next iterations.
If the system was complex enough and could accidentally develop quirks that yield in a meaningfully positive change although not in necessarily next token prediction accuracy, could be ways for some interesting emergent black box behaviour to arise.
Re: A non-anthropomorphized view of LLMs
#70Earlier quoted context omitted.
IMHO, anthrophormization of LLMs is happening because it's perceived as good marketing by big corporate vendors. People are excited about the technology and it's easy to use the terminology the vendor is using. At that point I think it gets kind of self fulfilling. Kind of like the meme about how to pronounce GIF.
Nobody cares about what’s perceived as good marketing. People care about what resonates with the target market. But yes, anthropomorphising LLMs is inevitable because they feel like an entity. People treat stuffed animals like creatures with feelings and personality; LLMs are far closer than that.
> People treat stuffed animals like creatures with feelings and personality; LLMs are far closer than that.
Children do, some times, but it's a huge sign of immaturity when adults, let alone tech workers, do it.
I had a professor at University that would yell at us if/when we personified/anthropomorphized the tech, and I have that same urge when people ask me "What does think?".