Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A non-anthropomorphized view of LLMs

addxorrol.blogspot.com

Post reply on HN
Print view View original

101–110 of 432 posts

Re: A non-anthropomorphized view of LLMs

#101

Post by lostmsu »

positron26 wrote: post #98
barrkel wrote: post #5

The problem with viewing LLMs as just sequence generators, and malbehaviour as bad sequences, is that it simplifies too much. LLMs have hidden state not necessarily directly reflected in the tokens being produced and it is possible for LLMs to output tokens in opposition to this hidden state to achieve longer term outcomes (or predictions, if you prefer). Is it too anthropomorphic to say that this is a lie? To say th…

> Is it too anthropomorphic to say that this is a lie? Yes. Current LLMs can only introspect from output tokens. You need hidden reasoning that is within the black box, self-knowing, intent, and motive to lie. I rather think accusing an LLM of lying is like accusing a mousetrap of being a murderer. When models have online learning, complex internal states, and reflection, I might consider one to have consciousness an…

> Current LLMs can only introspect from output tokens

The only interpretation of this statement I can come up with is plain wrong. There's no reason LLM shouldn't be able to introspect without any output tokens. As the GP correctly says, most of the processing in LLMs happens over hidden states. Output tokens are just an artefact for our convenience, which also happens to be the way the hidden state processing is trained.

Re: A non-anthropomorphized view of LLMs

#102

Post by lostmsu »

gpm wrote: post #86
barrkel wrote: post #32

Earlier quoted context omitted.

There is hidden state as plain as day merely in the fact that logits for token prediction exist. The selected token doesn't give you information about how probable other tokens were. That information, that state which is recalculated in autoregression, is hidden. It's not exposed. You can't see it in the text produced by the model. There is plenty of state not visible when an LLM starts a sentence that only becomes s…

The LLM does not "have" a plan. Arguably there's reason to believe it comes up with a plan when it is computing token propabilities, but it does not store it between tokens. I.e. it doesn't possess or "have" it. It simply comes up with a plan, emits a token, and entirely throws all its intermediate thoughts (including any plan) to start again from scratch on the next token.

This is wrong, intermediate activations are preserved when going forward.

Re: A non-anthropomorphized view of LLMs

#103

Post by positron26 »

lostmsu wrote: post #101
positron26 wrote: post #98

Earlier quoted context omitted.

> Is it too anthropomorphic to say that this is a lie? Yes. Current LLMs can only introspect from output tokens. You need hidden reasoning that is within the black box, self-knowing, intent, and motive to lie. I rather think accusing an LLM of lying is like accusing a mousetrap of being a murderer. When models have online learning, complex internal states, and reflection, I might consider one to have consciousness an…

> Current LLMs can only introspect from output tokens The only interpretation of this statement I can come up with is plain wrong. There's no reason LLM shouldn't be able to introspect without any output tokens. As the GP correctly says, most of the processing in LLMs happens over hidden states. Output tokens are just an artefact for our convenience, which also happens to be the way the hidden state processing is tra…

There are no recurrent paths besides tokens. How may I introspect something if it is not an input? I may not.

Re: A non-anthropomorphized view of LLMs

#104

Post by lostmsu »

ChadNauseam wrote: post #87
brookst wrote: post #43

Earlier quoted context omitted.

State typically means between interactions . By this definition a simple for loop has “hidden state” in the counter.

Hidden layer is a term of art in machine learning / neural network research. See https://en.wikipedia.org/wiki/Hidden_layer . Somehow this term mutated into "hidden state", which in informal contexts does seem to be used quite often the way the grandparent comment used it.

It makes sense in LLM context because the processing of these is time-sequential in LLM's internal time.

Re: A non-anthropomorphized view of LLMs

#105

Post by lostmsu »

CharlesW wrote: post #10
szvsw wrote: post #3

So the author’s core view is ultimately a Searle-like view: a computational, functional, syntactic rules based system cannot reproduce a mind. Plenty of people will agree, plenty of people will disagree, and the answer is probably unknowable and just comes down to whatever axioms you subscribe to in re: consciousness. The author largely takes the view that it is more productive for us to ignore any anthropomorphic re…

> The flip side of all this is of course the idea that there is still something emergent, unplanned, and mind- like. For people who have only a surface-level understanding of how they work, yes. A nuance of Clarke's law that "any sufficiently advanced technology is indistinguishable from magic" is that the bar is different for everybody and the depth of their understanding of the technology in question. That bar is s…

Nah, as a person that knows in detail how LLMs work with probably unique alternative perspective in addition to the commonplace one, I found any claims of them not having emergent behaviors to be of the same fallacy as claiming that crows can't be black because they have DNA of a bird.

Re: A non-anthropomorphized view of LLMs

#106

Post by lostmsu »

krackers wrote: post #97
fastball wrote: post #73

"Don't anthropomorphize token predictors" is a reasonable take assuming you have demonstrated that humans are not in fact just SOTA token predictors. But AFAIK that hasn't been demonstrated. Until we have a much more sophisticated understanding of human intelligence and consciousness, any claim of "these aren't like us" is either premature or spurious.

Every time this discussion comes up, I'm reminded of this tongue-in-cheek paper. https://ai.vixra.org/pdf/2506.0065v1.pdf

I expected to find the link to https://arxiv.org/abs/1703.10987 (which is much better imo)

Re: A non-anthropomorphized view of LLMs

#107

Post by 3eb7988a1663 »

NiloCK wrote: post #89
8note wrote: post #33

Earlier quoted context omitted.

do LLM models consider future tokens when making next token predictions? eg. pick 'the' as the next token because there's a strong probability of 'planet' as the token after? is it only past state that influences the choice of 'the'? or that the model is predicting many tokens in advance and only returning the one in the output? if it does predict many, id consider that state hidden in the model weights.

The most obvious case of this is in terms of `an apple` vs `a pear`. LLMs never get the a-an distinction wrong, because their internal state 'knows' the word that'll come next.

If I give an LLM a fragment of text that starts with, "The fruit they ate was an ", regardless of any plan, the grammatically correct answer is going to force a noun starting with a vowel. How do you disentangle the grammar from planning?

Going to be a lot more "an apple" in the corpus than "an pear"

Re: A non-anthropomorphized view of LLMs

#108

Post by viccis »

barrkel wrote: post #5

The problem with viewing LLMs as just sequence generators, and malbehaviour as bad sequences, is that it simplifies too much. LLMs have hidden state not necessarily directly reflected in the tokens being produced and it is possible for LLMs to output tokens in opposition to this hidden state to achieve longer term outcomes (or predictions, if you prefer). Is it too anthropomorphic to say that this is a lie? To say th…

I think that the hidden state is really just at work improving the model's estimation of the joint probability over tokens. And the assumption here, which failed miserably in the early 20th century in the work of the logical posivitists, is that if you can so expertly estimate that joint probability of language, then you will be able to understand "knowledge." But there's no well grounded reason to believe that and plenty of the reasons (see: the downfall of logical posivitism) to think that language is an imperfect representation of knowledge. In other words, what humans do when we think is more complicated than just learning semiotic patterns and regurgitating them. Philosophical skeptics like Hume thought so, but most epistemology writing after that had better answers for how we know things.

Re: A non-anthropomorphized view of LLMs

#109

Post by delusional »

lostmsu wrote: post #101
positron26 wrote: post #98

Earlier quoted context omitted.

> Is it too anthropomorphic to say that this is a lie? Yes. Current LLMs can only introspect from output tokens. You need hidden reasoning that is within the black box, self-knowing, intent, and motive to lie. I rather think accusing an LLM of lying is like accusing a mousetrap of being a murderer. When models have online learning, complex internal states, and reflection, I might consider one to have consciousness an…

> Current LLMs can only introspect from output tokens The only interpretation of this statement I can come up with is plain wrong. There's no reason LLM shouldn't be able to introspect without any output tokens. As the GP correctly says, most of the processing in LLMs happens over hidden states. Output tokens are just an artefact for our convenience, which also happens to be the way the hidden state processing is tra…

> Output tokens are just an artefact for our convenience

That's nonsense. The hidden layers are specifically constructed to increase the probability that the model picks the right next word. Without the output/token generation stage the hidden layers are meaningless. Just empty noise.

It is fundamentally an algorithm for generating text. If you take the text away it's just a bunch of fmadds. A mute person can still think, an LLM without output tokens can do nothing.

Re: A non-anthropomorphized view of LLMs

#110

Post by delusional »

cuttothechase wrote: post #100
ants_everywhere wrote: post #96

Earlier quoted context omitted.

> It seems like, we can at best, claim that we have modeled the human thought process for reasoning/analytic/quantitative through Linear Algebra....I don't understand is what about linear algebra that makes it so special that it creates a fully functioning life or aspects of a life? Not linear algebra. Artificial neural networks create arbitrarily non-linear functions. That's the point of non-linear activation functi…

ANNs are just mathematical transformations, powered by linear algebra + non-linear functions. They simulate certain cognitive processes — but they are fundamentally math, not magic.

I wouldn't say they "simulate cognitive processes". They do statistics. Advanced multivariadic statistics.

An LLM thinks in the same way excel thinks when you ask it to fit a curve.

Post reply on HN