Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A non-anthropomorphized view of LLMs

addxorrol.blogspot.com

Post reply on HN
Print view View original

211–220 of 432 posts

Re: A non-anthropomorphized view of LLMs

#211

Post by FeepingCreature »

latexr wrote: post #172
lostmsu wrote: post #105

Earlier quoted context omitted.

Nah, as a person that knows in detail how LLMs work with probably unique alternative perspective in addition to the commonplace one, I found any claims of them not having emergent behaviors to be of the same fallacy as claiming that crows can't be black because they have DNA of a bird.

> the same fallacy as claiming that crows can't be black because they have DNA of a bird. What fallacy is that? I’m a fan of logical fallacies and never heard that claim before nor am I finding any reference with a quick search.

(Not the parent)

It doesn't have a name, but I have repeatedly noticed arguments of the form "X cannot have Y, because ". I wanna call it "fallacy of reduction" maybe: the idea that because a trait can be explained with a process, that this proves the trait absent.

(Ie. in this case, "LLMs cannot think, because they just predict tokens." Yes, inasmuch as they think, they do so by predicting tokens. You have to actually show why predicting tokens is insufficient to produce thought.)

Re: A non-anthropomorphized view of LLMs

#212

Post by latexr »

raincole wrote: post #207
latexr wrote: post #185

Earlier quoted context omitted.

> One talking about a program's "behaviors", "actions" or "goals" doesn't mean they believe the program is sentient. Except that is exactly what we’re seeing with LLMs. People believing exactly that.

Perhaps a few mentally unhinged people do. A bit of anecdote: last year I hung out with a bunch of old classmates that I hadn't seen for quite a while. None of them works in tech. Surprisingly to me, all of them have ChatGPT installed on their phones. And unsurprisingly to me, none of them treated it like an actual intelligence. That makes me wonder where those who think ChatGPT is sentient come from. (It's a bit wor…

> Perhaps a few mentally unhinged people do.

I think it’s more than a few and it’s still rising, and therein lies the issue.

Which is why it is paramount to talk about this now, when we may still turn the tide. LLMs can be useful, but it’s important to have the right mental model, understanding, expectations, and attitude towards them.

Re: A non-anthropomorphized view of LLMs

#214

Post by gugagore »

gugagore wrote: post #199
barrkel wrote: post #175

Earlier quoted context omitted.

Sure it's state. It logically evolves stepwise per token generation. It encapsulates the LLM's understanding of the text so far so it can predict the next token. That it is merely a fixed function of other data isn't interesting or useful to say. All deterministic programs are fixed functions of program code, inputs and computation steps, but we don't say that they don't have state. It's not a useful distinction for…

I'll say it once more: I think it is useful to distinguish between autoregressive and recurrent architectures. A clear way to make that distinction is to agree that the recurrent architecture has hidden state, while the autoregressive one does not. A recurrent model has some point in a space that "encapsulates its understanding". This space is "hidden" in the sense that it doesn't correspond to text tokens or any oth…

I'll also point out what is most important part from your original message:

> LLMs have hidden state not necessarily directly reflected in the tokens being produced, and it is possible for LLMs to output tokens in opposition to this hidden state to achieve longer-term outcomes (or predictions, if you prefer).

But what does it mean for an LLM to output a token in opposition to its hidden state? If there's a longer-term goal, it either needs to be verbalized in the output stream, or somehow reconstructed from the prompt on each token.

There’s some work (a link would be great) that disentangles whether chain-of-thought helps because it gives the model more FLOPs to process, or because it makes its subgoals explicit—e.g., by outputting “Okay, let’s reason through this step by step...” versus just "...." What they find is that even placeholder tokens like "..." can help.

That seems to imply some notion of evolving hidden state! I see how that comes in!

But crucially, in autoregressive models, this state isn’t persisted across time. Each token is generated afresh, based only on the visible history. The model’s internal (hidden) layers are certainly rich and structured and "non verbal".

But any nefarious intention or conclusion has to be arrived at on every forward pass.

Re: A non-anthropomorphized view of LLMs

#215

Post by psychoslave »

Al-Khwarizmi wrote: post #117

I have the technical knowledge to know how LLMs work, but I still find it pointless to not anthropomorphize, at least to an extent. The language of "generator that stochastically produces the next word" is just not very useful when you're talking about, e.g., an LLM that is answering complex world modeling questions or generating a creative story. It's at the wrong level of abstraction, just as if you were discussing…

LLM are as far away from your description as ASM is from the underlying architecture. The anthropomorohic abstraction is as nice as any metaphore which fall apart the very moment you put a foot outside what it allows to shallowoly grab. But some people will put far more amount to push force a confortable analogy rather than admit it has some limits and to use the new tool in a more relevant way you have to move away from this confort zone.

Re: A non-anthropomorphized view of LLMs

#217

Post by psychoslave »

flir wrote: post #154
cmenge wrote: post #126

Earlier quoted context omitted.

I kinda agree with both of you. It might be a required abstraction, but it's a leaky one. Long before LLMs, I would talk about classes / functions / modules like "it then does this, decides the epsilon is too low, chops it up and adds it to the list". The difference I guess it was only to a technical crowd and nobody would mistake this for anything it wasn't. Everybody know that "it" didn't "decide" anything. With AI…

Agreeing with you, this is a "can a submarine swim" problem IMO. We need a new word for what LLMs are doing. Calling it "thinking" is stretching the word to breaking point, but "selecting the next word based on a complex statistical model" doesn't begin to capture what they're capable of. Maybe it's cog-nition (emphasis on the cog).

It does some kind of automatic inference (AI), and that's it.

Re: A non-anthropomorphized view of LLMs

#218

Post by psychoslave »

whilenot-dev wrote: post #157
flir wrote: post #154

Earlier quoted context omitted.

Agreeing with you, this is a "can a submarine swim" problem IMO. We need a new word for what LLMs are doing. Calling it "thinking" is stretching the word to breaking point, but "selecting the next word based on a complex statistical model" doesn't begin to capture what they're capable of. Maybe it's cog-nition (emphasis on the cog).

"predirence" -> prediction meets inference and it sounds a bit like preference

Except -ence is a regular morph, and you would rather suffix it to predict(at)-.

And prediction is already an hyponym of inference. Why not just use inference then?

Re: A non-anthropomorphized view of LLMs

#219

Post by Xss3 »

mercer wrote: post #163
Al-Khwarizmi wrote: post #117

I have the technical knowledge to know how LLMs work, but I still find it pointless to not anthropomorphize, at least to an extent. The language of "generator that stochastically produces the next word" is just not very useful when you're talking about, e.g., an LLM that is answering complex world modeling questions or generating a creative story. It's at the wrong level of abstraction, just as if you were discussing…

I get the impression after using language models for quite a while that perhaps the one thing that is riskiest to anthropomorphise is the conversational UI that has become the default for many people. A lot of the issues I'd have when 'pretending' to have a conversation are much less so when I either keep things to a single Q/A pairing, or at the very least heavily edit/prune the conversation history. Based on my und…

This. If an AI spits out incorrect code then i immediately create a new chat and reprompt with additional context.

'Dont use regex for this task' is a common addition for the new chat. Why does AI love regex for simple string operations?

Re: A non-anthropomorphized view of LLMs

#220

Post by psychoslave »

tliltocatl wrote: post #196
amelius wrote: post #140

Earlier quoted context omitted.

I don't agree. Most LLMs have been trained on human data, so it is best to talk about these models in a human way.

Anthropomorphising implicitly assumes motivation, goals and values. That's what the core of anthropomorphism is - attempting to explain behavior of a complex system in teleological terms. And prompt escapes make it clear LLMs doesn't have any teleological agency yet. Whenever their course of action is, it is to easy to steer them of. Try to do it with a sufficiently motivated human.

>. Try to do it with a sufficiently motivated human.

That's what they call marketing, propaganda or brain washing, acculturation , education depending on who you ask and at which scale you operate, apparently.

Post reply on HN