Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A non-anthropomorphized view of LLMs

addxorrol.blogspot.com

Post reply on HN
Print view View original

311–320 of 432 posts

Re: A non-anthropomorphized view of LLMs

#311

Post by godshatter »

barrkel wrote: post #174
gpm wrote: post #86

Earlier quoted context omitted.

The LLM does not "have" a plan. Arguably there's reason to believe it comes up with a plan when it is computing token propabilities, but it does not store it between tokens. I.e. it doesn't possess or "have" it. It simply comes up with a plan, emits a token, and entirely throws all its intermediate thoughts (including any plan) to start again from scratch on the next token.

I believe saying the LLM has a plan is a useful anthropomorphism for the fact that it does have hidden state that predicts future tokens, and this state conditions the tokens it produces earlier in the stream.

Are the devs behind the models adding their own state somehow? Do they have code that figures out a plan and use the LLM on pieces of it and stitch them together? If they do, then there is a plan, it's just not output from a magical black box. Unless they are using a neural net to figure out what the plan should be first, I guess.

I know nothing about how things work at that level, so these might not even be reasonable questions.

Re: A non-anthropomorphized view of LLMs

#312

Post by variadix »

losvedir wrote: post #265
grey-area wrote: post #120

Earlier quoted context omitted.

On the contrary, anthropomorphism IMO is the main problem with narratives around LLMs - people are genuinely talking about them thinking and reasoning when they are doing nothing of that sort (actively encouraged by the companies selling them) and it is completely distorting discussions on their use and perceptions of their utility.

Well "reasoning" refers to Chain-of-Thought and if you look at the generated prompts it's not hard to see why it's called that. That said, it's fascinating to me that it works (and empirically, it does work; a reasoning model generating tens of thousands of tokens while working out the problem does produce better results). I wish I knew why. A priori I wouldn't have expected it, since there's no new input. That means…

Using more tokens = more compute to use for a given problem. I think most of the benefit of CoT has more to do with autoregressive models being unable to “think ahead” and revise their output, and less to do with actual reasoning. The fact that an LLM can have incorrect reasoning in its CoT and still produce the right answer, or that it can “lie” in its CoT to avoid being detected as cheating on RL tasks, makes me believe that the semantic content of CoT is an illusion, and that the improved performance is from being able to explore and revise in some internal space using more compute before producing a final output.

Re: A non-anthropomorphized view of LLMs

#313

Post by Terr_ »

losvedir wrote: post #265
grey-area wrote: post #120

Earlier quoted context omitted.

On the contrary, anthropomorphism IMO is the main problem with narratives around LLMs - people are genuinely talking about them thinking and reasoning when they are doing nothing of that sort (actively encouraged by the companies selling them) and it is completely distorting discussions on their use and perceptions of their utility.

Well "reasoning" refers to Chain-of-Thought and if you look at the generated prompts it's not hard to see why it's called that. That said, it's fascinating to me that it works (and empirically, it does work; a reasoning model generating tens of thousands of tokens while working out the problem does produce better results). I wish I knew why. A priori I wouldn't have expected it, since there's no new input. That means…

I like this mental-model, which rests heavily on the "be careful not to anthropomorphize" approach:

It was already common to use a document extender (LLM) against a hidden document, which resembles a movie or theater play where a character named User is interrogating a character named Bot.

Chain-of-thought switches the movie/script style to film noir, where the [Detective] Bot character has additional content which is not actually "spoken" at the User character. The extra words in the script add a certain kind of metaphorical inertia.

Re: A non-anthropomorphized view of LLMs

#314

Post by Wowfunhappy »

fenomas wrote: post #143
grey-area wrote: post #120

Earlier quoted context omitted.

On the contrary, anthropomorphism IMO is the main problem with narratives around LLMs - people are genuinely talking about them thinking and reasoning when they are doing nothing of that sort (actively encouraged by the companies selling them) and it is completely distorting discussions on their use and perceptions of their utility.

When I see these debates it's always the other way around - one person speaks colloquially about an LLM's behavior, and then somebody else jumps on them for supposedly believing the model is conscious, just because the speaker said "the model thinks.." or "the model knows.." or whatever. To be honest the impression I've gotten is that some people are just very interested in talking about not anthropomorphizing AI, an…

As I write this, Claude Code is currently opening and closing various media files on my computer. Sometimes it plays the file for a few seconds before closing it, sometimes it starts playback and then seeks to a different position, sometimes it fast forwards or rewinds, etc.

I asked Claude to write a E-AC3 audio component so I can play videos with E-AC3 audio in the old version of QuickTime I really like using. Claude's decoder includes the ability to write debug output to a log file, so Claude is studying how QuickTime and the component interact, and it's controlling QuickTime via Applescript.

Sometimes QuickTime crashes, because this ancient API has its roots in the classic Mac OS days and is not exactly good. Claude reads the crash logs on its own—it knows where they are—and continues on its way. I'm just sitting back and trying to do other things while Claude works, although it's a little distracting that something else is using my computer at the same time.

I really don't want to anthropomorphize these programs, but it's just so hard when it's acting so much like a person...

Re: A non-anthropomorphized view of LLMs

#315

Post by scarface_74 »

Tijdreiziger wrote: post #306
scarface_74 wrote: post #244

Earlier quoted context omitted.

Wait until a conversation about “serverless” comes up and someone says there is no such thing because there are servers somewhere as if everyone - especially on HN -doesn’t already know that.

Why would everyone know that? Not everyone has experience in sysops, especially not beginners. E.g. when I first started learning webdev, I didn’t think about ‘servers’. I just knew that if I uploaded my HTML/PHP files to my shared web host, then they appeared online. It was only much later that I realized that shared webhosting is ‘just’ an abstraction over Linux/Apache (after all, I first had to learn about those t…

I am saying that most people who come on HN and say “there is no such thing as serverless and there are servers somewhere” think they are sounding smart when they are adding nothing to the conversation.

I’m sure you knew that your code was running on computers somewhere even when you first started and wasn’t running in a literal “cloud”.

It’s about as tiring as people on HN who know just a little about LLMs thinking they are sounding smart when they say they are just advanced autocomplete. Both responses are just as unproductive

Re: A non-anthropomorphized view of LLMs

#316

Post by Veen »

bunderbunder wrote: post #308
grey-area wrote: post #120

Earlier quoted context omitted.

On the contrary, anthropomorphism IMO is the main problem with narratives around LLMs - people are genuinely talking about them thinking and reasoning when they are doing nothing of that sort (actively encouraged by the companies selling them) and it is completely distorting discussions on their use and perceptions of their utility.

"All models are wrong, but some models are useful," is the principle I have been using to decide when to go with an anthropomorphic explanation. In other words, no, they never accurately describe what the LLM is actually doing. But sometimes drawing an analogy to human behavior is the most effective way to pump others' intuition about a particular LLM behavior. The trick is making sure that your audience understands…

Some models are useful in some contexts but wrong enough to be harmful in others.

Re: A non-anthropomorphized view of LLMs

#317

Post by lawlessone »

Al-Khwarizmi wrote: post #117

I have the technical knowledge to know how LLMs work, but I still find it pointless to not anthropomorphize, at least to an extent. The language of "generator that stochastically produces the next word" is just not very useful when you're talking about, e.g., an LLM that is answering complex world modeling questions or generating a creative story. It's at the wrong level of abstraction, just as if you were discussing…

One thing i find i keep forgetting is that asking an LLM why it makes a particular decision is almost pointless.

It's reply isn't actually going to be why i did a thing. It's reply is going to be whatever is the most probably string of words that fit as a reason.

Re: A non-anthropomorphized view of LLMs

#318

Post by j0057 »

LeonardoTolstoy wrote: post #192
flir wrote: post #154

Earlier quoted context omitted.

Agreeing with you, this is a "can a submarine swim" problem IMO. We need a new word for what LLMs are doing. Calling it "thinking" is stretching the word to breaking point, but "selecting the next word based on a complex statistical model" doesn't begin to capture what they're capable of. Maybe it's cog-nition (emphasis on the cog).

What does a submarine do? Submarine? I suppose you "drive" a submarine which is getting to the idea: submarines don't swim because ultimately they are "driven"? I guess the issue is we don't make up a new word for what submarines do, we just don't use human words. I think the above poster gets a little distracted by suggesting the models are creative which itself is disputed. Perhaps a better term, like above, would…

A submarine is a boat and boats sail.

Re: A non-anthropomorphized view of LLMs

#319

Post by labrador »

I find it useful to pretend that I'm talking to a person while brainstorming because then the conversation flows naturally. But I maintain awareness that I'm pretending, much like Tom Hanks talking to Wilson the volleyball in the movie Castaway. The suspension of disbelief serves a purpose, but I never confuse the volleyball for a real person.

Re: A non-anthropomorphized view of LLMs

#320

Post by Tijdreiziger »

scarface_74 wrote: post #315
Tijdreiziger wrote: post #306

Earlier quoted context omitted.

Why would everyone know that? Not everyone has experience in sysops, especially not beginners. E.g. when I first started learning webdev, I didn’t think about ‘servers’. I just knew that if I uploaded my HTML/PHP files to my shared web host, then they appeared online. It was only much later that I realized that shared webhosting is ‘just’ an abstraction over Linux/Apache (after all, I first had to learn about those t…

I am saying that most people who come on HN and say “there is no such thing as serverless and there are servers somewhere” think they are sounding smart when they are adding nothing to the conversation. I’m sure you knew that your code was running on computers somewhere even when you first started and wasn’t running in a literal “cloud”. It’s about as tiring as people on HN who know just a little about LLMs thinking…

> I’m sure you knew that your code was running on computers somewhere even when you first started and wasn’t running in a literal “cloud”.

Meh, I just knew that the browser would display HTML if I wrote it, and that uploading the HTML files made them available on my domain. I didn’t really think about where the files went, specifically.

Try asking an average high school kid how cloud storage works. I doubt you’ll get any further than ‘I make files on my Google Docs and then they are saved there’. This is one step short of ‘well, the files must be on some system in some data center’.

I really disagree that “people who come on HN and say “there is no such thing as serverless and there are servers somewhere” think they are sounding smart when they are adding nothing to the conversation.” On the contrary, it’s an invitation to beginning coders to think about what the ‘serverless’ abstraction actually means.

Post reply on HN