Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

arxiv.org

Post reply on HN
Print view View original

81–90 of 296 posts

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#81

Post by roughly »

Jtarii wrote: post #77
JohnMakin wrote: post #35

Earlier quoted context omitted.

Because plenty of people, even ones that should know better, really believe it's a conscious, thinking entity, not just some turn of phrase. I have a coworker that spends at least 10 hours a week arguing with his like you would with a conscious person. I've gently tried to explain it's like arguing with your compiler for giving you an incoherent error message - it's pointless. It doesn't understand, it can't understa…

>Because plenty of people, even ones that should know better, really believe it's a conscious, thinking entity You need evidence to make the positive claim that LLMs do not posses any form of consciousness.

Well, our current set of evidence is that it’s a mechanistic mechanical algorithm with an RNG embedded in it and we can both get it to repeatedly produce the same output for the same input and also get it to repeatedly do absolutely nothing at all, which are not characteristics we usually find in objects evincing consciousness.

LLMs bear absolutely none of the traits we’ve come to recognize as the external hallmarks of consciousness in biological organisms, nor anything that would seem analogous in a non-biological substrate.

That said, we don’t have a rigorous definition of consciousness that includes the actual phenomenology of consciousness, so I daresay if you’re going to go around asserting the LLM is conscious despite all existing evidence to the contrary, I think the impetus is on you to define some version of consciousness that isn’t also satisfied by a book or a movie.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#82

Post by tavavex »

alexey-salmin wrote: post #45
JohnMakin wrote: post #35

Earlier quoted context omitted.

Because plenty of people, even ones that should know better, really believe it's a conscious, thinking entity, not just some turn of phrase. I have a coworker that spends at least 10 hours a week arguing with his like you would with a conscious person. I've gently tried to explain it's like arguing with your compiler for giving you an incoherent error message - it's pointless. It doesn't understand, it can't understa…

> It doesn't understand, it can't understand, and even if it could, you arguing with it isn't going to make it "learn" or act differently. I know people who are like that too. I'm not sure anthropomorphizing is a problem. Seeing analogies everywhere is an innate human trait, sometimes it can be harmful but more often it's useful.

Anthropomorphizing is a problem when you're talking about treating something that's not living as if it were. Using humanizing language invites discussions of things like the rights and feelings of an algorithm. A judge that is misled by the application of human-centric language to an algorithm can lead to some terrible outcomes. Not everyone is an LLM expert and the language people use leads to them treating LLMs like actual, real humans. That is terrifying.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#83

Post by tavavex »

Jtarii wrote: post #77
JohnMakin wrote: post #35

Earlier quoted context omitted.

Because plenty of people, even ones that should know better, really believe it's a conscious, thinking entity, not just some turn of phrase. I have a coworker that spends at least 10 hours a week arguing with his like you would with a conscious person. I've gently tried to explain it's like arguing with your compiler for giving you an incoherent error message - it's pointless. It doesn't understand, it can't understa…

>Because plenty of people, even ones that should know better, really believe it's a conscious, thinking entity You need evidence to make the positive claim that LLMs do not posses any form of consciousness.

Consciousness is a slippery word that is notoriously difficult to debate over. But often, people use 'consciousness' as a shortcut or a familiar word to describe a more complex idea. The point they're driving across isn't about the precise definition of the word 'consciousness', but about people treating LLMs as if they were actual human beings, assigning them all the traits and behaviors they would expect of a human.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#84

Post by qarl2 »

fedpost wrote: post #65
qarl2 wrote: post #34

Earlier quoted context omitted.

Did I say "it's human and we should treat it so"? Sheesh. Yes, I agree with you entirely. I'm merely pointing out that ignoring this behavior is dumb, too. And probably not rationally based. Leads people to make crazy jumps. :)

Yeah sorry, I read too far into your position. There's a certain faction within these AI discussions that wants to over-anthropomorphize the LLMs in kind of a borderline spiritual way.

Oh that's a shame - I hadn't seen that, but I can believe it.

The philosophers who study these things have been clear for a long time - we can never know what it feels like to be in a digital brain. Or any brain for that matter. When push comes to shove we all might be phantoms in some guy's dream.

Don't know + can't know. I think that was the real point of the Turing test. Not: this means it's conscious. Just: this is the best we can ever hope to do.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#85

Post by FuckButtons »

Jtarii wrote: post #77
JohnMakin wrote: post #35

Earlier quoted context omitted.

Because plenty of people, even ones that should know better, really believe it's a conscious, thinking entity, not just some turn of phrase. I have a coworker that spends at least 10 hours a week arguing with his like you would with a conscious person. I've gently tried to explain it's like arguing with your compiler for giving you an incoherent error message - it's pointless. It doesn't understand, it can't understa…

>Because plenty of people, even ones that should know better, really believe it's a conscious, thinking entity You need evidence to make the positive claim that LLMs do not posses any form of consciousness.

I would argue, that the null hypothesis is that it is not, and that anyone claiming that there is a mote of consciousness are the ones with the burden of proof.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#86

Post by yreg »

FuckButtons wrote: post #85
Jtarii wrote: post #77

Earlier quoted context omitted.

>Because plenty of people, even ones that should know better, really believe it's a conscious, thinking entity You need evidence to make the positive claim that LLMs do not posses any form of consciousness.

I would argue, that the null hypothesis is that it is not, and that anyone claiming that there is a mote of consciousness are the ones with the burden of proof.

The null hypothesis is that we don't know jack shit about consciousness. Any claim of certainty seems extraordinary to me and I want to hear the evidence.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#87

Post by clhodapp »

throw310822 wrote: post #80
clhodapp wrote: post #4

Seems like they are closer to scratch than reasoning... Generating some scratch to draw from helps make it easier to compute the real answer.

It's also interesting because in humans the existence of "Aha!" moments that are not preceded by or are only loosely related to a chain of thought is taken as the proof of the fundamental mystery and irreproducibility of human intelligence. Now the same argument is made to deny that LLMs actually think. Go figure.

It feels apparent to me that LLMs don't do what is colloquially thought of as thinking.

What is less apparent is that humans do.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#88

Post by JohnMakin »

red75prime wrote: post #59
JohnMakin wrote: post #35

Earlier quoted context omitted.

Because plenty of people, even ones that should know better, really believe it's a conscious, thinking entity, not just some turn of phrase. I have a coworker that spends at least 10 hours a week arguing with his like you would with a conscious person. I've gently tried to explain it's like arguing with your compiler for giving you an incoherent error message - it's pointless. It doesn't understand, it can't understa…

> if it could, you arguing with it isn't going to make it "learn" or act differently. Are you talking about a specific harness that doesn't have context retention mechanisms? For example, ChatGPT with disabled memory feature? Or in general where "it" is a fixed-weights network? The latter is trivially true, of course.

Even claude with “memory” enabled isn’t really “remembering” anything. It just injects it into the context and you hope it happens to find it relevant in its attention mechanisms, and then remembers to actually act on it. Anthropic’s own documentation states claude can and will ignore/truncate these. It’s a context trick, nothing approaching actual “memory,” and in fact, arguing with it will make a bunch of memory files, sometimes contradictory, and clutter up the context and act even worse.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#89

Post by paxys »

DoctorOetker wrote: post #61

Peculiarly vocal, where were all these people when they started calling the machines computers, anthropomorphizing them akin to the original human (most often female) computers that used to run such calculations? And how dangerous the consequences, we've been dead reckoning for 60-70 years with the wrong terminology without course correction! Where were these vocal people when the "raster-oriented ink deposition mach…

Computing is a task. Printing is a task. It’s not wrong to call both a human and a machine a “computer”, because computing (applying an algorithm to an input and producing an output) is literally what they are doing. Same way you can have human and machine diggers, cleaners, calculators and lots more.

Anthropomorphizing comes in when we attribute much more complex behaviors to them - chain of thought, reasoning, intelligence. At that point you aren't talking about mechanical concepts but claiming that these machines are exhibiting human behavior.

Post reply on HN