Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

arxiv.org

Post reply on HN
Print view View original

211–220 of 296 posts

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#211

Post by hypfer »

DonHopkins wrote: post #208
hypfer wrote: post #186

Earlier quoted context omitted.

I've been going through comment revisions here, and none of them really cut it, but all of them wasted time. But regardless, you're using this as an insult. I did not. There is no debate to be had here. Just a bad faith shouting match

[dead]

I saw that your reply got filtered by the system, googled your name, thought "huh", then tried vouching, but to no avail.

But either way. I think you might be operating in the mindset of a high-trust internet that no longer exists. You do not engage with sea lions, concern trolls, grifters, marketing people and the like. You shut them down.

inb4 "who are you to decide that?" - "Me. I am me."

That said, I appreciate what you did for computing at large, but you're missing the point here. This is (unfortunately) not a good faith debate club. The Internet as we know and love it is dead.

___

Okay, I've spent some time reading through the dead comments and..

Look, man, as you can see, I have (a somewhat dwindling) respect for your life's work. But this ain't it. This is not about Turing or really whatever is in your academic rebuttal there.

This is about the fact that the AI companies are making people sick to push their stock value by telling stories about how AI is sentient and all. Real people are having breakdowns due to this and real families get hurt.

The academic debate about what counts as sentience is irrelevant here because it was never in the room. It was just enough in the room to bait someone like you (who wants to have good faith debates) and do.. well. whatever happened here.

You're protecting grifters that hurt us all. Not because you'd want to, but because they're amazing at making us work against each other.

Please let us not do that. Let's not get weaponized like that.

___

Fwiw, I would love to go back to the internet of your time. But we can't.

Because it doesn't exist anymore, because the pathogens adapted to the conditions there.

At least that is my read of reality as of now. I might be wrong, but I don't see how that might come to be.

I think you might've had a similar response as I had as to "there's a troll/something bad that must be destroyed so that the space can flourish"?

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#212

Post by nkrisc »

throw849499 wrote: post #202
mdp2021 wrote: post #183

Earlier quoted context omitted.

> Actual real people believe The problem with rapists is mental development, not "those sexy tables and chairs and pots and everything". It's not that if people do not understand metaphors we should stop using them. It 's not that if people faint when they hear words we should stop using them. It is not that the free unduly associations in a large shared by many "collective subconscious" should hinder us...

[flagged]

I think they do it because they know they can get away with it since it would be hard to prove in cases where they are already in an intimate relationship with their victim. For a long time it wasn’t even illegal if they were married to their victim.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#213

Post by DonHopkins »

hypfer wrote: post #189
Dilettante_ wrote: post #159

Earlier quoted context omitted.

Nice Unthinkable Topic you have there.

As I said, context. In the right space with real people, worthy of a debate. On HN? No. Not like this. Not here. Not without filtering the participants for real human beings.

[flagged]

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#214

Post by azan_ »

kelnos wrote: post #147
kgeist wrote: post #28

Is anthropomorphizing a real problem? From what I know, none of the serious LLM researchers believe it has anything to do with human reasoning, apart from Anthropic with their click-baity terminology like "LLM biology". It's just a metaphor. "Reasoning tokens" is simpler to say than "learned prompt augmentation tokens". I used to (and still do) anthropomorphize things long before LLMs, and I've seen my colleagues do…

> Is anthropomorphizing a real problem? Yes. Actual real people believe that LLMs are actual, thinking, intelligences, perhaps even with consciousness. Your bit with MySQL is harmless because it's obvious that a database isn't a sentient lifeform. But LLMs can look like they're the real deal, and people believe it is. Using terminology like "thinking" and "reasoning" to describe what they do only reinforces this. Hav…

I'd like to hear definition of intelligence that would classify median person as intelligent but LLMs not (and then I'd wait few months for another moving of the goalpost and new definition). Bonus points if it's not recursive e.g. feature that only biological brains can possess.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#215

Post by user43928 »

wizzwizz4 wrote: post #9
Jeff_Brown wrote: post #5

Earlier quoted context omitted.

By itself, "aha" carries no insight, but the insight is probably stated immediately after it. In that case the aha is semantically useful, by identifying the insight it is near.

> but the insight is probably stated immediately after it. If the intermediate tokens represent reasoning or thought, you would expect "aha" to occur after the thoughts that led to the realisation, including the thoughts encoding the explanation: they don't have any other state. There is no reason to draw the conclusion you've drawn. Furthermore, what LLMs are doing isn't thought.

I don't get your argument.

Let's say that the forward pass that selected "Aha" produces activations that indicate a wrong assumption, and a plausible explanation.

It puts learned projections of the activation into the KV Cache and outputs Aha.

Both the cached projections and the current Aha token can now influence further activations in an additional Forward pass that the Aha bought the model.

At least that's how I thought it works.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#216

Post by perrygeo »

Coming from a more traditional stats/ML background, I try to view "thinking" traces as a way to explore the search space without getting caught in a local maximum.

A better analogy for me is annealing; you can't cool metal down instantly or the result is brittle. You must cool down gradually, which allows the molecules to arrange into more durable structures. Random but controlled.

In the same way, thinking traces are testing out all sorts of novel connections between tokens ("But wait..", "Actually,.."). Like a highly divergent branching mind map that gets pruned over time, rather than settling directly into the initial answer.

Now consider human cognition. We're constantly diverging, daydreaming, playing "what if" scenarios and measuring up those ideas against our internal objective functions (proxy for reality) to see which ideas stick. Not too dissimilar. But it's hard to call what we do "thinking" either - it's the default mode network wandering.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#217

Post by naasking »

kelnos wrote: post #147
kgeist wrote: post #28

Is anthropomorphizing a real problem? From what I know, none of the serious LLM researchers believe it has anything to do with human reasoning, apart from Anthropic with their click-baity terminology like "LLM biology". It's just a metaphor. "Reasoning tokens" is simpler to say than "learned prompt augmentation tokens". I used to (and still do) anthropomorphize things long before LLMs, and I've seen my colleagues do…

> Is anthropomorphizing a real problem? Yes. Actual real people believe that LLMs are actual, thinking, intelligences, perhaps even with consciousness. Your bit with MySQL is harmless because it's obvious that a database isn't a sentient lifeform. But LLMs can look like they're the real deal, and people believe it is. Using terminology like "thinking" and "reasoning" to describe what they do only reinforces this. Hav…

> Yes. Actual real people believe that LLMs are actual, thinking, intelligences, perhaps even with consciousness.

Yes, because it's possible. They don't have human consciousness/thought/intelligence, but it's entirely possible they have some form of consciousness, some form of real thought and some form of intelligence.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#218

Post by mdp2021 »

phoghed wrote: post #207
mdp2021 wrote: post #201

Earlier quoted context omitted.

> why would you be so concerned when it’s uttered When I am told "beware of the xyphucymmon", I do beware, though not of the xyphucymmon .

And this means nothing, yet I’m not complaining endlessly about it on social media, or harassing my coworkers about it. Curious

My post «When I am told "beware of the xyphucymmon", I do beware, though not of the xyphucymmon» had meaning, which should be pretty clear. The reply remains: debate is not performance art in which you convey "through the medium of dance and howls". So, yes, communication has obligations for the locutor.

So, yes, imperfect speech is a problem - because societies are not performance art arenas.

--

@Razengan: speaking as myself a vocal critic of the downvoting system here, do notice that the post you replied to had a substantive reply at the time of downvote - mine. That is sufficent to justify the downvote - of a post that was not «perfectly fine». It somehow said that people should be free to say whatever they want and there are obvious reasons why we do not agree.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#219

Post by sega_sai »

I don't find this take useful. We have little understanding of what conscience is, and how human train of thought really works. The modern LLMs IMO resemble more and more Chinese room problem. Maybe we don't like the mechanistic linear-algebra-based steps involved in the production of the output, but the end result is closer and closer to people's output. IMO this is very natural to start anthropomorphizing that.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#220

Post by Lorkki »

nostrademons wrote: post #53
JohnMakin wrote: post #48

Earlier quoted context omitted.

Do you? I’m not sure what you’re getting at.

Not GP, but one of the challenges with debating whether LLMs are "conscious" is that we don't even really know what it means for a human to be "conscious", or even if consciousness is experienced by other humans the same way it is for ourselves. What we do know: neurons carry electrical impulses across their synapses to trigger other neurons to fire, and more frequently used synapses are strengthened while infrequent…

> What we do know: neurons carry electrical impulses across their synapses to trigger other neurons to fire, and more frequently used synapses are strengthened while infrequently used ones are pruned. This is not all that dissimilar to how a multi-layer perceptron is trained

ANNs have been inspired by biological processes, but in practice you have to squint very tightly to see the similarity. Biological neurons are multiple orders of magnitude more connected than the nodes in an ANN, plastic in terms of their connectedness and continuously learning, and their activations are also affected in complex ways by the levels of various transmitter molecules in the brain.

Post reply on HN