Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

arxiv.org

Post reply on HN
Print view View original

11–20 of 296 posts

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#11

Post by FloorEgg »

basedpolymer wrote: post #8

The anthropomorphization of LLMs should be discouraged as much as possible. It perpetuates bad practices and encourages the use of these bots for tasks they are not intended for (particularly as chatbots). Thinking traces should be treated as black boxes. There is no point in reading them. Only the LLMs’ conclusions are relevant. This is particularly true of Opus 5, which employs reasoning that seems highly questiona…

Sometimes I monitor thinking traces for misunderstandings (missing context / bad assumptions). If it's going to go off on a ~20 min task and I can catch it's going in the wrong direction in the first minute I save a lot of tokens and wasted time. I don't monitor the whole thing, mostly just the first bit to see if there was a gap or misalignment in intention.

As an aside, anthropomorphization has nothing to do with my motivations.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#12

Post by abitmoa »

paimapi wrote: post #7
Jeff_Brown wrote: post #5

Earlier quoted context omitted.

By itself, "aha" carries no insight, but the insight is probably stated immediately after it. In that case the aha is semantically useful, by identifying the insight it is near.

it's a rhetorical heuristic that a writer should know to use when directing a reader to a declarative that they want them to pay attention to, usually because it's a non-obvious or roundabout insight when utilized by AI, it's a probabilistic output and it's variable whether or not that rhetorical trick is useful. it also pushes a non-skeptical reader to focus too much on the following text or even to believe that the…

It amounts to noise overall, but it has further unwanted and potentially misleading 'properties'. I think it's rather sobering to see how much bandwidth is still being wasted.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#13

Post by c0_0p_ »

cyanydeez wrote: post #6
clhodapp wrote: post #4

Seems like they are closer to scratch than reasoning... Generating some scratch to draw from helps make it easier to compute the real answer.

I assume theyre searching the local gradient to see if theres a better descent before proceeding.

I don't think there's anything like that going on. They just word vomit into a secondary area, and then there is an internal prompt that says "clean this up and summarize for the user".

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#15

Post by eigenspace »

cyanydeez wrote: post #6
clhodapp wrote: post #4

Seems like they are closer to scratch than reasoning... Generating some scratch to draw from helps make it easier to compute the real answer.

I assume theyre searching the local gradient to see if theres a better descent before proceeding.

LLMs dont do gradient descent to generate tokens.

They are trained by gradient descent, but inference doesnt involve it.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#16

Post by qarl2 »

basedpolymer wrote: post #8

The anthropomorphization of LLMs should be discouraged as much as possible. It perpetuates bad practices and encourages the use of these bots for tasks they are not intended for (particularly as chatbots). Thinking traces should be treated as black boxes. There is no point in reading them. Only the LLMs’ conclusions are relevant. This is particularly true of Opus 5, which employs reasoning that seems highly questiona…

> The anthropomorphization of LLMs should be discouraged as much as possible.

And yet, they have extensive human-like behavior. If you treat them nicely or encourage them, they perform better.

Ignoring that human-like behavior is wrong headed.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#17

Post by forgotTheLast »

clhodapp wrote: post #4

Seems like they are closer to scratch than reasoning... Generating some scratch to draw from helps make it easier to compute the real answer.

That's my personal theory too. The model is stuffing its own context with vaguely related tokens, which helps the attention heads retrieve the right tokens.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#18

Post by andai »

qarl2 wrote: post #16
basedpolymer wrote: post #8

The anthropomorphization of LLMs should be discouraged as much as possible. It perpetuates bad practices and encourages the use of these bots for tasks they are not intended for (particularly as chatbots). Thinking traces should be treated as black boxes. There is no point in reading them. Only the LLMs’ conclusions are relevant. This is particularly true of Opus 5, which employs reasoning that seems highly questiona…

> The anthropomorphization of LLMs should be discouraged as much as possible. And yet, they have extensive human-like behavior. If you treat them nicely or encourage them, they perform better. Ignoring that human-like behavior is wrong headed.

A while back I made an "OpenClaw in 50 lines" by just wrapping Claude Code in a Telegram bot.

I asked it for the weather. "I don't know that. I'm just a programmer."

I added "believe in yourself, you can do anything" to sysprompt, suddenly it had the confidence to Google the weather...

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#19

Post by thaanpaa »

qarl2 wrote: post #16
basedpolymer wrote: post #8

The anthropomorphization of LLMs should be discouraged as much as possible. It perpetuates bad practices and encourages the use of these bots for tasks they are not intended for (particularly as chatbots). Thinking traces should be treated as black boxes. There is no point in reading them. Only the LLMs’ conclusions are relevant. This is particularly true of Opus 5, which employs reasoning that seems highly questiona…

> The anthropomorphization of LLMs should be discouraged as much as possible. And yet, they have extensive human-like behavior. If you treat them nicely or encourage them, they perform better. Ignoring that human-like behavior is wrong headed.

That's not a consequence of an LLM. It's a consequence of the training data. In fact, I would argue that the latest models aren't nearly as sensitive to the tone of input anymore. It's an issue that has been addressed by better curating training data.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#20

Post by fedpost »

qarl2 wrote: post #16
basedpolymer wrote: post #8

The anthropomorphization of LLMs should be discouraged as much as possible. It perpetuates bad practices and encourages the use of these bots for tasks they are not intended for (particularly as chatbots). Thinking traces should be treated as black boxes. There is no point in reading them. Only the LLMs’ conclusions are relevant. This is particularly true of Opus 5, which employs reasoning that seems highly questiona…

> The anthropomorphization of LLMs should be discouraged as much as possible. And yet, they have extensive human-like behavior. If you treat them nicely or encourage them, they perform better. Ignoring that human-like behavior is wrong headed.

I think you're ending that train of thought too early. Why does this occur?

Well... We can hypothesize that these things are largely trained on internet dialogue so there's probably some correlation between threads where people are not flaming each other and the quality of the replies. They're just statistical engines so anything you can do to raise the odds of a helpful next token...

I'm essentially just making shit up here, maybe it's right, maybe it isn't, but rather than saying "it's human and we should treat it so" we're trying to get to the ground truth of how it works.

Post reply on HN