Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

arxiv.org

Post reply on HN
Print view View original

241–250 of 296 posts

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#241

Post by tavavex »

altmanaltman wrote: post #138
tavavex wrote: post #82

Earlier quoted context omitted.

Anthropomorphizing is a problem when you're talking about treating something that's not living as if it were. Using humanizing language invites discussions of things like the rights and feelings of an algorithm. A judge that is misled by the application of human-centric language to an algorithm can lead to some terrible outcomes. Not everyone is an LLM expert and the language people use leads to them treating LLMs li…

> Anthropomorphizing is a problem when you're talking about treating something that's not living as if it were. I mean, that is the entire definition of the word. And you also anthropomorphize living beings like many people genuinely attach human qualities to their pets etc. Yes, the risks are very high when it comes to chatbots in particular, especially to people who are not technically inclined. But you'll be surpr…

> I mean, that is the entire definition of the word.

Not quite - my wording there was very deliberate. By saying that it's a problem when you're treating something that's not living (not non-human!) as if it were, that excludes pets and all animals from the equation. I understand how common it is for humans to assign human qualities to other things and beings, but there is also an unspoken variable of intensity. Representing abstract concepts as humans, interpreting living things in a human-like way or traditionally referring to ships as living beings has a different degree of belief and intensity compared to implying a genuine belief that algorithms are beings that can be enslaved, like what the sibling comment to this one does.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#242

Post by roughly »

Jtarii wrote: post #199
roughly wrote: post #106

Earlier quoted context omitted.

There’s plenty. First, when we say “LLM”, what are we referring to? What is the entity that would be conscious in this case? The reason this is important is because powerful people are currently trying to use the dodge that LLMs are conscious to launder liability for their own policy choices, so the sloppy thinking and half-assed conjecture about LLM consciousness has real-world consequences, and every time you asser…

>The reason this is important is because powerful people are currently trying to use the dodge that LLMs are conscious to launder liability for their own policy choices, so the sloppy thinking and half-assed conjecture about LLM consciousness has real-world consequences I'm not going to change my beliefs or how I think about interesting questions just because its the "socially conscious" thing to do. >There’s a riche…

You're not engaging with the topic. Engaging with the topic is where one either asks questions and listens to the answer or actually seeks to increase one's knowledge on the topic. You're just saying things. That's lazy.

And, you're welcome to do what you want to do, it's your god given right to stay as ignorant as you want about any particular topic, but that comes with consequences. If you want to call that being socially conscious, sure, you do you, but if you're interested in why people keep getting annoyed at your loud proclamations of ignorance which you're trying to proffer as evidence of a curious mind, well, that's why.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#243

Post by DoctorOetker »

paxys wrote: post #89
DoctorOetker wrote: post #61

Peculiarly vocal, where were all these people when they started calling the machines computers, anthropomorphizing them akin to the original human (most often female) computers that used to run such calculations? And how dangerous the consequences, we've been dead reckoning for 60-70 years with the wrong terminology without course correction! Where were these vocal people when the "raster-oriented ink deposition mach…

Computing is a task. Printing is a task. It’s not wrong to call both a human and a machine a “computer”, because computing (applying an algorithm to an input and producing an output) is literally what they are doing. Same way you can have human and machine diggers, cleaners, calculators and lots more. Anthropomorphizing comes in when we attribute much more complex behaviors to them - chain of thought, reasoning, inte…

Making predictions is also a task.

Whenever we reason, we are not neurotically bruteforcing the possibilities (although sometimes we do, like proof by exhaustion), usually we make predictions heuristically of which assumptions or theorems apply or might apply. It's not any different for machine proof assistants...

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#244

Post by lern_too_spel »

randomImmigrant wrote: post #239
lern_too_spel wrote: post #128

Earlier quoted context omitted.

A "machine" is a term that is well defined in science. https://en.wikipedia.org/wiki/Machine

And biological machine is? Don’t get me wrong. Biology I is full of molecules that we call machines. But you’re making a broader claim, saying that biology is only this. This needs you to answer some questions: 1. Why do the machine parts in biology show such flexible application? A gear cog won’t ever moonlight as a signaling chip, but in biology you often have molecules doing double and triple duty. 2. How is the b…

> And biological machine is?

"Biological," too, is well defined. It relates to living things and their processes.

> Why do the machine parts in biology show such flexible application?

Evolution.

> A gear cog won’t ever moonlight as a signaling chip

A gear cog was purpose built for that purpose, but you will find that people often recycle parts into other systems, often in completely different roles.

> How is the biological machine able to build itself?

Protein synthesis.

> What does self assembly imply for the machines function?

The way that a machine is built has no bearing on how the machine functions. I could build the same machine using a 3d printer or a CNC router.

> Where does this machine get its inner drive?

Evolution selected for organisms that survive long enough to reproduce. Different biological systems handle this differently.

> No LLM has been found that starts outputting text unprompted.

If you give an agent a goal, it will perform actions to achieve that goal. This is just as true for artificial agents as it is for biological agents.

> Why is no manufactured machine able to do this?

Many do. Even robotic vacuum cleaners will charge themselves without human prompting.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#245

Post by sega_sai »

armada651 wrote: post #224
sega_sai wrote: post #219

I don't find this take useful. We have little understanding of what conscience is, and how human train of thought really works. The modern LLMs IMO resemble more and more Chinese room problem. Maybe we don't like the mechanistic linear-algebra-based steps involved in the production of the output, but the end result is closer and closer to people's output. IMO this is very natural to start anthropomorphizing that.

What is the value in anthropomorphizing it? Often I get the feeling that it's more about the sales pitch of selling these AI as general intelligence than it is about providing a truthful insight into how these models actually arrive at the output.

I don't think there is value per se, but it is just natural (at least in the environment I am in). I.e. when we discuss the analysis/code/ideas from one LLM or another, we describe it Claude/Gemini/Codex/etc did that and it comes like a person.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#246

Post by solid_fuel »

davidguetta wrote: post #68

Im waiting for the article called "stop desantropomorphizing llms" when everybody will finally accept they think like us, partly because maybe the intelligence is universal and partly because, well the datasets are fucking human bro

You’re going to be waiting a long time, considering they don’t think like us. LLMs don’t ‘think’ at all. They are capable of limited reasoning using the meaning and context embedded in human language. Essentially, the grammatical equivalent to a mathematical constraint solver. Nothing more.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#247

Post by wizzwizz4 »

user43928 wrote: post #215
wizzwizz4 wrote: post #9

Earlier quoted context omitted.

> but the insight is probably stated immediately after it. If the intermediate tokens represent reasoning or thought, you would expect "aha" to occur after the thoughts that led to the realisation, including the thoughts encoding the explanation: they don't have any other state. There is no reason to draw the conclusion you've drawn. Furthermore, what LLMs are doing isn't thought.

I don't get your argument. Let's say that the forward pass that selected "Aha" produces activations that indicate a wrong assumption, and a plausible explanation. It puts learned projections of the activation into the KV Cache and outputs Aha. Both the cached projections and the current Aha token can now influence further activations in an additional Forward pass that the Aha bought the model. At least that's how I t…

A cache is just a cache. I'm not sure what significance you're ascribing to it.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#248

Post by user43928 »

wizzwizz4 wrote: post #247
user43928 wrote: post #215

Earlier quoted context omitted.

I don't get your argument. Let's say that the forward pass that selected "Aha" produces activations that indicate a wrong assumption, and a plausible explanation. It puts learned projections of the activation into the KV Cache and outputs Aha. Both the cached projections and the current Aha token can now influence further activations in an additional Forward pass that the Aha bought the model. At least that's how I t…

A cache is just a cache. I'm not sure what significance you're ascribing to it.

What is put in the cache?

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#249

Post by rightbyte »

perrygeo wrote: post #216

Coming from a more traditional stats/ML background, I try to view "thinking" traces as a way to explore the search space without getting caught in a local maximum. A better analogy for me is annealing; you can't cool metal down instantly or the result is brittle. You must cool down gradually, which allows the molecules to arrange into more durable structures. Random but controlled. In the same way, thinking traces ar…

Isn't search a strange analogy for how weight terms propagate?

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#250

Post by wizzwizz4 »

user43928 wrote: post #248
wizzwizz4 wrote: post #247

Earlier quoted context omitted.

A cache is just a cache. I'm not sure what significance you're ascribing to it.

What is put in the cache?

Things that the software running the model would otherwise recompute, if not for the cache. What special meaning are you assigning to it?
Post reply on HN