Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

arxiv.org

Post reply on HN
Print view View original

231–240 of 296 posts

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#231

Post by wrsh07 »

ModernMech wrote: post #209
kelnos wrote: post #147

Earlier quoted context omitted.

> Is anthropomorphizing a real problem? Yes. Actual real people believe that LLMs are actual, thinking, intelligences, perhaps even with consciousness. Your bit with MySQL is harmless because it's obvious that a database isn't a sentient lifeform. But LLMs can look like they're the real deal, and people believe it is. Using terminology like "thinking" and "reasoning" to describe what they do only reinforces this. Hav…

I dunno, sometimes I encounter humans who lack actual, thinking, intelligence, perhaps even consciousness. I still treat them like people.

To quote Sarah Constantin:

> Humans Who Are Not Concentrating Are Not General Intelligences^

But yeah, you should still treat them with humanity. (Related: you should treat LLMs well, not because they're human but because you are^^)

^ https://srconstantin.github.io/2019/02/25/humans-who-are-not...

^^ hmm couldn't find this tweet but didn't look too hard

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#232

Post by heed »

kelnos wrote: post #147
kgeist wrote: post #28

Is anthropomorphizing a real problem? From what I know, none of the serious LLM researchers believe it has anything to do with human reasoning, apart from Anthropic with their click-baity terminology like "LLM biology". It's just a metaphor. "Reasoning tokens" is simpler to say than "learned prompt augmentation tokens". I used to (and still do) anthropomorphize things long before LLMs, and I've seen my colleagues do…

> Is anthropomorphizing a real problem? Yes. Actual real people believe that LLMs are actual, thinking, intelligences, perhaps even with consciousness. Your bit with MySQL is harmless because it's obvious that a database isn't a sentient lifeform. But LLMs can look like they're the real deal, and people believe it is. Using terminology like "thinking" and "reasoning" to describe what they do only reinforces this. Hav…

Actual real people believe that people are actual, thinking, intelligences, perhaps even with consciousness.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#233

Post by ModernMech »

wrsh07 wrote: post #231
ModernMech wrote: post #209

Earlier quoted context omitted.

I dunno, sometimes I encounter humans who lack actual, thinking, intelligence, perhaps even consciousness. I still treat them like people.

To quote Sarah Constantin: > Humans Who Are Not Concentrating Are Not General Intelligences^ But yeah, you should still treat them with humanity. (Related: you should treat LLMs well, not because they're human but because you are^^) ^ https://srconstantin.github.io/2019/02/25/humans-who-are-not... ^^ hmm couldn't find this tweet but didn't look too hard

I gotta say though, turning on caps lock does seem to get them to cut out the inane banter.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#234

Post by qsera »

wccrawford wrote: post #179
qsera wrote: post #174

Earlier quoted context omitted.

>I'm not sure anthropomorphizing is a problem. It is a problem because it does not really understand stuff. For example, if you ask a human "Do you understand that doing X will kill you 100%?". If the human answers "Yes", then you can expect the human to act according to that understanding. That they will not do X But an LLM will happily acknowledges the consequences of doing X, but will still proceed to do X. So rep…

Have you not met humans that do that, too? When it was found that smoking causes cancer, many, many people kept doing it. Then the government taxed the crap out of it, and some people still do it. It's easy to say that people have other motivations, but doesn't the AI, too? You tell it not to do X, but you've also told it to do something that would benefit from X. It's going to "want" to still do X, to support that o…

>smoking causes cancer

Which part of "X will kill you 100%" did you not get?

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#235

Post by hypfer »

DonHopkins wrote: post #223
hypfer wrote: post #211

Earlier quoted context omitted.

I saw that your reply got filtered by the system, googled your name, thought "huh", then tried vouching, but to no avail. But either way. I think you might be operating in the mindset of a high-trust internet that no longer exists. You do not engage with sea lions, concern trolls, grifters, marketing people and the like. You shut them down. inb4 "who are you to decide that?" - "Me. I am me." That said, I appreciate w…

44 years ago I was operating in the mindset of a high-trust arpanet that no longer exists, connecting to the MIT-AI Lab's PDP-10 (@L 134) through the NBS TIP (301-948-3850) at 300 baud with no password, asking nicely for an account to learn LISP and receiving it the next day for free, then learning MACLISP from Kent Pitman and EMACS from Richard Stallman and ZORK by spying on the output of other high-school kids conn…

Okay, but, fair enough, now that you're here (and you're a person with a name and history attached to it), it's less of a waste of time to ponder this.

So personally, I'm not really interested in the hypotheticals of emergent behavior and all that and whether something without memory that works as ours does could develop some different but practically identical form of sentience.

That is possible, but at the end of that hypothetical chain of thoughts might be an outcome in which the human is not the top of the food chain anymore. Which is not to be understood as "oh god, I am full of fear of being obsolete", but "why would I even just spend a second on something that leads to the obsolescence of my kind".

The definition of sentience I believe we must use is one of skin in the game. Only what is bound by the same constraints as we are (mortality, disease, physical presence) may be treated as equal or equal-ish by us, because only then it has constraints that force it to do the same.

Not sure if that answers the questions you've had, but that's that.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#236

Post by wrsh07 »

ModernMech wrote: post #233
wrsh07 wrote: post #231

Earlier quoted context omitted.

To quote Sarah Constantin: > Humans Who Are Not Concentrating Are Not General Intelligences^ But yeah, you should still treat them with humanity. (Related: you should treat LLMs well, not because they're human but because you are^^) ^ https://srconstantin.github.io/2019/02/25/humans-who-are-not... ^^ hmm couldn't find this tweet but didn't look too hard

I gotta say though, turning on caps lock does seem to get them to cut out the inane banter.

There are other methods, including asking them to eg speak using the Google style guide rules, or to speak at 2/10 verbosity, or to speak as if they're talking to a very intelligent middle schooler, or to use simple sentences in subject verb object form

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#237

Post by epihelix »

azan_ wrote: post #214
kelnos wrote: post #147

Earlier quoted context omitted.

> Is anthropomorphizing a real problem? Yes. Actual real people believe that LLMs are actual, thinking, intelligences, perhaps even with consciousness. Your bit with MySQL is harmless because it's obvious that a database isn't a sentient lifeform. But LLMs can look like they're the real deal, and people believe it is. Using terminology like "thinking" and "reasoning" to describe what they do only reinforces this. Hav…

I'd like to hear definition of intelligence that would classify median person as intelligent but LLMs not (and then I'd wait few months for another moving of the goalpost and new definition). Bonus points if it's not recursive e.g. feature that only biological brains can possess.

I wouldn't classify the median human as intelligent.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#238

Post by epihelix »

yreg wrote: post #86
FuckButtons wrote: post #85

Earlier quoted context omitted.

I would argue, that the null hypothesis is that it is not, and that anyone claiming that there is a mote of consciousness are the ones with the burden of proof.

The null hypothesis is that we don't know jack shit about consciousness. Any claim of certainty seems extraordinary to me and I want to hear the evidence.

That's not a null hypothesis, that's a strawman.

Regardless -- if we don't know jack shit about consciousness (your words), then any claims of LLMs being conscious are by definition untestable, pure speculation and based on no evidence at all.

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#239

Post by randomImmigrant »

lern_too_spel wrote: post #128
randomImmigrant wrote: post #119

Earlier quoted context omitted.

Please define "simply biological machines". I'm not sure "biological machine" had a proper definition. What's machine like about biology exactly?

A "machine" is a term that is well defined in science. https://en.wikipedia.org/wiki/Machine

And biological machine is? Don’t get me wrong. Biology I is full of molecules that we call machines. But you’re making a broader claim, saying that biology is only this.

This needs you to answer some questions:

1. Why do the machine parts in biology show such flexible application? A gear cog won’t ever moonlight as a signaling chip, but in biology you often have molecules doing double and triple duty.

2. How is the biological machine able to build itself? What does self assembly imply for the machines function?

3. Where does this machine get its inner drive? No LLM has been found that starts outputting text unprompted. A car doesn’t decide to move to a shady parking spot. Why? Where in the machine to biological machine continuum does the ability to make internally driven decisions come in? Why does it come in for biology? A bacterium is able to make such agentic decisions unprompted. Why is no manufactured machine able to do this?

Re: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces (2025)

#240

Post by tavavex »

salawat wrote: post #110
tavavex wrote: post #82

Earlier quoted context omitted.

Anthropomorphizing is a problem when you're talking about treating something that's not living as if it were. Using humanizing language invites discussions of things like the rights and feelings of an algorithm. A judge that is misled by the application of human-centric language to an algorithm can lead to some terrible outcomes. Not everyone is an LLM expert and the language people use leads to them treating LLMs li…

What is terrifying is the propensity of people hoping for a mechanical slave to do everything possible to avoid touching on the possibility for being-ness of the technology they are desperately hoping will work as a basis for that implementation.

Slavery only exists for living beings because humans have limited lifetimes, experience pain and can have their own desires and needs that can be forcibly taken away from them. Algorithms have none of these things and they don't need them (adding them would be the real cruel thing to do). There is no slavery for something that can't experience coercion.
Post reply on HN