Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is AI reasoning right for the wrong reasons?

quantamagazine.org

Post reply on HN
Print view View original

261–268 of 268 posts

Re: Is AI reasoning right for the wrong reasons?

#261

Post by solid_fuel »

Windchaser wrote: post #260
esikich wrote: post #190

Earlier quoted context omitted.

Tell me why only human brains are capable of whatever thought, conscious, or experience is (if you can define them) and I'll congratulate you on solving the hard problem of consciousness.

Reading both your comments, I don't see them suggest that only human brains are capable of experience. Not even "only animal brains", for that matter.

The above from `esikich` is just a classic response from LLM maximalists. I point out that LLMs and actual brains - like found in nature - are vastly different in architecture and that the differences create actual, material issues when considering if LLMs are conscious. Instead of actually addressing any of the points, they just pull the classic "well no one has fully solved consciousness so my supposition that LLMs are conscious must be as good as any argument against it."

It's not really worth debating further, since they aren't actually arguing from a reasoned position, but I appreciate that some people are actually taking time to read the arguments presented.

Re: Is AI reasoning right for the wrong reasons?

#262

Post by solid_fuel »

poetaster wrote: post #204
solid_fuel wrote: post #187

Earlier quoted context omitted.

No, the other comments pointing out that LLMs do not have senses and because of that they do not "experience" an apple at all are correct. When given a sequence of tokens e.g. "an apple tastes " an LLM performs a mathematical transformation that modifies the input tokens using the stored weights within the model to predict the most statistically likely next token(s) (which may be "sweet and crisp..."). It's important…

I'll go so far as to propose that many brains, dog or cat, certainly display 'signs' of consciousness. They are nothing like LLMs either. Confusing some stream of language with consciousness is missing the a priori.

Well yes, I would say that furthers my point though. Cats, dogs, flies, toads... they all display an embodied behavior and internal motivation that LLMs completely lack. Cats and dogs are typically considered both conscious and sentient for what it's worth - no language required.

Re: Is AI reasoning right for the wrong reasons?

#263

Post by ForHackernews »

mdp2021 wrote: post #248
ForHackernews wrote: post #232

Earlier quoted context omitted.

You're arguing that we should be deliberately build evil and manipulative technology... why? To assert your own superior discernment?

Nobody has said that one should build «evil and manipulative technology». An interlocutor that says "I" is not «evil and manipulative technology». People (normal people) are not manipulated by a generator that says "I". There is an I in that generator (it is the generator that outputs its output); and those who have weakminded reactions in front of innocent phenomena (e.g. those who would believe that "My name is Ish…

I suspect you will not accept this (because you have been manipulated by the generator?) but there is no "I" in repeated inference calls to large language models. There is no coherent entity behind the text. Try switching backend APIs mid-session, flip from Claude to Gemini and then to ChatGPT and the output will happily burble onwards "Oops, you're right, I see my last answer contained an error..."

It's lying to users just as much as an email autoreply that pretended to be a human response is lying.

Recommended reading: https://transformer-circuits.pub/2025/attribution-graphs/bio...

Re: Is AI reasoning right for the wrong reasons?

#264

Post by mdp2021 »

drob518 wrote: post #253
mdp2021 wrote: post #249

Earlier quoted context omitted.

> proper definition of reasoning must be provided It will be related to the context. If a process tells me that "if the light is blue and the sound is sharp then the lever must be lowered", it outputs reasoning; if what the activity did was running on conditional paths like an Expert System, what it did was different from the content and only improper reasoning; if the activity produced semantic evaluations about the…

> If a process tells me that "if the light is blue and the sound is sharp then the lever must be lowered", it outputs reasoning By this definition, all computer programs using simple IF statements are reasoning.

You misread the post. Pls read it again.

Re: Is AI reasoning right for the wrong reasons?

#265

Post by ricardobeat »

solid_fuel wrote: post #187
esikich wrote: post #152

Earlier quoted context omitted.

It seems like you are talking about Kant's "thing-in-itself". We can't know any more about an apple than an LLM can. We experience it through our faulty senses. We do not experience the actual apple. This is still open and debatable and is philosophy 101 stuff. I really dislike the confidence on this site, philosophically, that things are just obviously so. The philosophy of mind is massively open for interpretation…

No, the other comments pointing out that LLMs do not have senses and because of that they do not "experience" an apple at all are correct. When given a sequence of tokens e.g. "an apple tastes " an LLM performs a mathematical transformation that modifies the input tokens using the stored weights within the model to predict the most statistically likely next token(s) (which may be "sweet and crisp..."). It's important…

CS101 != philosophy

Re: Is AI reasoning right for the wrong reasons?

#266

Post by Plont »

jameshart wrote: post #157
andy99 wrote: post #96

Earlier quoted context omitted.

That’s just conflating reasoning and “parallel reconstruction” - there is such thing as reasoning, and I agree it’s probably less common in decision making, but it’s fundamental to many tasks where we figure out a solution, like writing an algorithm, math proofs, etc.

Is it really? I think the assumption that there is some sort of formal ‘reasoning’ process that is inherently superior to whatever ‘non-reason-based’ decision process ‘non-reasoning’ systems use is not immediately justifiable. Most reasoning is ultimately based in heuristics: this is the right thing to do next because it has worked in the past. This is the right thing to do because our intuition tells us doing it wil…

It's true that humans are not perfectly logical actors, but even so, most humans won't tell you seriously that the swimming pools in the Titanic are empty, or that glue is a great pizza topping.

At the end of the day, there's a difference between showing your work for a math problem by writing down each logical step as you work through it, vs. just blindly copying both the answer and the "show your work" part from your classmates.

That humans can fail at math, or at other kinds of logic problems, is immaterial. It's still an entirely different process, and it leads to different results.

The fact that humans often fail because they're actively prioritizing something else can also be for the best. Like, it's reasonable to get a math problem wrong because you're distracted by argueing classmates who might be about to start a fight right next to you. Or you might realize that the premise of a question itself is wrong. Or you might logically know that the Earth goes around the sun, but decide to pretend otherwise because it's not worth dying over.

Humans are always balancing priorities, and it doesn't always make sense to judge their success by a single metric like "do these math problems and show your work within 10 minutes". LLMs do not have these conflicts, even when we would want them to.

Re: Is AI reasoning right for the wrong reasons?

#267

Post by m348e912 »

inigyou wrote: post #135
m348e912 wrote: post #76

Earlier quoted context omitted.

I'll make the argument that part of the value of "consciousness" and "humanity" is uniqueness. Here is an absurd example: If I could carbon copy 100,000 of you.... and later I decided to "take out" one of the copies, would there be a whole lot of societal blowback? Especially if I could make another one.

I think slave owners did exactly that.

They cloned their slaves and killed some of them?

Re: Is AI reasoning right for the wrong reasons?

#268

Post by mdp2021 »

ForHackernews wrote: post #263
mdp2021 wrote: post #248

Earlier quoted context omitted.

Nobody has said that one should build «evil and manipulative technology». An interlocutor that says "I" is not «evil and manipulative technology». People (normal people) are not manipulated by a generator that says "I". There is an I in that generator (it is the generator that outputs its output); and those who have weakminded reactions in front of innocent phenomena (e.g. those who would believe that "My name is Ish…

I suspect you will not accept this (because you have been manipulated by the generator?) but there is no "I" in repeated inference calls to large language models. There is no coherent entity behind the text. Try switching backend APIs mid-session, flip from Claude to Gemini and then to ChatGPT and the output will happily burble onwards "Oops, you're right, I see my last answer contained an error..." It's lying to use…

> but there is no "I" in repeated inference calls to large language models

We do not assume that. We do not assume any "I" as any "persistent flow" with the current architectures. We find it bewildering that you assume that people - not just /some/ people - would do that. We call it legitimate that a formulator may say "I". Your idea of "I" and its constraints is just yours.

> as much as an email autoreply that pretended to be a human response is lying

Nobody in their right mind confuses an autoreply with anything else that some billboard.

> because you have been manipulated by the generator?

Duude...

I must repeat: there exist lucid minded people, not just the delirious junkies you see.

Post reply on HN