Is AI reasoning right for the wrong reasons? • Hacker News Board

Re: Is AI reasoning right for the wrong reasons?

#181

Post by solid_fuel » Sat, Aug 01, 2026, 6:16 AM UTC

thesmtsolver2 wrote: post #173

andrewla wrote: post #7

I'll admit that I find this discussion a bit navel-gazy. It has become a question of semantics not a question of actual functionality. The question has become "what do we mean when we use the word 'reasoning'" which is uninteresting. Dijkstra said[1] "... the question whether computers can think. The question is just as relevant and just as meaningful as the question whether submarines can swim." I don't see a clear…

It is not semantics. For decades, logic and CS researchers have known what reasoning is. LLM folks suddenly can’t claim an approximation of that is what constitutes full scale reasoning just because they can achieve only an approximation. Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t hav…

> Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t have it in its training dataset.

> We won’t say the program actually multiplies numbers.

That's a good analogy. To extend it further, in cases where the calculator can't handle a question - e.g. numbers too large - a properly designed calculator returns an error instead of a randomly hallucinated answer. We haven't even achieved that level of safeguard around token predictors yet.

Re: Is AI reasoning right for the wrong reasons?

#182

Post by zuzululu » Sat, Aug 01, 2026, 6:27 AM UTC

bigfishrunning wrote: post #123

zuzululu wrote: post #117

Earlier quoted context omitted.

You sound very upset and angry but there is no need to use curse words and shift all the blame on me. I just figured out a way to make AI work for me, I am qualified to work the jobs, that doesn't mean I am gate keeping it. More importantly I am not responsible for other's employment situation, that's on them, there are plenty of remote jobs out there, just keep your head up, and I'm sure you will land on your feet.

I apologize for my language. I was recently burned by an coworker who was moonlighting, and they left me with a really big mess to clean up (both technically and politically). Just know that your behavior has an effect on those around you; bugs leak information, and cost money. Reviewing 3 programmer's worth of output is a lot of work, and it's only a matter of time before something gets missed.

and what does that have to do with me ?

Re: Is AI reasoning right for the wrong reasons?

#183

Post by kulahan » Sat, Aug 01, 2026, 6:37 AM UTC

thesmtsolver2 wrote: post #172

kgeist wrote: post #167

Transformers lack recursion and are limited by the network's fixed depth, so "reasoning", IMHO, is basically a way to emulate deeper recursion. As we go through the layers, concepts are pattern-matched and refined, but at some point we have to stop and cannot refine them any further (no more layers). Usually, this refinement continues during the generation of the next token (the previous intermediate results needed t…

Don't humans make mistakes too? Never understood this argument. Humans get simple multiplication wrong often, so ok for computers to make mistakes multiplying numbers?

The point is that LLMs are doing human-ish things on their own, and that's what the value in them is, so OP is bringing attention to the fact that the tradeoff is "thinks dynamically, sometimes makes mistakes". Computers have never been built to think dynamically before.

Re: Is AI reasoning right for the wrong reasons?

#184

Post by astrobe_ » Sat, Aug 01, 2026, 6:39 AM UTC

> How much does it matter whether or not we can accurately observe, characterize, and validate the processes at work inside large reasoning models

If you don't or cannot, then it's the kind of sufficiently advanced technology that looks like magic. Except it is the problematic type of magic, like X-ray. They "magically" let you see the bones inside bodies. But you discover that you shouldn't do it too often only when the damage has already been done.

Re: Is AI reasoning right for the wrong reasons?

#185

Post by notrealyme123 » Sat, Aug 01, 2026, 6:48 AM UTC

astrobe_ wrote: post #184

> How much does it matter whether or not we can accurately observe, characterize, and validate the processes at work inside large reasoning models If you don't or cannot, then it's the kind of sufficiently advanced technology that looks like magic. Except it is the problematic type of magic, like X-ray. They "magically" let you see the bones inside bodies. But you discover that you shouldn't do it too often only when…

The assumption is that those reasoning traces are the lowest level. But we can't observe the reasoning-for-reasoning (the reasoning in a transformer block.)

Saying that one is enough is pretty arbitrary.

Re: Is AI reasoning right for the wrong reasons?

#186

Post by urbsgpw » Sat, Aug 01, 2026, 6:52 AM UTC

solid_fuel wrote: post #181

thesmtsolver2 wrote: post #173

Earlier quoted context omitted.

It is not semantics. For decades, logic and CS researchers have known what reasoning is. LLM folks suddenly can’t claim an approximation of that is what constitutes full scale reasoning just because they can achieve only an approximation. Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t hav…

> Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t have it in its training dataset. > We won’t say the program actually multiplies numbers. That's a good analogy. To extend it further, in cases where the calculator can't handle a question - e.g. numbers too large - a properly designed calcu…

But isn't it the case that we can't reach this safeguard with the current architecture? I remember Karpathy making an interesting point 2 years (cca) back, that I would summarize somehow like this: the mechanics behind every LLM answer are the same, what you then call hallucination is more or less a consequence of whether or not the answer was factually correct/useful.

Which would mean, as is so often the case, that the "killer feature" of the LLMs is also its biggest weakness and the two can't be disentangled. Now, we are inventive creatures and we might come up with a remedy for these issues, but what you basically see so far is more guardrails, the use of harnesses and building a whole bunch of infrastructure around the LLMs to get useful work out of them.

Which, btw is not a critique, I do it as well and it's a fun engineering challenge.

Re: Is AI reasoning right for the wrong reasons?

#187

Post by solid_fuel » Sat, Aug 01, 2026, 6:54 AM UTC

esikich wrote: post #152

zarzavat wrote: post #83

LLMs lack qualia, among other things. If I ask an LLM "what is an apple?" it tells me: > An apple is the edible fruit of the apple tree, scientifically known as Malus domestica. It is one of the world's most widely grown fruits and is eaten fresh or used in many foods and drinks. If I ask an LLM "what is a mundu fruit?" it tells me: > Mundu is a tropical fruit native to Southeast Asia, especially found in Indonesia,…

It seems like you are talking about Kant's "thing-in-itself". We can't know any more about an apple than an LLM can. We experience it through our faulty senses. We do not experience the actual apple. This is still open and debatable and is philosophy 101 stuff. I really dislike the confidence on this site, philosophically, that things are just obviously so. The philosophy of mind is massively open for interpretation…

No, the other comments pointing out that LLMs do not have senses and because of that they do not "experience" an apple at all are correct.

When given a sequence of tokens e.g. "an apple tastes " an LLM performs a mathematical transformation that modifies the input tokens using the stored weights within the model to predict the most statistically likely next token(s) (which may be "sweet and crisp...").

It's important to understand how these things work - and how different they are from a human brain. Despite it being a common point of confusion, the architecture of a transformer is massively different from the neural architecture of a human brain - and the human brain isn't just neurons. Just as a start, if you ever look at some of the attempts to map synaptic networks in brains you will notice they are full of recurrent loops, while a transform is feed-forward only. There's also tons of mediating chemicals, several (known) distinct kinds of synapses, and other kinds of cells as well.

Just one example of the consequences of these differences: LLMs stop "experiencing" anything once a stop token has been generated. Meanwhile, Humans in sensory deprivation tanks may hallucinate vividly - but keep thinking.

This is pretty simple, CS101 stuff.

Re: Is AI reasoning right for the wrong reasons?

#188

Post by astrobe_ » Sat, Aug 01, 2026, 6:57 AM UTC

kulahan wrote: post #183

thesmtsolver2 wrote: post #172

Earlier quoted context omitted.

Don't humans make mistakes too? Never understood this argument. Humans get simple multiplication wrong often, so ok for computers to make mistakes multiplying numbers?

The point is that LLMs are doing human-ish things on their own, and that's what the value in them is, so OP is bringing attention to the fact that the tradeoff is "thinks dynamically, sometimes makes mistakes". Computers have never been built to think dynamically before.

"think dynamically" is the type of discourse TFA denounces, though. The thing designated by "think" is already largely unclear when you think about it, "think dynamically" sounds like self-improvement-coach nonsense.

Re: Is AI reasoning right for the wrong reasons?

#189

Post by esikich » Sat, Aug 01, 2026, 7:14 AM UTC

drekipus wrote: post #171

esikich wrote: post #168

Earlier quoted context omitted.

And our nerves are a series of electrochemical impulses. There are nerves that do optical processing before it hits the brain. It's not obvious, and that's what's frustrating around here. "Computers can't possibly experience things". I'd argue it's just as unlikely for meat to do it. Again, philosophy 101 stuff.

I think you're trying to degrade the entire school of philosophy down to your "philosophy 101" understanding and then shoving it where it doesn't fit in order to feel like this is all complicated. Philosophy is a intrinsically human thing. It has to be. That is the whole point. There really is nothing else. Plants feel and experience things too. But there's no philosophy of the plant experience from the point of view…

"Philosophy is an intrinsically human thing. It has to be."

What is philosophy? Thinking? Reasoning? Thinking about thinking?

You're asserting that only a human brain can think and reason? Surely other animals can as well. Couldn't aliens as well? Why can't this crazy electronic box?

Where do we draw the line? Do dolphins not have internal experiences? Dogs? Cats? Mice? Fish? Worms? Bugs? Amoebas?

We don't even really know what thinking or experience is. You can train a caterpillar to do tricks. That's pattern recognition. You can train a human to spike their heart rate when you get a Teams notification.

I mean, I'm watching an LLM run benchmarks on my caching strategy for a webpage right now. It's certainly doing something.

Re: Is AI reasoning right for the wrong reasons?

#190

Post by esikich » Sat, Aug 01, 2026, 7:19 AM UTC

solid_fuel wrote: post #187

esikich wrote: post #152

Earlier quoted context omitted.

It seems like you are talking about Kant's "thing-in-itself". We can't know any more about an apple than an LLM can. We experience it through our faulty senses. We do not experience the actual apple. This is still open and debatable and is philosophy 101 stuff. I really dislike the confidence on this site, philosophically, that things are just obviously so. The philosophy of mind is massively open for interpretation…

No, the other comments pointing out that LLMs do not have senses and because of that they do not "experience" an apple at all are correct. When given a sequence of tokens e.g. "an apple tastes " an LLM performs a mathematical transformation that modifies the input tokens using the stored weights within the model to predict the most statistically likely next token(s) (which may be "sweet and crisp..."). It's important…

Tell me why only human brains are capable of whatever thought, conscious, or experience is (if you can define them) and I'll congratulate you on solving the hard problem of consciousness.