Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is AI reasoning right for the wrong reasons?

quantamagazine.org

Post reply on HN
Print view View original

201–210 of 268 posts

Re: Is AI reasoning right for the wrong reasons?

#201

Post by fragmede »

thesmtsolver2 wrote: post #173
andrewla wrote: post #7

I'll admit that I find this discussion a bit navel-gazy. It has become a question of semantics not a question of actual functionality. The question has become "what do we mean when we use the word 'reasoning'" which is uninteresting. Dijkstra said[1] "... the question whether computers can think. The question is just as relevant and just as meaningful as the question whether submarines can swim." I don't see a clear…

It is not semantics. For decades, logic and CS researchers have known what reasoning is. LLM folks suddenly can’t claim an approximation of that is what constitutes full scale reasoning just because they can achieve only an approximation. Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t hav…

> For decades, logic and CS researchers have known what reasoning is.

Oh good. Can you share that definition with the rest of us then? We're out here fumbling around trying to define what reasoning is without the benefit of their definition.

Re: Is AI reasoning right for the wrong reasons?

#202

Post by mdp2021 »

Borealid wrote: post #128
mdp2021 wrote: post #90

Earlier quoted context omitted.

In order to properly engineer things, we must know how they work. We cannot just jump from emergent property to more convincing emergent property as if the rocks on the surface of a lake. "Reasoning" is an important part in said framework: yes, we must understand how it works, how it works properly, how its simulations work, how they work properly...

No, we only need to know how the thing-building thing works. We don't have to understand the resulting product at all. Genetic algorithms are an example - we understand the evolution function, not what evolves from it.

You must have misunderstood the statement: «In order to properly engineer things, we must know how they work»: of course they may work anyway, of course there may be emergent properties (as was explicitly written), but the deontic part of knowledge augmentation, the scientific part, is missing.

AI is full of non deterministic devices like genetic algorithms, but there exists a problem of transparency which is paramount in the discipline. Automatically produced solution S works: why does it seem to work? Where would it fail? We must understand that.

Re: Is AI reasoning right for the wrong reasons?

#203

Post by mdp2021 »

kubanczyk wrote: post #147
mdp2021 wrote: post #87

Earlier quoted context omitted.

> We want them to understand what's We want them to assess what's true and what's not.

We want them to better predict consequences of their outputs.

> consequences of their outputs

For what purpose exactly? When the computer is asked the solution to a problem, it must be absolute: "What is the sum of 12 and 45"; "Is Jack cognitively apt" etc.: we want a truth, strictly. What to do with that truth, it is another problem, and not the priority of the computer vis-a-vis the requirement from truthiness. This is, of course, still in the context of adults.

Re: Is AI reasoning right for the wrong reasons?

#204

Post by poetaster »

solid_fuel wrote: post #187
esikich wrote: post #152

Earlier quoted context omitted.

It seems like you are talking about Kant's "thing-in-itself". We can't know any more about an apple than an LLM can. We experience it through our faulty senses. We do not experience the actual apple. This is still open and debatable and is philosophy 101 stuff. I really dislike the confidence on this site, philosophically, that things are just obviously so. The philosophy of mind is massively open for interpretation…

No, the other comments pointing out that LLMs do not have senses and because of that they do not "experience" an apple at all are correct. When given a sequence of tokens e.g. "an apple tastes " an LLM performs a mathematical transformation that modifies the input tokens using the stored weights within the model to predict the most statistically likely next token(s) (which may be "sweet and crisp..."). It's important…

I'll go so far as to propose that many brains, dog or cat, certainly display 'signs' of consciousness. They are nothing like LLMs either. Confusing some stream of language with consciousness is missing the a priori.

Re: Is AI reasoning right for the wrong reasons?

#205

Post by ForHackernews »

mdp2021 wrote: post #200
ForHackernews wrote: post #119

Earlier quoted context omitted.

Irrelevant. Humans are frail and we shouldn't design systems with dark patterns that take advantage of human frailty.

> we shouldn't design systems with dark patterns that take advantage of human frailty First of all: «humans are frail», so the paramount social and individual objective, goal, is to overcome frailty (in cognitive and psychological and judgement functions). Secondly: you are using a bad argument putting even innocent and somehow proper implementations in the set of "exploitative dark patterns". And importantly: we the…

That's a lot of fancy words to say, "I think I'm too smart to fall for the same tricks that work on other humans."

How many hours did you spend on your phone last month? How many fewer hours would you have spent if the apps you interact with weren't designed to maximize "engagement"?

Re: Is AI reasoning right for the wrong reasons?

#206

Post by poetaster »

drekipus wrote: post #154
esikich wrote: post #152

Earlier quoted context omitted.

It seems like you are talking about Kant's "thing-in-itself". We can't know any more about an apple than an LLM can. We experience it through our faulty senses. We do not experience the actual apple. This is still open and debatable and is philosophy 101 stuff. I really dislike the confidence on this site, philosophically, that things are just obviously so. The philosophy of mind is massively open for interpretation…

LLMs don't experience because they are word generators. They don't have senses. Some things are allowed to be obvious. It's a series of bits and bytes.

If by 101 you mean the decades long debate between Dennet and everyone else, I think you'll find the 101 gloss throws up more questions than answers. Presuming that language is necessary for consciousness is not a settled question, for instance. As you complain, it is a more complicated field of study than cs, philosophy, biology and chemistry combined.

Re: Is AI reasoning right for the wrong reasons?

#207

Post by Jtarii »

zarzavat wrote: post #83

LLMs lack qualia, among other things. If I ask an LLM "what is an apple?" it tells me: > An apple is the edible fruit of the apple tree, scientifically known as Malus domestica. It is one of the world's most widely grown fruits and is eaten fresh or used in many foods and drinks. If I ask an LLM "what is a mundu fruit?" it tells me: > Mundu is a tropical fruit native to Southeast Asia, especially found in Indonesia,…

Do you think there is a difference between an LLM describing a photo that has been sent to it, and generating a description of the photo from its training data (assuming the photo itself isn't in its training data). If the end description produced happens to be the same.

Is this not what you are describing when you say that the thing itself and the description of the thing are different experiences?

Re: Is AI reasoning right for the wrong reasons?

#208

Post by tim333 »

Diogenesian wrote: post #17

What an asshole: On the other side of the AI-reasoning fence, the disdain seems to be mutual. “These ‘scientific’ papers from last summer — I would put this in big, big air quotes,” said Sébastien Bubeck, a member of OpenAI’s technical staff (and a prominent evangelist for the company’s reasoning models among scientists and mathematicians). He called earlier Apple results critiquing AI reasoning “wrong,” claiming tha…

I got the same impression as Bubeck that the ‘scientific’ papers were a bit more like blog post saying this LLM got stuff wrong so LLMs can't reason, but as he says they can now do that so it's not an inherent limit of the technology, just the 2025 versions weren't up to it.

Re: Is AI reasoning right for the wrong reasons?

#209

Post by drob518 »

mdp2021 wrote: post #198
drob518 wrote: post #106

Earlier quoted context omitted.

Sure, other living, non-human things clearly reason and think. My German shepherd dog is quite smart and demonstrates these processes all the time. But is a machine following an algorithm (even a stochastic algorithm) “thinking” or “reasoning?” I think not. The algorithm has gotten so difficult for us to explain that we struggle with it, but in the end it’s all just matrix math and some randomization. From a given nu…

> is a machine following an algorithm (even a stochastic algorithm) “thinking” or “reasoning”? If the process is sound, yes, it can be said to be thinking or reasoning - it produced "sound thought or reasoning". The "reasoning" is there and elsewhere the absolute (from the producing entity) output which is valid as a representation of proper reasoning. If "A is B and B is C, then A is C" is reasoning, than what produ…

Yes, a proper definition of reasoning must be provided. How are you defining reasoning, exactly? You imply that formal logic is reasoning. Is reasoning just formal logic or is formal logic just a subset of reasoning? If a subset, what’s the rest of it? Since they implement logic, do Prolog programs “reason” or are they just performing inference according to the SLD resolution algorithm?

Re: Is AI reasoning right for the wrong reasons?

#210

Post by mdp2021 »

ForHackernews wrote: post #205
mdp2021 wrote: post #200

Earlier quoted context omitted.

> we shouldn't design systems with dark patterns that take advantage of human frailty First of all: «humans are frail», so the paramount social and individual objective, goal, is to overcome frailty (in cognitive and psychological and judgement functions). Secondly: you are using a bad argument putting even innocent and somehow proper implementations in the set of "exploitative dark patterns". And importantly: we the…

That's a lot of fancy words to say, "I think I'm too smart to fall for the same tricks that work on other humans." How many hours did you spend on your phone last month? How many fewer hours would you have spent if the apps you interact with weren't designed to maximize "engagement"?

> How many hours did you spend on your phone last month? How many fewer hours would you have spent if the apps you interact with weren't designed to maximize "engagement"?

?? Very strictly zero? You are assuming we would install engagement-relevant third party apps on our palm computers?!

I use my time exactly as I decide moment to moment.

> fancy

Eh?!

> too smart to fall for

Very ill framed. We are adults, we behave like adults, and we need a society of adults.

Post reply on HN