Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is AI reasoning right for the wrong reasons?

quantamagazine.org

Post reply on HN
Print view View original

191–200 of 268 posts

Re: Is AI reasoning right for the wrong reasons?

#191

Post by esikich »

spankalee wrote: post #155
esikich wrote: post #152

Earlier quoted context omitted.

It seems like you are talking about Kant's "thing-in-itself". We can't know any more about an apple than an LLM can. We experience it through our faulty senses. We do not experience the actual apple. This is still open and debatable and is philosophy 101 stuff. I really dislike the confidence on this site, philosophically, that things are just obviously so. The philosophy of mind is massively open for interpretation…

There is a significant difference between experiencing an apple through your own faulty senses and reading words from someone else who experienced an apple through their senses.

You've just described Mary's Room - https://en.wikipedia.org/wiki/Knowledge_argument

Re: Is AI reasoning right for the wrong reasons?

#192

Post by gr_norm »

thesmtsolver2 wrote: post #173
andrewla wrote: post #7

I'll admit that I find this discussion a bit navel-gazy. It has become a question of semantics not a question of actual functionality. The question has become "what do we mean when we use the word 'reasoning'" which is uninteresting. Dijkstra said[1] "... the question whether computers can think. The question is just as relevant and just as meaningful as the question whether submarines can swim." I don't see a clear…

It is not semantics. For decades, logic and CS researchers have known what reasoning is. LLM folks suddenly can’t claim an approximation of that is what constitutes full scale reasoning just because they can achieve only an approximation. Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t hav…

Agree, I've raised this point often. And certainly what remains is still useful, once you accept it! But under no circumstances can we allow scientific achievements to be falsely claimed in service of justifying huge capital investments. Attempting an end-run around the truth, here by redefining words to mean things they don't, always slows down real progress.

Re: Is AI reasoning right for the wrong reasons?

#193

Post by esikich »

crent wrote: post #175
esikich wrote: post #168

Earlier quoted context omitted.

And our nerves are a series of electrochemical impulses. There are nerves that do optical processing before it hits the brain. It's not obvious, and that's what's frustrating around here. "Computers can't possibly experience things". I'd argue it's just as unlikely for meat to do it. Again, philosophy 101 stuff.

I’m with you in this thread. I’m by no means well studied in philosophy but I enjoy the domain so I’ve learned enough to know how difficult nailing down some of these topics can be. I often find the egotistical viewpoint to be banal. I don’t mean egotistical in a demeaning way - I mean it just to frame the types of thinking that revolve around the self, inner world or “my individual experience”. It’s an obvious, but…

Even the 5 senses are arbitrary. Sensing where your body is positioned, where it's boundaries are, whatever tickling is, whatever itchy is, whatever sour is, whatever salty is, green, blue, etc. When we say 5, we are just saying where on the body they happen. Even though there are more axis on each and use different sensors. Warmth, cold, orgasm - is that really "touch" etc. Bunch of syscall enums maybe.

Re: Is AI reasoning right for the wrong reasons?

#194

Post by quietbritishjim »

thesmtsolver2 wrote: post #173
andrewla wrote: post #7

I'll admit that I find this discussion a bit navel-gazy. It has become a question of semantics not a question of actual functionality. The question has become "what do we mean when we use the word 'reasoning'" which is uninteresting. Dijkstra said[1] "... the question whether computers can think. The question is just as relevant and just as meaningful as the question whether submarines can swim." I don't see a clear…

It is not semantics. For decades, logic and CS researchers have known what reasoning is. LLM folks suddenly can’t claim an approximation of that is what constitutes full scale reasoning just because they can achieve only an approximation. Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t hav…

> Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples

This is a poor analogy because:

* Multiplying numbers has a single objective answer. Whether a code is good (sometimes even just whether it's correct) can be quite subjective.

* LLMs certainly do some level of composition between the data sources they were trained on i.e. they are more than just lookup tables.

* We have calculators that actually do multiply large numbers accurately. We don't have anything that automatically writes code that is definitely correct and "good".

To address only the last point: imagine you had a device that would quickly factor large numbers used in modern criticality, but occasionally got it wrong. You could waste a lot time debating whether it's a "calculator", but it's still certainly useful to have one.

Re: Is AI reasoning right for the wrong reasons?

#195

Post by simianwords »

solid_fuel wrote: post #181
thesmtsolver2 wrote: post #173

Earlier quoted context omitted.

It is not semantics. For decades, logic and CS researchers have known what reasoning is. LLM folks suddenly can’t claim an approximation of that is what constitutes full scale reasoning just because they can achieve only an approximation. Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t hav…

> Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t have it in its training dataset. > We won’t say the program actually multiplies numbers. That's a good analogy. To extend it further, in cases where the calculator can't handle a question - e.g. numbers too large - a properly designed calcu…

This is a strawman. LLMs have the ability to identify when they can’t solve a question just like humans.

Re: Is AI reasoning right for the wrong reasons?

#196

Post by mdp2021 »

charcircuit wrote: post #176
mdp2021 wrote: post #100

Earlier quoted context omitted.

The paradox of a reduced importance of the contents of a CoT must be explained.

What's the paradox? You can have increased performance over no CoT by just having random tokens as your "CoT".

> What's the paradox

The textual development of a thesis must be sound, the proof accompanying a theorem etc. That an output be accompanied by a non-relevant text is a paradox.

It is also part of the "problem of transparency in AI (in its subset of the non deterministic algorithms)".

Re: Is AI reasoning right for the wrong reasons?

#197

Post by yakbarber »

I keep hearing we don’t know “how llm’s work”, in mean yes we know the algorithms but the WHY I suppose. I’m not sure if that’s really true, does the best researcher at OpenAI, Anthropic, Gemini not know why it works? Would they say that?

Anyway these discussions always make me think we are too generous to humans. I know very few humans who are good at reasoning. It’s surprisingly hard to just think really hard through a problem. We mostly intuit, act, repeat. It’s a rare thing for someone to deeply reason.

The other problem I have with this is that we apply the word “reasoning” here because we haven’t really got another language for it, so we anthropomorphise the llm because that’s our reductive mental model and then complain that it’s not human enough.

Re: Is AI reasoning right for the wrong reasons?

#198

Post by mdp2021 »

drob518 wrote: post #106
mdp2021 wrote: post #99

Earlier quoted context omitted.

We do not academically consider reasoning or thinking an activity restricted to humans, but as absolute processes.

Sure, other living, non-human things clearly reason and think. My German shepherd dog is quite smart and demonstrates these processes all the time. But is a machine following an algorithm (even a stochastic algorithm) “thinking” or “reasoning?” I think not. The algorithm has gotten so difficult for us to explain that we struggle with it, but in the end it’s all just matrix math and some randomization. From a given nu…

> is a machine following an algorithm (even a stochastic algorithm) “thinking” or “reasoning”?

If the process is sound, yes, it can be said to be thinking or reasoning - it produced "sound thought or reasoning".

The "reasoning" is there and elsewhere the absolute (from the producing entity) output which is valid as a representation of proper reasoning.

If "A is B and B is C, then A is C" is reasoning, than what produces it in a way "reasons". If not, then a proper definition of "reasoning" must be provided, for it to be discussed.

Re: Is AI reasoning right for the wrong reasons?

#199

Post by raducu »

andrewla wrote: post #7

I'll admit that I find this discussion a bit navel-gazy. It has become a question of semantics not a question of actual functionality. The question has become "what do we mean when we use the word 'reasoning'" which is uninteresting. Dijkstra said[1] "... the question whether computers can think. The question is just as relevant and just as meaningful as the question whether submarines can swim." I don't see a clear…

> I don't see a clear demarcation of the things that only "reasoning" can accomplish.

Producing beautiful, coherent, maintainable code. RL trained the models to produce correct code but boy oh boy is it ugly if it's 10k+ LOC

Re: Is AI reasoning right for the wrong reasons?

#200

Post by mdp2021 »

ForHackernews wrote: post #119
mdp2021 wrote: post #97

Earlier quoted context omitted.

> discourage anthropomorphisation by avoiding personal pronouns The fault is in the naïve antropomorphizer not in that which writes "I" being in fact a "first person".

Irrelevant. Humans are frail and we shouldn't design systems with dark patterns that take advantage of human frailty.

> we shouldn't design systems with dark patterns that take advantage of human frailty

First of all: «humans are frail», so the paramount social and individual objective, goal, is to overcome frailty (in cognitive and psychological and judgement functions).

Secondly: you are using a bad argument putting even innocent and somehow proper implementations in the set of "exploitative dark patterns".

And importantly: we the adults must not be bound in the Procuste's bed of a world for the use of children. If people cry when e.g. somebody multiplies 123 and 321, they must be brought back to health, and we must be in the condition of making our free and legitimate multiplications.

Post reply on HN