Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is AI reasoning right for the wrong reasons?

quantamagazine.org

Post reply on HN
Print view View original

221–230 of 268 posts

Re: Is AI reasoning right for the wrong reasons?

#221

Post by dannyw »

nearbuy wrote: post #174
andy99 wrote: post #13

Back in the day it was a bit of a cliche to bring up “clever Hans”, the horse that could do math, when talking about machine learning. He couldn’t do math but he read some cues from his handler of pick the write answers, the handler iirc wasn’t in on it. The point of the story was that classifiers can be right for the wrong reasons and almost inevitably are. At least there’s zero guarantee that the reason for making…

LLMs are not classifiers. A classifier is an algorithm or neural net that assigns a label from a fixed set of labels to an input. You can broaden the definition of classifier to anything that internally divides its input space into regions, but that definition would include every neural network, whether biological or artificial. So it's not very meaningful, and certainly doesn't give any insight into how they differ…

But a LLM literally does assign a token from a fixed set of tokens (its vocab) to an input, rinse and repeat, until the stop token.

Classifiers have been giving logits since decades ago.

Re: Is AI reasoning right for the wrong reasons?

#222

Post by lanstin »

totetsu wrote: post #159
andrewla wrote: post #7

I'll admit that I find this discussion a bit navel-gazy. It has become a question of semantics not a question of actual functionality. The question has become "what do we mean when we use the word 'reasoning'" which is uninteresting. Dijkstra said[1] "... the question whether computers can think. The question is just as relevant and just as meaningful as the question whether submarines can swim." I don't see a clear…

> "what do we mean when we use the word 'reasoning'" This is exactly the most interesting and important point. Because the word reason has been subject to so much philosophical debate throughout history, it is very laden with meanings and implications. It sets all kinds of expectations about normativity, objectivity, truth, and humans convincing other humans. To use the word reason for a stochastic generator is misle…

Consider the difference between “reasonable” and “rational”

Re: Is AI reasoning right for the wrong reasons?

#223

Post by zarzavat »

esikich wrote: post #152
zarzavat wrote: post #83

LLMs lack qualia, among other things. If I ask an LLM "what is an apple?" it tells me: > An apple is the edible fruit of the apple tree, scientifically known as Malus domestica. It is one of the world's most widely grown fruits and is eaten fresh or used in many foods and drinks. If I ask an LLM "what is a mundu fruit?" it tells me: > Mundu is a tropical fruit native to Southeast Asia, especially found in Indonesia,…

It seems like you are talking about Kant's "thing-in-itself". We can't know any more about an apple than an LLM can. We experience it through our faulty senses. We do not experience the actual apple. This is still open and debatable and is philosophy 101 stuff. I really dislike the confidence on this site, philosophically, that things are just obviously so. The philosophy of mind is massively open for interpretation…

> We can't know any more about an apple than an LLM can

When I hold and taste an apple I know that the apple exists. Not an issue for apples, you might say, but there's plenty of real information that is not written down and is only derivable from our senses. Such as instructions for using car washes, apparently.

Humans are naturally sceptical of descriptions, preferring first-hand experiences because descriptions can be unreliable, misreported, fabricated, etc.

Because LLMs live in the world of descriptions, they lack the dynamic range of credibility that humans have available.

LLMs can only distinguish between fictitious descriptions and factual descriptions. They must map all of their credibility into this range.

Humans meanwhile can distinguish between fictitious descriptions, presumed-to-be-factual descriptions, and actually factual personal experiences.

It's not intrinsic to LLMs being inorganic, it's intrinsic to LLMs being unembodied.

Re: Is AI reasoning right for the wrong reasons?

#225

Post by paulddraper »

kevinwang wrote: post #219
thesmtsolver2 wrote: post #173

Earlier quoted context omitted.

It is not semantics. For decades, logic and CS researchers have known what reasoning is. LLM folks suddenly can’t claim an approximation of that is what constitutes full scale reasoning just because they can achieve only an approximation. Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t hav…

> It is not semantics. For decades, logic and CS researchers have known what reasoning is. Curious what this is!

Me too.

What does a reasoning program look like and why is matrix multiplication not that?

Re: Is AI reasoning right for the wrong reasons?

#226

Post by randomImmigrant »

andrewla wrote: post #62
goatlover wrote: post #58

Earlier quoted context omitted.

I believe Djikstra's quote has long been taken out of context. It was a criticism of other computer scientists anthropomorphizing machines and applying human concepts like thinking and reasoning to them. Djikstra wasn't saying it's functionally the same so it's just a semantic quibble. He was saying those words don't apply to machines. Just like we don't say submarines swim because that's how animals move through wat…

I posted a link to the original context. I don't know whether you are saying that I have taken it out of context or a more general statement of how other people have interpreted the quote. To me it is quite clear -- the question of whether machines can reason is just as meaningful as the question as to whether submarines can swim, which is to say not at all meaningful. Can a submarine accomplish by other means what a…

Of course it's meaningful! It would be highly meaningful if submarines could actually swim. And we are building submersible bots that swim, they just tend to be small and made of biological matter.

The problem with calling what AI does "reasoning" is both that there may be alternate paths to artificial reasoning we may be closing off by declaring victory, and the simple fact that you shouldn't call a thing something else, to avoid confusion.

Whatever AI is doing needs to be studied. Why muddy that study by pretending AI is your thinking buddy?

Re: Is AI reasoning right for the wrong reasons?

#227

Post by randomImmigrant »

nearbuy wrote: post #174
andy99 wrote: post #13

Back in the day it was a bit of a cliche to bring up “clever Hans”, the horse that could do math, when talking about machine learning. He couldn’t do math but he read some cues from his handler of pick the write answers, the handler iirc wasn’t in on it. The point of the story was that classifiers can be right for the wrong reasons and almost inevitably are. At least there’s zero guarantee that the reason for making…

LLMs are not classifiers. A classifier is an algorithm or neural net that assigns a label from a fixed set of labels to an input. You can broaden the definition of classifier to anything that internally divides its input space into regions, but that definition would include every neural network, whether biological or artificial. So it's not very meaningful, and certainly doesn't give any insight into how they differ…

It's pretty meaningful actually. Because biological neurons don't sit alone. There's a third player in every synapse: the astrocyte. And astrocytes form a large syncitium, a collection of cells that are "open" to each other through gap junctions. This syncitium has gotten more extensive over evolution, and human astrocytes (which are larger and more complex) when transplanted in mice, make the mice smarter at maze tasks etc.

So every neural network is actually entertained in an astrocytic syncitium. Every synapse is being "listened to" and modulated by it's astrocytic end foot (which forms the third arm of what we now call the tripartite synapse).

If neurons are about breaking down signals, the astrocytes are the slow, steady integrators that (potentially) can serve as the global workspace that scientists have been theorizing but not yet found.

So there actually is a problem with modern neural networks, in so far as comparisons with human brains go: they're missing the design spec for literally half of it.

Which is fine. Connectionist models come from a time when the height of neuroscience said neurons were the bees knees and glia were just support staff. Today, astrocytes, and other glia, are forcefully back in the conversation. And they don't seem to do anything like neural networks.

Re: Is AI reasoning right for the wrong reasons?

#228

Post by randomImmigrant »

zarzavat wrote: post #223
esikich wrote: post #152

Earlier quoted context omitted.

It seems like you are talking about Kant's "thing-in-itself". We can't know any more about an apple than an LLM can. We experience it through our faulty senses. We do not experience the actual apple. This is still open and debatable and is philosophy 101 stuff. I really dislike the confidence on this site, philosophically, that things are just obviously so. The philosophy of mind is massively open for interpretation…

> We can't know any more about an apple than an LLM can When I hold and taste an apple I know that the apple exists. Not an issue for apples, you might say, but there's plenty of real information that is not written down and is only derivable from our senses. Such as instructions for using car washes, apparently. Humans are naturally sceptical of descriptions, preferring first-hand experiences because descriptions ca…

Yep. They also don't experience time. They can count compute steps. They can look at the wall clock and tell you the time. But give them a task at 11:59 PM, and if they get it done by 12 AM, they won't know just a minute has passed.

Now, humans can count time fairly accurately if they focus, but we often lose focus. However, we have an internal subjective time that tracks the solar day excellently, and this LLMs lack entirely.

Timeless, disembodied abstractions. Impressive ones, but they are far from actual experience and thus limited.

Re: Is AI reasoning right for the wrong reasons?

#229

Post by randomImmigrant »

esikich wrote: post #168
drekipus wrote: post #154

Earlier quoted context omitted.

LLMs don't experience because they are word generators. They don't have senses. Some things are allowed to be obvious. It's a series of bits and bytes.

And our nerves are a series of electrochemical impulses. There are nerves that do optical processing before it hits the brain. It's not obvious, and that's what's frustrating around here. "Computers can't possibly experience things". I'd argue it's just as unlikely for meat to do it. Again, philosophy 101 stuff.

It's very obvious actually, because it isn't just nerves in the brain. There's an equal number of glial cells, which form a continuous network, or syncitium. Their activity is in the form of calcium waves, which are both responsive to and shape neuronal oscillations.

What biologists find frustrating is folks looking at one kind of dancer in the field and ignoring the rest of the spectacle, and calling it done.

Re: Is AI reasoning right for the wrong reasons?

#230

Post by mdp2021 »

ForHackernews wrote: post #215
mdp2021 wrote: post #210

Earlier quoted context omitted.

> How many hours did you spend on your phone last month? How many fewer hours would you have spent if the apps you interact with weren't designed to maximize "engagement"? ?? Very strictly zero? You are assuming we would install engagement-relevant third party apps on our palm computers?! I use my time exactly as I decide moment to moment. > fancy Eh?! > too smart to fall for Very ill framed. We are adults, we behave…

Millions of adults get hooked by gambling, social media, and AI that pretends to be a person. We need a better standard than "@mdp2021 thinks he's too grown up for this."

The standard is "do grow up; build a society that facilitates growing up; and do not hinder adults with rules for children - that is very much not a society".

And your insulting attitude disservices the loser more than the normal people: you know, there are people who can count, there are people who won't lie, there are people who do not take drugs... There do exist adults.

Post reply on HN