Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is AI reasoning right for the wrong reasons?

quantamagazine.org

Post reply on HN
Print view View original

211–220 of 268 posts

Re: Is AI reasoning right for the wrong reasons?

#211

Post by solarkraft »

thesmtsolver2 wrote: post #172
kgeist wrote: post #167

Transformers lack recursion and are limited by the network's fixed depth, so "reasoning", IMHO, is basically a way to emulate deeper recursion. As we go through the layers, concepts are pattern-matched and refined, but at some point we have to stop and cannot refine them any further (no more layers). Usually, this refinement continues during the generation of the next token (the previous intermediate results needed t…

Don't humans make mistakes too? Never understood this argument. Humans get simple multiplication wrong often, so ok for computers to make mistakes multiplying numbers?

Yes, this would be acceptable too, if that is all we had. We would build systems do deal with this unreliability, just like we have with humans.

Re: Is AI reasoning right for the wrong reasons?

#212

Post by [deleted] »

zarzavat wrote: post #83

LLMs lack qualia, among other things. If I ask an LLM "what is an apple?" it tells me: > An apple is the edible fruit of the apple tree, scientifically known as Malus domestica. It is one of the world's most widely grown fruits and is eaten fresh or used in many foods and drinks. If I ask an LLM "what is a mundu fruit?" it tells me: > Mundu is a tropical fruit native to Southeast Asia, especially found in Indonesia,…

[deleted]

Re: Is AI reasoning right for the wrong reasons?

#213

Post by bluejay2387 »

thesmtsolver2 wrote: post #173
andrewla wrote: post #7

I'll admit that I find this discussion a bit navel-gazy. It has become a question of semantics not a question of actual functionality. The question has become "what do we mean when we use the word 'reasoning'" which is uninteresting. Dijkstra said[1] "... the question whether computers can think. The question is just as relevant and just as meaningful as the question whether submarines can swim." I don't see a clear…

It is not semantics. For decades, logic and CS researchers have known what reasoning is. LLM folks suddenly can’t claim an approximation of that is what constitutes full scale reasoning just because they can achieve only an approximation. Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t hav…

"For decades, logic and CS researchers have known what reasoning is." ... this is a fairly significant overstatement. There is not complete agreement on this term and our understanding continues to evolve. The models don't have to think like humans to think.

Saying that LLM's only offer an 'approximation' of reasoning is also an overstatement as it is not a resolved topic.

But to the original point, its not exactly just semantics if thought traces are not doing the job that they were originally thought to do. There is value in knowing how these things actually work. If chain of thought is just grounding the latent space and not directly contributing to the process of generating a response it has implications on how we test and verify the reliability of models if nothing else... doesn't mean they aren't useful but it definitely impacts many of the tools we could have to evaluate their performance.

Re: Is AI reasoning right for the wrong reasons?

#215

Post by ForHackernews »

mdp2021 wrote: post #210
ForHackernews wrote: post #205

Earlier quoted context omitted.

That's a lot of fancy words to say, "I think I'm too smart to fall for the same tricks that work on other humans." How many hours did you spend on your phone last month? How many fewer hours would you have spent if the apps you interact with weren't designed to maximize "engagement"?

> How many hours did you spend on your phone last month? How many fewer hours would you have spent if the apps you interact with weren't designed to maximize "engagement"? ?? Very strictly zero? You are assuming we would install engagement-relevant third party apps on our palm computers?! I use my time exactly as I decide moment to moment. > fancy Eh?! > too smart to fall for Very ill framed. We are adults, we behave…

Millions of adults get hooked by gambling, social media, and AI that pretends to be a person.

We need a better standard than "@mdp2021 thinks he's too grown up for this."

Re: Is AI reasoning right for the wrong reasons?

#216

Post by Yopolo »

verbify wrote: post #145
petesergeant wrote: post #103

Earlier quoted context omitted.

Aren't humans also classifiers? Where, precisely, is the dividing line between a sufficiently large model and an intelligence?

Is that all humans are? When clockwork was the frontier, people thought the brain was akin to clockwork (which is why the mechanical turk fooled people). By analogy, now that classifiers are the frontier, we think the brain is a classifier. Our model of the mind is whatever the most complex artifact of our era is . Just to be clear: I do think the brain classifies. But I wonder if it does other operations as well tha…

The mechanical clock or a von-neumann based computer were never as similiar as neural networks.

We can find similiar structures in brain and in neural networks.

The real question is only if the complexity of the structure of the brain is critical and how we can mimik it or if we can make it appear through learning architecture.

Re: Is AI reasoning right for the wrong reasons?

#217

Post by Yopolo »

zarzavat wrote: post #83

LLMs lack qualia, among other things. If I ask an LLM "what is an apple?" it tells me: > An apple is the edible fruit of the apple tree, scientifically known as Malus domestica. It is one of the world's most widely grown fruits and is eaten fresh or used in many foods and drinks. If I ask an LLM "what is a mundu fruit?" it tells me: > Mundu is a tropical fruit native to Southeast Asia, especially found in Indonesia,…

Thats an assumption you made up.

Your 'experience' is probably an emerging feature of your complexity and not the reason for some qualia.

Re: Is AI reasoning right for the wrong reasons?

#218

Post by drekipus »

esikich wrote: post #189
drekipus wrote: post #171

Earlier quoted context omitted.

I think you're trying to degrade the entire school of philosophy down to your "philosophy 101" understanding and then shoving it where it doesn't fit in order to feel like this is all complicated. Philosophy is a intrinsically human thing. It has to be. That is the whole point. There really is nothing else. Plants feel and experience things too. But there's no philosophy of the plant experience from the point of view…

"Philosophy is an intrinsically human thing. It has to be." What is philosophy? Thinking? Reasoning? Thinking about thinking? You're asserting that only a human brain can think and reason? Surely other animals can as well. Couldn't aliens as well? Why can't this crazy electronic box? Where do we draw the line? Do dolphins not have internal experiences? Dogs? Cats? Mice? Fish? Worms? Bugs? Amoebas? We don't even reall…

I suggest you print out this exchange and spend a week away from the computer, then re-read what was printed.

I'm not trying to be rude but I gave you the answers. Alien philosophy is irrelevant to us because we're not aliens. Tree philosophy is irrelevant to us because we're not trees. Dolphin philosophy is irrelevant to us because we're not dolphins, etc.

Everything else you mentioned is "hallucinatory" and you need to probably have a break from screens for a while.

As a legitimate exercise, go ponder the philosophy of a bash script. Think about philosophy of trees in a forest vs trees in a plantation. You'll find it and see how silly you're being

Re: Is AI reasoning right for the wrong reasons?

#219

Post by kevinwang »

thesmtsolver2 wrote: post #173
andrewla wrote: post #7

I'll admit that I find this discussion a bit navel-gazy. It has become a question of semantics not a question of actual functionality. The question has become "what do we mean when we use the word 'reasoning'" which is uninteresting. Dijkstra said[1] "... the question whether computers can think. The question is just as relevant and just as meaningful as the question whether submarines can swim." I don't see a clear…

It is not semantics. For decades, logic and CS researchers have known what reasoning is. LLM folks suddenly can’t claim an approximation of that is what constitutes full scale reasoning just because they can achieve only an approximation. Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t hav…

> It is not semantics. For decades, logic and CS researchers have known what reasoning is.

Curious what this is!

Re: Is AI reasoning right for the wrong reasons?

#220

Post by Borealid »

mdp2021 wrote: post #202
Borealid wrote: post #128

Earlier quoted context omitted.

No, we only need to know how the thing-building thing works. We don't have to understand the resulting product at all. Genetic algorithms are an example - we understand the evolution function, not what evolves from it.

You must have misunderstood the statement: «In order to properly engineer things, we must know how they work»: of course they may work anyway, of course there may be emergent properties (as was explicitly written), but the deontic part of knowledge augmentation, the scientific part, is missing. AI is full of non deterministic devices like genetic algorithms, but there exists a problem of transparency which is paramou…

No, I think you misunderstood me. We engineer the model-training algorithm and then we USE the model. The model is a mess of random gobbledegook that happens to be the best optimizer for the training algorithm's reward function.

This is still science, in the same way it's still science when we breed cows to produce more milk without understanding the full bovine genome.

If something isn't right with the model we pour extra effort into our science - the training algorithm - and train a new one.

At no point does anything ever require peering into the slurry of random digits that make up the model itself, any more than using a computer requires understanding its DRAM training values computed at run time, or any more than pouring water requires understanding the laminar flow along the pitcher's surface (an unsolved math problem!).

Post reply on HN