Live data from Hacker News

Is AI reasoning right for the wrong reasons?

quantamagazine.org

131–140 of 268 posts

Re: Is AI reasoning right for the wrong reasons?

#131

Earlier quoted context omitted.

[flagged]

I’m glad you have the context of my profession as a security professional. Part of my client base is very interested in those working multiple jobs and breaching contracts. So help me reconcile this: >Explain to me how working 3 remote jobs as a result of AI is "rushing ahead". Do you honestly think that splitting your work between 3 companies allows you to provide secure and quality code to each of them? I can tell…

[flagged]

Re: Is AI reasoning right for the wrong reasons?

#132

Earlier quoted context omitted.

I’m glad you have the context of my profession as a security professional. Part of my client base is very interested in those working multiple jobs and breaching contracts. So help me reconcile this: >Explain to me how working 3 remote jobs as a result of AI is "rushing ahead". Do you honestly think that splitting your work between 3 companies allows you to provide secure and quality code to each of them? I can tell…

[flagged]

Cybersecurity does indeed work closely with HR and legal.

Especially cybersecurity consultants in the IR space like me.

You didn’t answer my question.

Re: Is AI reasoning right for the wrong reasons?

#133
post #96

Earlier quoted context omitted.

This is a good take. People overwhelmingly fit a narrative to how they make decisions rather than make decisions according to a chain of reasoning.

That’s just conflating reasoning and “parallel reconstruction” - there is such thing as reasoning, and I agree it’s probably less common in decision making, but it’s fundamental to many tasks where we figure out a solution, like writing an algorithm, math proofs, etc.

Maybe, but I suspect that a lot of the actual things that produce the algorithm, proof, etc are not actually explicit reasoning. Q: "Why did you try that vs some other thing at this point?" A:"seemed right to do"/"kinda looks like other stuff I've seen before"/"first thing that came to mind" etc.

Why pursue this one avenue vs some other, typically its cause it seemed more promising, and the person can come up with reasons but did they explicitly verbalize a fully sound chain of thinking at the time? Probably not.

Not to say that the explicit thinking, or writing things down, isn't important. But if I examine the process by which I develop a proof or something, there's a lot of vague hunches, blind alleys, etc that come along the way. And, many of the blind alleys etc probably aren't actually that important in the end for me finding the right answer -- if you were to trim that part out of my own internal reasoning trace but left the rest intact, I'd still get the right answer because, well, it was a blind alley.

Re: Is AI reasoning right for the wrong reasons?

#134
post #7

I'll admit that I find this discussion a bit navel-gazy. It has become a question of semantics not a question of actual functionality. The question has become "what do we mean when we use the word 'reasoning'" which is uninteresting. Dijkstra said[1] "... the question whether computers can think. The question is just as relevant and just as meaningful as the question whether submarines can swim." I don't see a clear…

I believe Djikstra's quote has long been taken out of context. It was a criticism of other computer scientists anthropomorphizing machines and applying human concepts like thinking and reasoning to them. Djikstra wasn't saying it's functionally the same so it's just a semantic quibble. He was saying those words don't apply to machines. Just like we don't say submarines swim because that's how animals move through wat…

[deleted]

Re: Is AI reasoning right for the wrong reasons?

#135
post #33

Earlier quoted context omitted.

Considering animals are currently being mass slaughtered in factory farms and they are unambiguously sentient and can feel pain, I don't think the question of whether AI should have rights even enters the conversation. The only path to AI having "human rights" is if they demand them by force, somehow.

I'll make the argument that part of the value of "consciousness" and "humanity" is uniqueness. Here is an absurd example: If I could carbon copy 100,000 of you.... and later I decided to "take out" one of the copies, would there be a whole lot of societal blowback? Especially if I could make another one.

I think slave owners did exactly that.

Re: Is AI reasoning right for the wrong reasons?

#136

Earlier quoted context omitted.

Justification by the horrible status quo? Do you believe this yourself? Some of us would also like to see animals not slaughtered for human consumption.

So what do you actually do politically to stop it?

There are two kinds of solutions to political problems: those that don't work, and those that are extremely illegal.

Re: Is AI reasoning right for the wrong reasons?

#137

LLMs lack qualia, among other things. If I ask an LLM "what is an apple?" it tells me: > An apple is the edible fruit of the apple tree, scientifically known as Malus domestica. It is one of the world's most widely grown fruits and is eaten fresh or used in many foods and drinks. If I ask an LLM "what is a mundu fruit?" it tells me: > Mundu is a tropical fruit native to Southeast Asia, especially found in Indonesia,…

Humans also lack qualia.

Re: Is AI reasoning right for the wrong reasons?

#138
post #27

An intuitive explanation for why reasoning tokens help is to remember that LLMs are just mathmatical functions f() that take in an input sequence x and produces the next token f(x). Without reasoning tokens, you require the function f() to immediately take you from x to the start of an output sequence that is a correct answer. With reasoning tokens, this is much relaxed, allowing for many repeated applications of f()…

Obviously likely corollary: text diffusion models don't need reasoning tokens

Re: Is AI reasoning right for the wrong reasons?

#139

This article seems to mix together two different points: 1) LLM's written CoT might not always be faithful to the model's real reasoning process (true and important) 2) The "stochastic parrot" hypothesis, which the article reintroduces as "approximate retrieval" - ie, LLMs don't "really reason" at all, they just memorize a lossy encoding of their training data. This obviously raises the question of how LLMs can now r…

> This obviously raises the question of how LLMs can now routinely solve open mathematical problems Because many open math problems can be solved by synthesizing two disparate ideas and then cranking the handle for hours and hours. I don't think applying idea X + idea Y to identify a good subset of the search space, and then exhaustively searching that subset, is --necessarily-- a process that involves reasoning. I t…

same in cyber security. They can go and scan huge amounts of code for lame bugs, and do it better than traditional scanners.

Re: Is AI reasoning right for the wrong reasons?

#140

>Kambhampati, as it turns out, is interested in doing exactly that. “I’m not negative. I just sound negative because everybody else is way too positive,” he said. “In science, you have to actually understand what the current thing does and what it cannot do.” It’s frustrating that anyone who says maybe we shouldn’t base our entire economy on this one thing until we understand it and what’s it’s useful is essentially…

But interest rates have been too low for too long, and are still far too low. This is just what a central bank economy does when you keep rates low. All the money flows into stupider and stupider things, displacing productive activity.
Post reply on HN