Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Case That A.I. Is Thinking

newyorker.com

Post reply on HN
Print view View original

531–540 of 1001 posts

Re: The Case That A.I. Is Thinking

#531

Post by popalchemist »

layer8 wrote: post #371
tkz1312 wrote: post #274

Having seen LLMs so many times produce coherent, sensible and valid chains of reasoning to diagnose issues and bugs in software I work on, I am at this point in absolutely no doubt that they are thinking. Consciousness or self awareness is of course a different question, and ones whose answer seems less clear right now. Knee jerk dismissing the evidence in front of your eyes because you find it unbelievable that we c…

Sometimes after a night’s sleep, we wake up with an insight on a topic or a solution to a problem we encountered the day before. Did we “think” in our sleep to come up with the insight or solution? For all we know, it’s an unconscious process. Would we call it “thinking”? The term “thinking” is rather ill-defined, too bound to how we perceive our own wakeful thinking. When conversing with LLMs, I never get the feelin…

There is simply put no ongoing process and no feedback loop. The model does not learn. The cognition ends when the inference cycle ends. It's not thinking, it just produces output that looks similar to the output of thinking. But the process by which it does that is wholly unreleated.

Re: The Case That A.I. Is Thinking

#532

Post by josefx »

becquerel wrote: post #487
josefx wrote: post #485

Earlier quoted context omitted.

Counterpoint: The seahorse emoji. The output repeats the same simple pattern of giving a bad result and correcting it with another bad result until it runs out of attempts. There is no reasoning, no diagnosis, just the same error over and over again within a single session.

A system having terminal failure modes doesn't inherently negate the rest of the system. Human intelligences fall prey to plenty of similarly bad behaviours like addiction.

I never met an addicted person that could be reduced to a simple while(true) print("fail") loop.

Re: The Case That A.I. Is Thinking

#533

Post by player1234 »

dimator wrote: post #431
aydyn wrote: post #414

Earlier quoted context omitted.

Learn to work on interesting problems? If the problem you are working on is novel and hard, the AI will stumble. Generalizing your experience to everyone else's betrays a lack of imagination.

This is my experience. For rote generation, it's great, saves me from typing out the same boilerplate unit test bootstrap, or refactoring something that exists, etc. Any time I try to get a novel insight, it flails wildly, and nothing of value comes out. And yes, I am prompting incrementally and building up slowly.

[flagged]

Re: The Case That A.I. Is Thinking

#534

Post by kkapelon »

lordnacho wrote: post #523
tkz1312 wrote: post #274

Having seen LLMs so many times produce coherent, sensible and valid chains of reasoning to diagnose issues and bugs in software I work on, I am at this point in absolutely no doubt that they are thinking. Consciousness or self awareness is of course a different question, and ones whose answer seems less clear right now. Knee jerk dismissing the evidence in front of your eyes because you find it unbelievable that we c…

I agree with you. If you took a Claude session into a time machine to 2019 and called it "rent a programmer buddy," how many people would assume it was a human? The only hint that it wasn't a human programmer would be things where it was clearly better: it types things very fast, and seems to know every language. You can set expectations in the way you would with a real programmer: "I have this script, it runs like t…

> If you took a Claude session into a time machine to 2019 and called it "rent a programmer buddy," how many people would assume it was a human?

Depends on the users. Junior devs might be fooled. Senior devs would quickly understand that something is wrong.

Re: The Case That A.I. Is Thinking

#535

Post by gloosx »

madaxe_again wrote: post #470
notepad0x90 wrote: post #465

Earlier quoted context omitted.

I don't get why you would say that. it's just auto-completing. It cannot reason. It won't solve an original problem for which it has no prior context to "complete" an approximated solution with. you can give it more context and more data,but you're just helping it complete better. it does not derive an original state machine or algorithm to solve problems for which there are no obvious solutions. it instead approxima…

The vast majority of human “thinking” is autocompletion. Any thinking that happens with words is fundamentally no different to what LLMs do, and everything you say applies to human lexical reasoning. One plus one equals two. Do you have a concept of one-ness, or two-ness, beyond symbolic assignment? Does a cashier possess number theory? Or are these just syntactical stochastic rules? I think the problem here is the d…

> Any thinking that happens with words is fundamentally no different from what LLMs do.

This is such a wildly simplified and naive claim. "Thinking with words" happens inside a brain, not inside a silicon circuit with artificial neurons bolted in place. The brain is plastic, it is never the same from one moment to the next. It does not require structured input, labeled data, or predefined objectives in order to learn "thinking with words." The brain performs continuous, unsupervised learning from chaotic sensory input to do what it does. Its complexity and efficiency are orders of magnitude beyond that of LLM inference. Current models barely scratch the surface of that level of complexity and efficiency.

> Do you have a concept of one-ness, or two-ness, beyond symbolic assignment?

Obviously we do. The human brain's idea of "one-ness" or "two-ness" is grounded in sensory experience — seeing one object, then two, and abstracting the difference. That grounding gives meaning to the symbol, something LLMs don't have.

Re: The Case That A.I. Is Thinking

#536

Post by ako »

popalchemist wrote: post #531
layer8 wrote: post #371

Earlier quoted context omitted.

Sometimes after a night’s sleep, we wake up with an insight on a topic or a solution to a problem we encountered the day before. Did we “think” in our sleep to come up with the insight or solution? For all we know, it’s an unconscious process. Would we call it “thinking”? The term “thinking” is rather ill-defined, too bound to how we perceive our own wakeful thinking. When conversing with LLMs, I never get the feelin…

There is simply put no ongoing process and no feedback loop. The model does not learn. The cognition ends when the inference cycle ends. It's not thinking, it just produces output that looks similar to the output of thinking. But the process by which it does that is wholly unreleated.

Most AI tooling is shipped with a feedback loop around the LLM. The quality of Claude Code for example lies in the feedback loop it provides on your code. Maybe the LLM itself isn't thinking, but the Agent which ships an LLM plus feedback loop definitely shows thinking qualities.

Just now in an debugging session with claude code:

  * let me read this file...
  * let me read this file...
  * I think there's a caching issue with the model after dropping the module. Let me check if there's a save or reload needed after DROP MODULE. First, let me verify something:
  * creates a bash/javascript script to verify its assumption
  * runs the script (after review and approval)
  * Aha! I found the problem! Look at the output...
How is this not thinking?

Re: The Case That A.I. Is Thinking

#537

Post by keiferski »

tkz1312 wrote: post #274

Having seen LLMs so many times produce coherent, sensible and valid chains of reasoning to diagnose issues and bugs in software I work on, I am at this point in absolutely no doubt that they are thinking. Consciousness or self awareness is of course a different question, and ones whose answer seems less clear right now. Knee jerk dismissing the evidence in front of your eyes because you find it unbelievable that we c…

I don’t see how being critical of this is a knee jerk response.

Thinking, like intelligence and many other words designating complex things, isn’t a simple topic. The word and concept developed in a world where it referred to human beings, and in a lesser sense, to animals.

To simply disregard that entire conceptual history and say, “well it’s doing a thing that looks like thinking, ergo it’s thinking” is the lazy move. What’s really needed is an analysis of what thinking actually means, as a word. Unfortunately everyone is loathe to argue about definitions, even when that is fundamentally what this is all about.

Until that conceptual clarification happens, you can expect endless messy debates with no real resolution.

“For every complex problem there is an answer that is clear, simple, and wrong.” - H. L. Mencken

Re: The Case That A.I. Is Thinking

#538

Post by dns_snek »

MrScruff wrote: post #520
dns_snek wrote: post #495

Earlier quoted context omitted.

No, that's decidedly not what is happening here. One is saying "I've seen an LLM spectacularly fail at basic reasoning enough times to know that LLMs don't have a general ability to think" (but they can sometimes reproduce the appearance of doing so). The other is trying to generalize "I've seen LLMs produce convincing thought processes therefore LLMs have the general ability to think" (and not just occasionally repr…

When we say "think" in this context, do we just mean generalize? LLMs clearly generalize (you can give one a problem that is not exactly in it's training data and it can solve it), but perhaps not to the extent a human can. But then we're talking about degrees. If it was able to generalize at a higher level of abstraction maybe more people would regard it as "thinking".

I meant it in the same way the previous commenter did:

> Having seen LLMs so many times produce incoherent, nonsensical and invalid chains of reasoning... LLMs are little more than RNGs. They are the tea leaves and you read whatever you want into them.

Of course LLMs are capable of generating solutions that aren't in their training data sets but they don't arrive at those solutions through any sort of rigorous reasoning. This means that while their solutions can be impressive at times they're not reliable, they go down wrong paths that they can never get out of and they become less reliable the more autonomy they're given.

Re: The Case That A.I. Is Thinking

#539

Post by Terr_ »

czl wrote: post #467
Terr_ wrote: post #367

Earlier quoted context omitted.

Well put, and if it doesn't notice/collapse under introduced contradictions, that's evidence it's not the kind of reasoning we were hoping for. The "real thing" is actually brittle when you do it right.

Human reasoning is, in practice, much closer to statistical association than to brittle rule-following. The kind of strict, formal deduction we teach in logic courses is a special, slow mode we invoke mainly when we’re trying to check or communicate something, not the default way our minds actually operate. Everyday reasoning is full of heuristics, analogies, and pattern matches: we jump to conclusions, then backfill…

> The kind of strict, formal deduction we teach in logic courses is a special, slow mode

Yes, but that seems like moving the goalposts.

The stricter blends of reasoning are what everybody is so desperate to evoke from LLMs, preferably along with inhuman consistency, endurance, and speed. Just imagine the repercussions if a slam-dunk paper came out tomorrow, which somehow proved the architectures and investments everyone is using for LLMs are a dead-end for that capability.

Re: The Case That A.I. Is Thinking

#540

Post by mrweasel »

jdauriemma wrote: post #229
mrob wrote: post #142

I don't believe LLMs can be conscious during inference because LLM inference is just repeated evaluation of a deterministic [0] pure function. It takes a list of tokens and outputs a set of token probabilities. Any randomness is part of the sampler that selects a token based on the generated probabilities, not the LLM itself. There is no internal state that persists between tokens [1], so there can be no continuity o…

I don't think the author is saying that LLMs are conscious or alive.

It would be kinda hilarious if the result of all this LLM research is that humans are basically just LLMs with more sensors and a long history.
Post reply on HN