Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Case That A.I. Is Thinking

newyorker.com

Post reply on HN
Print view View original

461–470 of 1001 posts

Re: The Case That A.I. Is Thinking

#461

Post by AnIrishDuck »

layer8 wrote: post #371
tkz1312 wrote: post #274

Having seen LLMs so many times produce coherent, sensible and valid chains of reasoning to diagnose issues and bugs in software I work on, I am at this point in absolutely no doubt that they are thinking. Consciousness or self awareness is of course a different question, and ones whose answer seems less clear right now. Knee jerk dismissing the evidence in front of your eyes because you find it unbelievable that we c…

Sometimes after a night’s sleep, we wake up with an insight on a topic or a solution to a problem we encountered the day before. Did we “think” in our sleep to come up with the insight or solution? For all we know, it’s an unconscious process. Would we call it “thinking”? The term “thinking” is rather ill-defined, too bound to how we perceive our own wakeful thinking. When conversing with LLMs, I never get the feelin…

> Sometimes after a night’s sleep, we wake up with an insight on a topic or a solution to a problem we encountered the day before.

The current crop of models do not "sleep" in any way. The associated limitations on long term task adaptation are obvious barriers to their general utility.

> When conversing with LLMs, I never get the feeling that they have a solid grasp on the conversation. When you dig into topics, there is always a little too much vagueness, a slight but clear lack of coherence, continuity and awareness, a prevalence of cookie-cutter verbiage. It feels like a mind that isn’t fully “there” — and maybe not at all.

One of the key functions of REM sleep seems to be the ability to generalize concepts and make connections between "distant" ideas in latent space [1].

I would argue that the current crop of LLMs are overfit on recall ability, particularly on their training corpus. The inherent trade-off is that they are underfit on "conceptual" intelligence. The ability to make connections between these ideas.

As a result, you often get "thinking shaped objects", to paraphrase Janelle Shane [2]. It does feel like the primordial ooze of intelligence, but it is clear we still have several transformer-shaped breakthroughs before actual (human comparable) intelligence.

1. https://en.wikipedia.org/wiki/Why_We_Sleep 2. https://www.aiweirdness.com/

Re: The Case That A.I. Is Thinking

#462

Post by mxkopy »

There’s a way to talk about this stuff already. LLMs can “think” counterfactually on continuous data, just like VAEs [0], and are able to interpolate smoothly between ‘concepts’ or projections of the input data. This is meaningless when the true input space isn’t actually smooth. It’s system I, shallow-nerve psychomotor reflex type of thinking.

What LLMs can’t do is “think” counterfactually on discrete data. This is stuff like counting or adding integers. We can do this very naturally because we can think discretely very naturally, but LLMs are bad at this sort of thing because the underlying assumption behind gradient descent is that everything has a gradient (i.e. is continuous). They need discrete rules to be “burned in” [1] since minor perturbations are possible for and can affect continuous-valued weights.

You can replace “thinking” here with “information processing”. Does an LLM “think” any more or less than say, a computer solving TSP on a very large input? Seeing as we can reduce the former to the latter I wouldn’t say they’re really at all different. It seems like semantics to me.

In either case, counterfactual reasoning is good evidence of causal reasoning, which is typically one part of what we’d like AGI to be able to do (causal reasoning is deductive, the other part is inductive; this could be split into inference/training respectively but the holy grail is having these combined as zero-shot training). Regression is a basic form of counterfactual reasoning, and DL models are basically this. We don’t yet have a meaningful analogue for discrete/logic puzzley type of problems, and this is the area where I’d say that LLMs don’t “think”.

This is somewhat touched on in GEB and I suspect “Fluid Concepts and Creative Analogies” as well.

[0] https://human-interpretable-ai.github.io/assets/pdf/5_Genera...

[1] https://www.sciencedirect.com/science/article/pii/S089360802...

Re: The Case That A.I. Is Thinking

#463

Post by jameswhitford »

ale wrote: post #21

This reads like 2022 hype. It's like people stil do not understand that there's a correlation between exaggerating AI's alleged world-threatening capabilities and AI companies' market share value – and guess who's doing the hyping.

Who would not want to say their product is the second coming of Christ if they could.

Re: The Case That A.I. Is Thinking

#464

Post by noiv »

tkz1312 wrote: post #274

Having seen LLMs so many times produce coherent, sensible and valid chains of reasoning to diagnose issues and bugs in software I work on, I am at this point in absolutely no doubt that they are thinking. Consciousness or self awareness is of course a different question, and ones whose answer seems less clear right now. Knee jerk dismissing the evidence in front of your eyes because you find it unbelievable that we c…

Different PoV: You have a local bug and ask the digital hive mind for a solution, but someone already solved the issue and their solution was incorporated... LLMs are just very effficient at compressing billions of solutions into a few GB.

Try to ask something no one ever came up with a solution so far.

Re: The Case That A.I. Is Thinking

#465

Post by notepad0x90 »

tkz1312 wrote: post #274

Having seen LLMs so many times produce coherent, sensible and valid chains of reasoning to diagnose issues and bugs in software I work on, I am at this point in absolutely no doubt that they are thinking. Consciousness or self awareness is of course a different question, and ones whose answer seems less clear right now. Knee jerk dismissing the evidence in front of your eyes because you find it unbelievable that we c…

I don't get why you would say that. it's just auto-completing. It cannot reason. It won't solve an original problem for which it has no prior context to "complete" an approximated solution with. you can give it more context and more data,but you're just helping it complete better. it does not derive an original state machine or algorithm to solve problems for which there are no obvious solutions. it instead approximates a guess (hallucination).

Consciousness and self-awareness are a distraction.

Consider that for the exact same prompt and instructions, small variations in wording or spelling change its output significantly. If it thought and reasoned, it would know to ignore those and focus on the variables and input at hand to produce deterministic and consistent output. However, it only computes in terms of tokens, so when a token changes, the probability of what a correct response would look like changes, so it adapts.

It does not actually add 1+2 when you ask it to do so. it does not distinguish 1 from 2 as discrete units in an addition operation. but it uses descriptions of the operation to approximate a result. and even for something so simple, some phrasings and wordings might not result in 3 as a result.

Re: The Case That A.I. Is Thinking

#466

Post by khafra »

paxys wrote: post #100
educasean wrote: post #69

The debate around whether or not transformer-architecture-based AIs can "think" or not is so exhausting and I'm over it. What's much more interesting is the question of "If what LLMs do today isn't actual thinking, what is something that only an actually thinking entity can do that LLMs can't?". Otherwise we go in endless circles about language and meaning of words instead of discussing practical, demonstrable capabi…

Don't be sycophantic. Disagree and push back when appropriate. Come up with original thought and original ideas. Have long term goals that aren't programmed by an external source. Do something unprompted. The last one IMO is more complex than the rest, because LLMs are fundamentally autocomplete machines. But what happens if you don't give them any prompt? Can they spontaneously come up with something, anything, with…

Note that model sycophancy is caused by RLHF. In other words: Imagine taking a human in his formative years, and spending several subjective years rewarding him for sycophantic behavior and punishing him for candid, well-calibrated responses.

Now, convince him not to be sycophantic. You have up to a few thousand words of verbal reassurance to do this with, and you cannot reward or punish him directly. Good luck.

Re: The Case That A.I. Is Thinking

#467

Post by czl »

Terr_ wrote: post #367
czl wrote: post #240

Earlier quoted context omitted.

> So how is it therefore not actual thinking? Many consider "thinking" something only animals can do, and they are uncomfortable with the idea that animals are biological machines or that life, consciousness, and thinking are fundamentally machine processes. When an LLM generates chain-of-thought tokens, what we might casually call “thinking,” it fills its context window with a sequence of tokens that improves its ab…

Well put, and if it doesn't notice/collapse under introduced contradictions, that's evidence it's not the kind of reasoning we were hoping for. The "real thing" is actually brittle when you do it right.

Human reasoning is, in practice, much closer to statistical association than to brittle rule-following. The kind of strict, formal deduction we teach in logic courses is a special, slow mode we invoke mainly when we’re trying to check or communicate something, not the default way our minds actually operate.

Everyday reasoning is full of heuristics, analogies, and pattern matches: we jump to conclusions, then backfill justification afterward. Psychologists call this “post hoc rationalization,” and there’s plenty of evidence that people form beliefs first and then search for logical scaffolding to support them. In fact, that’s how we manage to think fluidly at all; the world is too noisy and underspecified for purely deductive inference to function outside of controlled systems.

Even mathematicians, our best examples of deliberate, formal thinkers, often work this way. Many major proofs have been discovered intuitively and later found to contain errors that didn’t actually invalidate the final result. The insight was right, even if the intermediate steps were shaky. When the details get repaired, the overall structure stands. That’s very much like an LLM producing a chain of reasoning tokens that might include small logical missteps yet still landing on the correct conclusion: the “thinking” process is not literal step-by-step deduction, but a guided traversal through a manifold of associations shaped by prior experience (or training data, in the model’s case).

So if an LLM doesn’t collapse under contradictions, that’s not necessarily a bug; it may reflect the same resilience we see in human reasoning. Our minds aren’t brittle theorem provers; they’re pattern-recognition engines that trade strict logical consistency for generalization and robustness. In that sense, the fuzziness is the strength.

Re: The Case That A.I. Is Thinking

#468

Post by XenophileJKO »

aydyn wrote: post #414
bongodongobob wrote: post #411

Earlier quoted context omitted.

Ridiculous. I use it daily and get meaningful, quality results. Learn to use the tools.

Learn to work on interesting problems? If the problem you are working on is novel and hard, the AI will stumble. Generalizing your experience to everyone else's betrays a lack of imagination.

I'm genuinely curious what you work on that is so "novel" that an LLM doesn't work well on?

I feel like so little is TRUELY novel. Almost everything is built on older concepts and to some degree expertise can be applied or repurposed.

Re: The Case That A.I. Is Thinking

#469

Post by xanderlewis »

notepad0x90 wrote: post #465
tkz1312 wrote: post #274

Having seen LLMs so many times produce coherent, sensible and valid chains of reasoning to diagnose issues and bugs in software I work on, I am at this point in absolutely no doubt that they are thinking. Consciousness or self awareness is of course a different question, and ones whose answer seems less clear right now. Knee jerk dismissing the evidence in front of your eyes because you find it unbelievable that we c…

I don't get why you would say that. it's just auto-completing. It cannot reason. It won't solve an original problem for which it has no prior context to "complete" an approximated solution with. you can give it more context and more data,but you're just helping it complete better. it does not derive an original state machine or algorithm to solve problems for which there are no obvious solutions. it instead approxima…

> I don't get why you would say that.

Because it's hard to imagine the sheer volume of data it's been trained on.

Re: The Case That A.I. Is Thinking

#470

Post by madaxe_again »

notepad0x90 wrote: post #465
tkz1312 wrote: post #274

Having seen LLMs so many times produce coherent, sensible and valid chains of reasoning to diagnose issues and bugs in software I work on, I am at this point in absolutely no doubt that they are thinking. Consciousness or self awareness is of course a different question, and ones whose answer seems less clear right now. Knee jerk dismissing the evidence in front of your eyes because you find it unbelievable that we c…

I don't get why you would say that. it's just auto-completing. It cannot reason. It won't solve an original problem for which it has no prior context to "complete" an approximated solution with. you can give it more context and more data,but you're just helping it complete better. it does not derive an original state machine or algorithm to solve problems for which there are no obvious solutions. it instead approxima…

The vast majority of human “thinking” is autocompletion.

Any thinking that happens with words is fundamentally no different to what LLMs do, and everything you say applies to human lexical reasoning.

One plus one equals two. Do you have a concept of one-ness, or two-ness, beyond symbolic assignment? Does a cashier possess number theory? Or are these just syntactical stochastic rules?

I think the problem here is the definition of “thinking”.

You can point to non-verbal models, like vision models - but again, these aren’t hugely different from how we parse non-lexical information.

Post reply on HN