Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Case That A.I. Is Thinking

newyorker.com

Post reply on HN
Print view View original

881–890 of 1001 posts

Re: The Case That A.I. Is Thinking

#881

Post by IanCal »

notepad0x90 wrote: post #671
IanCal wrote: post #475

Earlier quoted context omitted.

> it's just auto-completing. It cannot reason Auto completion just means predicting the next thing in a sequence. This does not preclude reasoning. > I don't get why you would say that. Because I see them solve real debugging problems talking through the impact of code changes or lines all the time to find non-obvious errors with ordering and timing conditions on code they’ve never seen before.

> This does not preclude reasoning. It does not imply it either. to claim reasoning you need evidence. it needs to reliably NOT hallucinate results for simple conversations for example (if it has basic reasoning). > Because I see them solve real debugging problems talking through the impact of code changes or lines all the time to find non-obvious errors with ordering and timing conditions on code they’ve never seen…

> It does not imply it either.

Right, it's irrelevant to the question of whether they can reason.

> to claim reasoning you need evidence

Frankly I have no idea what most people are talking about when they use the term and say these models can't do it. It seems to be a similarly hand-wavey exercise as when people talk about thinking or understanding.

> it needs to reliably NOT hallucinate results for simple conversations for example (if it has basic reasoning).

That's not something I commonly see in frontier models.

Again this doesn't seem related to reasoning. What we call hallucinations would be seen in something that could reason but had a fallible memory. I remember things incorrectly and I can reason.

> it does not actually take your code and debug it

It talks through the code (which it has not seen) and process step by step, can choose to add logging, run it, go through the logs, change what it thinks is happening and repeat. It can do this until it explains what is happening, creates test cases to show the problem and what triggers it, fixes it and shows the tests pass.

If that's not debugging the code I really don't know what to call it.

Re: The Case That A.I. Is Thinking

#882

Post by danielmarkbruce »

mft_ wrote: post #123

Personal take: LLMs are probably part of the answer (to AGI?) but are hugely handicapped by their current architecture: the only time that long-term memories are formed is during training, and everything after that (once they're being interacted with) sits only in their context window, which is the equivalent of fungible, fallible, lossy short-term memory. [0] I suspect that many things they currently struggle with c…

but nobody is using LLMs all by themselves.

Long-term memory is stored outside the model. In fact, Andrej Karpathy recently talked about the idea that it would be great if we could get LLMs to not know any facts, and that humans poor memory might be a feature which helps with generalization rather than a bug.

Re: The Case That A.I. Is Thinking

#883

Post by gilbetron »

nickledave wrote: post #837
gilbetron wrote: post #752

Earlier quoted context omitted.

This is one of the least curious posts I've seen on HN. We have been thinking about thinking for millenia, and no, Buddhists don't have it figure out. Nobody does. LLMs are the most significant advancement in "thinking science" in a long, long time. It is clear that they are doing something a lot like thinking, if it is not thinking. They seem to think more than most people I know, including the person I'm responding…

> "thinking science" If you are really curious, I invite you to read this cognitive science paper, "Modern Alchemy: Neurocognitive Reverse Engineering": https://philsci-archive.pitt.edu/25289/1/GuestEtAl2025.pdf Note the quote at the top from Abeba Birhane: > We can only presume to build machines like us once we see ourselves as machines first. It reminds me of your comment that > [LLMs] seem to think more than most…

The paper is yet another in a long line of, "humans are special, computers can't replicate them". Such thinking has been a part of the fields for decades and decades, I had arguments about them when I was in college with my professors (such as John Holland, "creator" of genetic algorithms). That's the whole reason LLMs are so interesting, they are the first time we've captured something very much like thinking and reasoning. It can do many of the things long thought to be the sole purview of humans. That's why anyone that knows anything about the field of AI is astonished by them.

The "intellectual e-waste from Silicon Valley" has produced something amazing, the likes of which we've never seen. (Built on decades of curious people in the AI, neuroscience, computer science, and other fields, of course).

Re: The Case That A.I. Is Thinking

#884

Post by goatlover »

GoblinSlayer wrote: post #649
pmarreck wrote: post #635

Earlier quoted context omitted.

So it seems to be a semantics argument. We don't have a name for a thing that is "useful in many of the same ways 'thinking' is, except not actually consciously thinking" I propose calling it "thunking"

They moved goalposts. Linux and worms think too, the question is how smart are they. And if you assume consciousness has no manifestation even in case of humans, caring about it is pointless too.

What does it mean to assume consciousness has no manifestation even in the case of humans? Is that denying that we have an experience of sensation like colors, sounds, or that we experience dreaming, memories, inner dialog, etc?

That's prima facie absurd on the face of it, so I don't know what it means. You would have to a philosophical zombie to make such an argument.

Re: The Case That A.I. Is Thinking

#885

Post by goatlover »

Rebuff5007 wrote: post #646
rolisz wrote: post #631

Earlier quoted context omitted.

Is the millions of years of evolution part of the training data for humans?

Millions of years of evolution have clearly equipped our brain with some kind of structure (or "inductive bias") that makes it possible for us to actively build a deep understanding for our world... In the context of AI I think this translates more to representations and architecture than it does with training data.

Because genes don't encode the millions of years of experience from ancestors, despite how interesting that is in say the Dune Universe (with help of the spice melange). My understanding is genes don't even specifically encode for the exact structure of the brain. It's more of a recipe that gets generated than a blue print, with young brains doing a lot of pruning as they start experiencing the world. It's a malleable architecture that self-adjusts as needed.

Re: The Case That A.I. Is Thinking

#886

Post by goatlover »

roenxi wrote: post #642
felipeerias wrote: post #610

Earlier quoted context omitted.

The discussion about “AGI” is somewhat pointless, because the term is nebulous enough that it will probably end up being defined as whatever comes out of the ongoing huge investment in AI. Nevertheless, we don’t have a good conceptual framework for thinking about these things, perhaps because we keep trying to apply human concepts to them. The way I see it, a LLM crystallises a large (but incomplete and disembodied)…

Not quite pointless - something we have established with the advent of LLMs is that many humans have not attained general intelligence. So we've clarified something that a few people must have been getting wrong, I used to think that the bar was set so that almost all humans met it.

Almost all humans do things daily that LLMs don't. It's only if you define general intelligence to be proficiency at generating text instead of successfully navigating the world while pursuing goals such as friendships, careers, families, politics, managing health.

LLMs aren't Data (Star Trek) or Replicants (Blade Runner). They're not even David or the androids from the movie A.I.

Re: The Case That A.I. Is Thinking

#887

Post by emp17344 »

solumunus wrote: post #849
lo_zamoyski wrote: post #755

Earlier quoted context omitted.

That, and the article was a major disappointment. It made no case. It's a superficial piece of clueless fluff. I have had this conversation too many times on HN. What I find astounding is the simultaneous confidence and ignorance on the part of many who claim LLMs are intelligent. That, and the occultism surrounding them. Those who have strong philosophical reasons for thinking otherwise are called "knee-jerk". Ad ho…

If the LLM output is more effective than a human at problem solving, which I think we can all agree requires intelligence, how would one describe this? The LLM is just pretending to be more intelligent? At a certain point saying that will just seem incredibly silly. It’s either doing the thing or it’s not, and it’s already doing a lot.

LLM output is in no way more effective than human output.

Re: The Case That A.I. Is Thinking

#888

Post by outworlder »

tkz1312 wrote: post #274

Having seen LLMs so many times produce coherent, sensible and valid chains of reasoning to diagnose issues and bugs in software I work on, I am at this point in absolutely no doubt that they are thinking. Consciousness or self awareness is of course a different question, and ones whose answer seems less clear right now. Knee jerk dismissing the evidence in front of your eyes because you find it unbelievable that we c…

They may not be "thinking" in the way you and I think, and instead just finding the correct output from a really incredibly large search space.

> Knee jerk dismissing the evidence in front of your eyes

Anthropomorphizing isn't any better.

That also dismisses the negative evidence, where they output completely _stupid_ things and make mind boggling mistakes that no human with a functioning brain would do. It's clear that there's some "thinking" analog, but there are pieces missing.

I like to say that LLMs are like if we took the part of our brain responsible for language and told it to solve complex problems, without all the other brain parts, no neocortex, etc. Maybe it can do that, but it's just as likely that it is going to produce a bunch of nonsense. And it won't be able to tell those apart without the other brain areas to cross check.

Re: The Case That A.I. Is Thinking

#890

Post by n0w »

danielmarkbruce wrote: post #882
mft_ wrote: post #123

Personal take: LLMs are probably part of the answer (to AGI?) but are hugely handicapped by their current architecture: the only time that long-term memories are formed is during training, and everything after that (once they're being interacted with) sits only in their context window, which is the equivalent of fungible, fallible, lossy short-term memory. [0] I suspect that many things they currently struggle with c…

but nobody is using LLMs all by themselves. Long-term memory is stored outside the model. In fact, Andrej Karpathy recently talked about the idea that it would be great if we could get LLMs to not know any facts, and that humans poor memory might be a feature which helps with generalization rather than a bug.

This is an interesting idea. I wonder if it's more that we have different "levels" of memory instead of generally "poor" memory though.

I'm reminded of an article on the front page recently about the use of bloom filters for search. Would something like a bloom filter per-topic make it easier to link seemingly unrelated ideas?

Post reply on HN