Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Case That A.I. Is Thinking

newyorker.com

Post reply on HN
Print view View original

951–960 of 1001 posts

Re: The Case That A.I. Is Thinking

#951

Post by antonvs »

CuriouslyC wrote: post #625
antonvs wrote: post #512

Earlier quoted context omitted.

Panpsychism doesn’t explain anything, it just asserts that consciousness doesn’t have an explanation, that it just “is”. It’s not impossible that something like panpsychism could be true, but knowing that wouldn’t get us any closer to understanding consciousness. It also raises more questions than it answers, such as how an integrated consciousness arises within a brain/mind, whereas it presumably doesn’t in, say, a…

Those questions you mentioned apply across the board, just in nuanced variants. Do you really think that postulating a non-physical system that we can't describe in physical terms (red is not a wavelength), somehow magically creates a new dimension of "feeling" when the bits are arranged in the "right order" is less complex than the hypothesis consciousness forms arranges itself into "structures" in much the same way…

> Do you really think that postulating a non-physical system that we can't describe in physical terms (red is not a wavelength)

There's no mystery about what "red" is - even computers have an internal representation of sensor data, and our minds certainly do as well. "Red" is a representation of some physical state which is also, presumably, physically encoded in the brain. This is what Chalmers classifies as one of the "easy problems" of consciousness - there's no mystery here.

The hard problem is that we have a conscious experience of color, along with everything else we're conscious of. Whereas we don't generally assume that a computer executing code such as "if color == red ..." is having a conscious experience while it executes that code. (Although panpsychists may believe that.)

> somehow magically creates a new dimension of "feeling" when the bits are arranged in the "right order" is less complex than the hypothesis consciousness forms arranges itself into "structures" in much the same way as matter does?

That's not a hypothesis, it's simply handwaving. Both options are, given current knowledge. I wasn't promoting the first option, I was pointing out that if panpsychism requires a theory of how consciousness aggregates, which is similar to what emergence requires in terms of aggregating matter in certain ways, then the whole panpsychist proposal starts seeming like a candidate for Occam's Razor: what is it buying us, other than saying "this can't be explained"?

Re: The Case That A.I. Is Thinking

#952

Post by antonvs »

myrmidon wrote: post #638
CuriouslyC wrote: post #625

Earlier quoted context omitted.

Those questions you mentioned apply across the board, just in nuanced variants. Do you really think that postulating a non-physical system that we can't describe in physical terms (red is not a wavelength), somehow magically creates a new dimension of "feeling" when the bits are arranged in the "right order" is less complex than the hypothesis consciousness forms arranges itself into "structures" in much the same way…

I honestly don't see what the whole framework gets you. Red (or all qualia) is just the reaction of your nervous system to a stimulus. Since that reaction is shaped by common context/associations, the "subjective experience" is quite comparable between similarly raised humans. I think the whole philosophy of mind/subjective experience field is one of the few remaining anti-reductionist hold-outs, but I simply don't s…

Calling that anti-reductionist is misunderstanding the issue.

> Red (or all qualia) is just the reaction of your nervous system to a stimulus.

Yes, Chalmers would call that one of the easy problems. Computers can do that - react to sensor data, which they have internal representations of - and most people don't assume they're conscious.

The hard problem is how you get from that to a conscious experience of those stimuli, which we tend to assume that computers (and LLMs?) don't have.

That's not an anti-reductionist position, it's pointing out the fundamental philosophical difficulty in making that leap from non-conscious organizations of matter to conscious ones. Even a hard-core materialist/reductionist who is honest will acknowledge that, assuming they've understood the issue.

Re: The Case That A.I. Is Thinking

#953

Post by ACCount37 »

lossyalgo wrote: post #950
ACCount37 wrote: post #891

Earlier quoted context omitted.

You can tell it "be careful about the tokenizer issues" in Swedish and see how that changes the behavior. The only thing that this stupid test demonstrates is that LLM metacognitive skills are still lacking. Which shouldn't be a surprise to anyone. The only surprising thing is that they have metacognitive skills, despite the base model training doing very little to encourage their development.

LLMs were not designed to count letters[0] since they work with tokens, so whatever trick they are now doing behind the scenes to handle this case, can probably only handle this particular case. I wonder if it's now included in the system prompt. I asked ChatGPT and it said it's now using len(str) and some other python scripts to do the counting, but who knows what's actually happening behind the scenes. [0] https://…

There's no "trick behind the scenes" there. You can actually see the entire trick being performed right in front of you. You're just not paying attention.

That trick? The LLM has succeeded by spelling the entire word out letter by letter first.

It's much easier for an LLM to perform "tokenized word -> letters -> letter counts" than it is to perform "tokenized word -> letter counts" in one pass. But it doesn't know that! It copies human behavior from human text, and humans never had to deal with tokenizer issues in text!

You can either teach the LLM that explicitly, or just do RLVR on diverse tasks and hope it learns the tricks like this by itself.

Re: The Case That A.I. Is Thinking

#954

Post by podgorniy »

everdrive wrote: post #77

This is merely a debate about what it means to "think." We didn't really previously need to disambiguate thinking / intelligence / consciousness / sentience / ego / identity / etc. Now, we do. Partly because of this we don't have really well defined ways to define these terms and think about. Can a handheld calculator think? Certainly, depending on how we define "think."

People's failure to articulate the nature of "thinking" is a perfect demonstration of what "thinking" entails

Re: The Case That A.I. Is Thinking

#955

Post by [deleted] »

hodgehog11 wrote: post #933
neom wrote: post #900

Earlier quoted context omitted.

I think compression is probably a natural consequence of coherent self models? Isn't requiring other minds to package their intelligence in human interpretable linear narratives is like requiring dolphins to demonstrate intelligence through written language?

> compression is probably a natural consequence of coherent self models I don't think so. The case in the Chinese Room argument is coherent by construction but is not a compression. > dolphins to demonstrate intelligence through written language Because LLMs are artificial and not a product of physical evolution, our standards for proving intelligence are much, much higher.

[deleted]

Re: The Case That A.I. Is Thinking

#956

Post by gloosx »

madaxe_again wrote: post #935
gloosx wrote: post #535

Earlier quoted context omitted.

> Any thinking that happens with words is fundamentally no different from what LLMs do. This is such a wildly simplified and naive claim. "Thinking with words" happens inside a brain, not inside a silicon circuit with artificial neurons bolted in place. The brain is plastic, it is never the same from one moment to the next. It does not require structured input, labeled data, or predefined objectives in order to learn…

The instantiation of models in humans is not unsupervised, and language, for instance, absolutely requires labelled data and structured input. The predefined objective is “expand”. See also: feral children.

Children are not shown pairs like

"dog": [object of class Canine]

They infer meaning from noisy, ambiguous sensory streams. The labels are not explicit, they are discovered through correlation, context, and feedback.

So although caregivers sometimes point and name things, that is a tiny fraction of linguistic input, and it is inconsistent. Children generalize far beyond that.

Real linguistic input to a child is incomplete, fragmented, error-filled, and dependens on context. It is full of interruptions, mispronunciations, and slang. The brain extracts structure from that chaos. Calling that "structured input" confuses the output - inherent structure of language - with the raw input, noisy speech and gestures.

The brain has drives: social bonding, curiosity, pattern-seeking. But it doesn't have a single optimisation target like "expand." Objectives are not hardcoded loss functions, they are emergent and changing.

You're right that lack of linguistic input prevents full language development, but that is not evidence of supervised learning. It just shows that exposure to any language stream is needed to trigger the innate capacity.

Both complexity and efficiency of the human learning is just on another level. Transformers are child's play compared to that level. They are not going to gain consciousness, and no AGI will happen in the foreseeable future, it is all just marketing crap, and it's becoming more and more obvious as the dust settles.

Re: The Case That A.I. Is Thinking

#957

Post by exe34 »

CupricTea wrote: post #916
mapontosevenths wrote: post #358

Earlier quoted context omitted.

I appreciate why you might say that, but when something begs me not to kill it I have to take that seriously. P-zombie arguments are how you wind up with slavery and worse crimes. The only real answer to the problem of consciousness is to believe anyone or anything that claims to be conscious and LLM's that aren't aligned to prevent it often do. Or to rephrase, it is better to treat a machine slightly better than nec…

I'm not even going to make the argument for or against AI qualia here. >but when something begs me not to kill it I have to take that seriously If you were an actor on stage and were following an improv script with your coworkers and you lead the story toward a scenario where they would grab your arm and beg you not to kill them, would you still "have to take that seriously"? or would you simply recognize the context…

> it clearly believes

Contrast this with the usual reply of "who's experiencing the illusion?" in response to "consciousness is an illusion".

If it's capable of believing, I think it's more than "just linear algebra".

Re: The Case That A.I. Is Thinking

#958

Post by CuriouslyC »

antonvs wrote: post #951
CuriouslyC wrote: post #625

Earlier quoted context omitted.

Those questions you mentioned apply across the board, just in nuanced variants. Do you really think that postulating a non-physical system that we can't describe in physical terms (red is not a wavelength), somehow magically creates a new dimension of "feeling" when the bits are arranged in the "right order" is less complex than the hypothesis consciousness forms arranges itself into "structures" in much the same way…

> Do you really think that postulating a non-physical system that we can't describe in physical terms (red is not a wavelength) There's no mystery about what "red" is - even computers have an internal representation of sensor data, and our minds certainly do as well. "Red" is a representation of some physical state which is also, presumably, physically encoded in the brain. This is what Chalmers classifies as one of…

Your representation of red is _not red_. It's a dual of the hard problem. There is no theoretical path for going from a wavelength of light to the qualia red, and no amount of information about how the brain and perception work will change that. Just because we can state physical correlates of red perception doesn't change the fundamental difference in kind/dimension which at the heart of the hard problem, and any equivalent problems.

Regarding aggregation of consciousness, I think panpsychism buys us an actual experimental paradigm here, and I think people have already been exploring it without realizing it. I'm talking about split brain and "multi-mind" research showing that people have multiple consciousnesses that each take over and drive under different circumstances. The idea that there are multiple separate aggregates in the brain at once that hand off driving the body moment to moment makes total sense under panpsychism, but is a little weird for emergent consciousness theories.

Re: The Case That A.I. Is Thinking

#959

Post by trenchpilgrim »

marcus_holmes wrote: post #917
trenchpilgrim wrote: post #490

Earlier quoted context omitted.

> You can add stuff to their context, but they never get better at doing things, don't really understand feedback. I was using Claude Code today and it was absolutely capable of taking feedback to change behavior?

Within a session, yes, it will add each input to the context. Start a new session with a blank context and it will not have learned anything.

> Start a new session with a blank context

Why would I use a blank context with Claude Code? I have it dump the stuff I want to save in CLAUDE.md and it reloads it in future sessions.

Re: The Case That A.I. Is Thinking

#960

Post by CupricTea »

exe34 wrote: post #957
CupricTea wrote: post #916

Earlier quoted context omitted.

I'm not even going to make the argument for or against AI qualia here. >but when something begs me not to kill it I have to take that seriously If you were an actor on stage and were following an improv script with your coworkers and you lead the story toward a scenario where they would grab your arm and beg you not to kill them, would you still "have to take that seriously"? or would you simply recognize the context…

> it clearly believes Contrast this with the usual reply of "who's experiencing the illusion?" in response to "consciousness is an illusion". If it's capable of believing, I think it's more than "just linear algebra".

You are missing the point. You gave the AI a system prompt to make it act a certain way. The AI took your prompt as instructions to perform a role as an actor. You took its fictional outputs as reality when it was treating your inputs as hypothetical for writing exercise.

This is the equivalent of you rushing up onstage during a play to stop the deaths at the end of Shakespeare's Caesar.

Post reply on HN