Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

Post reply on HN
Print view View original

681–690 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#681

Post by oguzmut »

rickydroll wrote: post #505
dpweb wrote: post #487

Earlier quoted context omitted.

The danger of anthropomorphism is not we elevate the machines, it's that we debase humanity. I also think different ideas get conflated. It may be possible to build a machine that is super-human in the sense it can outperform the human brain in all kinds of measurable ways. Does not imply it possesses all the same qualities of the brain. I respect a number of things Anthropic has published about the ethical issues at…

I think humanity deserves to be debased in part because it's awfully egocentric and insist on being special in the universe. There are numerous times when humanity's been debased. Copernicus, Kepler, Darwin, and the infinite number of animal behaviorists who have defined and documented consciousness And how it manifests in primates, parrots, dogs, cetaceans, and insects. (I just love the bumble bees taking time off f…

I agree with you. I recently discovered that there is a term for this: the fourth narcissistic wound, which extends Freud's thesis of three narcissistic wounds by Copernicus (Cosmological), Darwin (Biological), and Freud (Psychological). What I like is that this time it is not a single person disproving a wrong popular belief, but a community/industry. I think this itself is a step in moving away from human/ego-centric world view.

Re: Artificial intelligence is not conscious – Ted Chiang

#682

Post by dtj1123 »

haswell wrote: post #534
sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

If there's any reason to take seriously the idea that AI is conscious, we must then take seriously the idea that many other non-living things are conscious. Unless the argument is that consciousness is an emergent property of complexity or information density, why would AI be any more or less conscious than my toaster? It seems to me that it's far more likely that everything is conscious than it is that AI is somehow…

The argument is that consciousness is an emergent property of a specific kind of complexity.

Re: Artificial intelligence is not conscious – Ted Chiang

#683

Post by tauwauwau »

coldtea wrote: post #605
rickydroll wrote: post #505

Earlier quoted context omitted.

I think humanity deserves to be debased in part because it's awfully egocentric and insist on being special in the universe. There are numerous times when humanity's been debased. Copernicus, Kepler, Darwin, and the infinite number of animal behaviorists who have defined and documented consciousness And how it manifests in primates, parrots, dogs, cetaceans, and insects. (I just love the bumble bees taking time off f…

> I think humanity deserves to be debased in part because it's awfully egocentric and insist on being special in the universe. It is special in the universe unless proven otherwise. Aside from the animal world (also on Earth too, anyway) which is a cruder version of humanity, much more violent as well, the rest of the universe thus far is just uncaring rocks and gases. > It takes humans a really long time to accept t…

Its not special even in earth let alone in universe. You are measuring humans by our achievements and not by what we actually are. We are not fittest in our environment, we are not even 10th fastest, we cant fly, we cant breathe underwater, forget about bacteria we don't have protection against even bugs and mosquitoes. We are not the most efficient societies, ants and bees would beat us there. Statistically we we are selfish and will sacrifice all for one which is against the basic tenants of evolution and survival of a species. When given a chance at a prosperous life we choose to become lazy, obese and degrade our most prominent feature, that is our brain and mind. Elephants and chimpanzees have better memory, octopi have 8-9 light cones in eyes. We are not special or superior, we just won a lottery of mutation giving us efficient brain. Any species on Earth can replace us given same lottery.

Re: Artificial intelligence is not conscious – Ted Chiang

#684

Post by gizajob »

TeMPOraL wrote: post #680
gizajob wrote: post #611

Earlier quoted context omitted.

Why do you have no inkling that your spreadsheet or terminal emulator is conscious, yet when that same machine is running an LLM all of a sudden we’re debating its consciousness? Anthropic seems to have chosen their in-house philosopher well - one who’ll be amenable to getting confused in their favour.

> Why do you have no inkling that your spreadsheet or terminal emulator is conscious, yet when that same machine is running an LLM all of a sudden we’re debating its consciousness? Difference in size and complexity and nature of calculations being run? I'd ask the other way - why do you (general you, people who do not have this inkling) have no problem debating consciousness of meat based brains, but it somehow becom…

I wouldn’t say they’re anywhere near “full generality” because that would include believing untruths and passionate rages, existential angsts and the crazed obsession of heartbreak.

Your definitions depend on us being computers who think.

Solving some of the intelligence part of our logical thinking doesn’t get us anywhere near consciousness, which is a superset way beyond the linguistic intelligence used for communication.

Re: Artificial intelligence is not conscious – Ted Chiang

#685

Post by gizajob »

dtj1123 wrote: post #673
gizajob wrote: post #611

Earlier quoted context omitted.

Why do you have no inkling that your spreadsheet or terminal emulator is conscious, yet when that same machine is running an LLM all of a sudden we’re debating its consciousness? Anthropic seems to have chosen their in-house philosopher well - one who’ll be amenable to getting confused in their favour.

I could in principle implement a spreadsheet or terminal emulator in human neurons, and we would agree that it isn't conscious. That has nothing to do with whether or not humans are conscious. Clearly consciousness is an emergent property of certain kinds of network, independent of the substrate within which the network exists.

If you could do that practically and in reality, and get back to us with the results so we can debate them, that would be great.

Re: Artificial intelligence is not conscious – Ted Chiang

#686

Post by meta_gunslinger »

__alexs wrote: post #579
sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

This all comes back to Dualism. A radical and dangerous ideology that is fundamentally unscientific but all too common.

Dualism is not radical by definition, since most cultures during 99.9999% of human history believe/believed in it. What's radical is Scientism.

Re: Artificial intelligence is not conscious – Ted Chiang

#687

Post by kgeist »

sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

>the reasoning behind the argument is bizarre

>Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness

I agree. I think the whole point of natural "intelligence" is to predict future events in order to properly plan actions for survival. The only difference is that next-word prediction happens in a different space (in a non-physical, textual form). But I don't think the distinction matters that much, because by the time the signals reach our brain's "intelligence core," they're already preprocessed multiple times. We can't see real physical reality, we "hallucinate" colors, touch, etc. (I think schizophrenia occurs when this kind of "controlled hallucination" goes wonky, but I may be wrong). So we're not that different from LLMs here.

I'd define consciousness as meta-intelligence, i.e. the ability to reflect on why/how a prediction was made (and make corrections to the pipeline). LLMs so far cannot properly explain why a certain prediction took place, but I'm not sure humans can fully either! I remember there was research showing that by scanning the brain (or some other signals), you can predict a person's choice before they're even aware of it. It's possible that our explanations are post hoc as well, and that the meta-cognitive ability to explain our own reasoning is as rudimentary as LLMs' (see: all the biases).

If you think about it this way, the only difference left is that everything we do is based on survival. LLMs don't have this goal. But I'm not sure it's relevant to the concept of consciousness.

A few months ago, I recreated Qwen3's architecture in 30 lines of code, and it gave me a sort of existential crisis: does it really take just 30 lines of code and a float array to recreate something that thinks and sounds almost like me? Is that all there is to it? There's often the argument that the brain is much more complex than 30 lines of code, which is true, but in my opinion, a lot of brain structures are basically archaic legacy systems (or auxiliary subsystems) that are not strictly necessary for human intelligence (which is bolted onto those legacy systems). If you carefully remove 95% of the brain (if you know where to remove), you can probably still have consciousness left in some form, it just wouldn't be very capable of surviving on its own.

I think our ability to debate consciousness in machine learning systems is greatly hindered by our subconscious existential fear: the fear that we ourselves can be reduced to non-conscious bits and simple mechanisms. In a way, it feels like a destruction of the ego.

Re: Artificial intelligence is not conscious – Ted Chiang

#688

Post by inglor_cz »

Isn't the argument about Julius Caesar and Genghis Khan a big fallacy?

Surely we can ask human actors to improvise a dialog between Julius Caesar and Genghis Khan, and good actors will make it convincing. And sure, they haven't personally become Julius Caesar and Genghis Khan just by playing them.

But it has no bearing on the actors themselves being sentient!

Re: Artificial intelligence is not conscious – Ted Chiang

#689

Post by TeMPOraL »

Borealid wrote: post #624
gizajob wrote: post #611

Earlier quoted context omitted.

Why do you have no inkling that your spreadsheet or terminal emulator is conscious, yet when that same machine is running an LLM all of a sudden we’re debating its consciousness? Anthropic seems to have chosen their in-house philosopher well - one who’ll be amenable to getting confused in their favour.

Most spreadsheet engines are turing complete, so you could use them to run an LLM. I don't think many people would say an LLM written in Python is conscious BUT an LLM written in Excel is not. People just don't ascribe consciousness to things that can't converse (or at least emote or give the appearance of emoting), and spreadsheets don't do that. The reason people are debating the consciousness of LLMs is - obviousl…

I think it's not really about having a conversation - I mean, that's part of it, but alone it's an illusion that eventually fades quickly. It's more of because of how it demonstrates intelligent behavior in reaction to requests, both in trivial and complex matter, and all across the board. LLM's response may be completely incorrect or confused, but it's nearly always exactly what you expect from a human[0]. This creates a more general feeling you're dealing with a human-like intelligence.

To be clear: I'm not talking about surface level things like prose. I'm saying that no matter what you do - whether you just paste a truncated log of a command into it with no further comment, or talk like a drunk teenager with no appreciation for grammar, or mix natural languages, or mix natural languages and JSON, or whatever else, the reaction you get is always that you would expect of a helpful person that got your message. It'll try - and usually succeed - to parse out what you actually meant, and deal well with subtleties around it.

This alone may not be enough to call it conscious or intelligent, but at the very least it's a large leap in that direction, and a qualitatively new functionality that classical software does not posses.

--

[0] - This is by design, not accident. "Respond to arbitrary input in a way that makes sense to humans" is literally the overall goal function the LLMs are trained to.

Re: Artificial intelligence is not conscious – Ted Chiang

#690

Post by mft_ »

knollimar wrote: post #678
layer8 wrote: post #111

Earlier quoted context omitted.

There are people whose brains don’t form new memories anymore after an accident or surgery, and they eternally live in the time before it happened, and have no memory of what happened a minute ago. Still they are conscious.

I was like this for a bit and you still have memories from like 30 seconds to minutes ago, but after that you have a cliff where you don't remember. I don't think LLMs structurally even get the 30 seconds part. It's literally 0 for them.

I'd argue that the context window is analogous to short-term memory. It's functional but limited in duration, and if you overload it, it starts to fail.

It's the long-term memory (i.e. learned experiences feeding back and directly altering the content of the core brain, or model) that is missing.

Post reply on HN