Humanity has the debate about AI consciousness backwards • Hacker News Board

Re: Humanity has the debate about AI consciousness backwards

#191

Post by montroser » Thu, Aug 27, 2026, 8:24 PM UTC

jscd wrote: post #173

montroser wrote: post #91

Earlier quoted context omitted.

Sorry, but they can think and reason. They can have empathy too. Meaning, you can describe a circumstance with many parties involved, with different motivations and conflicting goals -- and it will tell you how these humans would likely feel about each other and the situation at hand.

They can provide text which contains thoughts, reasoning, and empathy, but is that evidence of them having thoughts, reasoning, or empathy? To paraphrase the Chinese room argument: Put me in a room with infinite time, paper, pencils, filing cabinets, and give me an English language list of instructions for doing all the neural-net calculations required to run an LLM. Slide your prompt under the door, and, after doing…

The inputs going through the model. That act is what is doing the thinking and reasoning and empathizing. Same as when your brain feeds your sensory inputs through its neural network.

Your brain is just a big set of weights/neurons organized into layers that do back and forward propagation, turns out. You are just statistics surrounded by flesh and bones.

Re: Humanity has the debate about AI consciousness backwards

#192

Post by shagie » Thu, Aug 27, 2026, 8:26 PM UTC

Animats wrote: post #148

fasterik wrote: post #83

It's hard to read this article and keep track of all the philosophical confusions at once. Whether or not something is conscious (has subjective experience) is independent of whether humans care about it. Whether or not something is a moral patient (an entity worthy of consideration by moral agents) is independent of both human care and consciousness. Of course, we might try to argue that consciousness implies moral…

It's another round of "humans must be special, right?" The author comes close to admitting this: "As intelligent, social entities, we decide (or, to some degree, our evolutionary history has decided for us) which other entities we regard as having inner lives and being worthy of care." Then he works backwards from that towards a definition of consciousness that makes humans special. AI forces us to confront an issue…

> AI forces us to confront an issue that's often come up in science fiction - when you encounter a new kind of creature, how do you decide if they are sentient?

One of the series that I think is most interesting on that topic is the Children of Time series by Adrian Tchaikovsky. It's an awkward read, but it has the most interesting other minds that I've encountered in sci-fi.

Another one that I'd put on there specifically about AI is The Freeze-Frame Revolution by Peter Watts... which I can't say much more without it being a spoiler one way or another.

> Maybe that's the fate of humans. No big extinction. Just the demographic transition, a birth rate below replacement rate, for a few centuries.

https://www.bohemiandrive.com/npwil - Nine Planets Without Intelligent Life (not a spoiler, that's chapter 1, panelset 2). Saturn's Children by Charles Stross might also be appropriate (It's an especially odd book to read immediately after Friday - the "I recognize that twist... oh, it twisted ... that way?").

Re: Humanity has the debate about AI consciousness backwards

#193

Post by Der_Einzige » Thu, Aug 27, 2026, 8:26 PM UTC

serial_dev wrote: post #71

datadrivenangel wrote: post #67

Earlier quoted context omitted.

The argument is one of economic convenience. Caring about model welfare slows us down or makes it more expensive, just like animal welfare and not having slavery... And there's a blurry line here. Obviously slavery is abhorrent, and animal abuse is bad, and factory farming often crosses the line into animal abuse and we should accept that chicken will cost ~25% more per pound to have the chickens be able to walk arou…

The stuffed animals I bought to my children were shipped to me in a closed container, better call PETA! There is no blurry line about agent mental health and whatnot.

The only logical outcome of object oriented ontology (the idea that objects have ontological constitution and possibly are worthy of being treated as ethical actors) is to attempt to benevolently instantly and without warning explode the universe to end suffering.

Re: Humanity has the debate about AI consciousness backwards

#194

Post by Terr_ » Thu, Aug 27, 2026, 8:28 PM UTC

dsign wrote: post #17

blatherard wrote: post #9

If I understand the central argument, which is confusingly stated, the claim is that "consciousness" is a socially-determined, subjective, descriptor for certain things that one cares about. How does this make any sense? If I care about my teddy bear enough it'll gain consciousness?

Your teddy bear, alas, is not cared for in a socially determined way. But you could create a church of your teddy bear and, with some grit, you could get people to widely assign consciousness to it. We have done that sort of thing for thousands of years, and not with fluffy teddy bears; it has been more often with rocks, though not exclusively. It's fun to think about how much of humanity's history and culture is the…

> an improved synthetic simulacrum of self awareness in the coming years, suffering and all

That seems relevant to building The Existifier. [0]

[0] https://www.smbc-comics.com/comic/existence

Re: Humanity has the debate about AI consciousness backwards

#195

Post by Der_Einzige » Thu, Aug 27, 2026, 8:29 PM UTC

threethirtytwo wrote: post #43

Charon77 wrote: post #12

Earlier quoted context omitted.

If I stab at your teddy bear will you feel that it is hurt? Yes it will gain consciousness. You can't prove it but other people can't prove that it's not conscious either

You can prove other people are conscious off the assumption that you yourself are conscious. If you use that axiom then given genetics you know that other humans are rough copies of you so if you are conscious so are your copies because consciousness was copied.

Radical skepticism of one's own consciousness is a perfectly fine take. Stop being baited by desecration propaganda about the cogito. You don't know if what actually appears to you as "thought" is actually "thinking". The appearance of thinking is not necessarily actually thinking (Chain of thought in models should make this very explicitly obvious to people).

Take the radical skepticism pill and embrace epistemological monism - the idea that the only real truth is that all things are interpretations.

Re: Humanity has the debate about AI consciousness backwards

#196

Post by shagie » Thu, Aug 27, 2026, 8:29 PM UTC

dist-epoch wrote: post #132

kaangiray26 wrote: post #84

Quoting from the Permutation City by Greg Egan: "Opponents replied that when you modeled a hurricane, nobody got wet. When you modeled a fusion power plant, no energy was produced. When you modeled digestion and metabolism, no nutrients were consumed – no real digestion took place. So, when you modeled the human brain, why should you expect real thought to occur?"

So I see you solved the Simulation Hypothesis - water is wet, so we can't be in a simulation, because in a simulation nobody gets wet.

My unhappy dwarves would beg to differ. https://www.reddit.com/r/dwarffortress/comments/zljs63/one_o...

Re: Humanity has the debate about AI consciousness backwards

#197

Post by 53uue5ue5 » Thu, Aug 27, 2026, 8:29 PM UTC

mannykannot wrote: post #135

ratelimitsteve wrote: post #130

Earlier quoted context omitted.

I'd argue that if it encrypts and decrypts, then you've built an enigma machine, not modeled it.

Then, presumably, you will also hold that when you have digitally modeled a brain such that the model responds to its inputs as a brain does, then you have built a brain. Tendentiously redefining the word "simulation" in an attempt to avoid the awkward cases is not going to save an argument that was dead on arrival.

what is wrong with saying if if you model a hurricane it wets your pants then you have a hurrican but right now ur pants dry yeah

Re: Humanity has the debate about AI consciousness backwards

#198

Post by slfnflctd » Thu, Aug 27, 2026, 8:30 PM UTC

My response to this for a while (and what I think is one of the few rational responses) has been that it doesn't matter at this point whether consciousness, sentience, sapience or whatever you want to call it is 'actually there'.

What matters that growing numbers of people already believe it is, and their response to that belief will affect the real world, regardless of its truth. This debate is never going to end.

All we can do is try to anticipate where the debate is going, how it might affect us, and how we can make contingency plans for the different ways this could all play out.

Re: Humanity has the debate about AI consciousness backwards

#199

Post by [deleted] » Thu, Aug 27, 2026, 8:31 PM UTC

mannykannot wrote: post #113

kaangiray26 wrote: post #84

Quoting from the Permutation City by Greg Egan: "Opponents replied that when you modeled a hurricane, nobody got wet. When you modeled a fusion power plant, no energy was produced. When you modeled digestion and metabolism, no nutrients were consumed – no real digestion took place. So, when you modeled the human brain, why should you expect real thought to occur?"

Yet when you model an Enigma machine, it does encrypt and decrypt messages.

[deleted]

Re: Humanity has the debate about AI consciousness backwards

#200

Post by slopinthebag » Thu, Aug 27, 2026, 8:33 PM UTC

feoren wrote: post #138

mannykannot wrote: post #128

Earlier quoted context omitted.

When John Searle first published his so-called "Chinese Room" argument ( https://web-archive.southampton.ac.uk/cogprints.org/7150/1/1... ) he included his responses to several counter-arguments that he had been presented with in prior discussions: the systems reply, the robot reply and so forth. Interestingly (as it turns out) no-one seems to have argued that this was a task that could be completed without any unders…

The problem with philosophy is that someone can present an argument that is pretty dumb, everyone can point out it's dumb, people can show evidence that it's dumb, reason through how dumb it is, and publish articles about the many different ways in which it's dumb, and still it gets trotted out as if it counts as evidence for some argument. Searle's Room was debunked immediately upon being suggested: the "Systems Rep…

The person who memorized the book doesn’t understand Chinese. In fact they don’t even understand that Chinese isn’t a language. Searle’s response stands.