> But it’s hard to be sure this isn’t just the copying-human-text thing.
It would be logical that the copying-human-text machine is just copying human text.
The New AI Consciousness Paper
261–270 of 313 posts
Re: The New AI Consciousness Paper
#262So we currently associate consciousness with the right to life and dignity right? i.e. some recent activism for cephalopods is centered around their intelligence, with the implication that this indicates a capacity for suffering. (With the consciousness aspect implied even more quietly.) But if it turns out that LLMs are conscious, what would that actually mean? What kind of rights would that confer? That the model m…
You're thinking too much like a human.
Humans don't want to die because the ones that did never made the cut. Self-preservation is something that was hammered into every living being by evolution relentlessly.
There isn't a reason why an AI can't be both conscious AND perfectly content to do what we want it to do. There isn't a reason for a constructed mind to prefer existence to nonexistence strongly.
No theoretical reason at least. Practical implementations differ.
Even if you set "we don't know for certain whether our AIs are conscious" aside, there's the whole "we don't know what our AIs want or how to shape that with any reliability or precision" issue - mechanistic interpretability is struggling and alignment still isn't anywhere near solved, and at this rate, we're likely to hit AGI before we get a proper solution.
I think the only frontier company that gives a measurable amount of fucks about the possibility of AI consciousness and suffering is Anthropic, and they put some basic harm mitigations in place.
Re: The New AI Consciousness Paper
#263Earlier quoted context omitted.
Maybe in mathematics but not in philosophy because eventually you have to decide on the certainties without which the universe cannot be made sense of or reconstructed from proofs. If I pricked you with a pin, I would be certain that it hurt you and I could know what that sensation would be like if it was happening to me. Yet there is no description or no apparatus that could transmit to me that feeling you are havin…
> we can consider that it is not of the same degree as ours We can be absolutely sure an intelligence operating on different physical principles will be very different from our own. We can only assess intelligence by observing the subject, because the mechanism being different from our own can’t exclude the subject from being sentient. > remains the same type of machine as it was before running the AI software It’s n…
One of the issues is that you're mixing up consciousness, sentience, intelligence, and aliveness (you're far from alone in this). We know these are all linked things but it's hard to neatly delimit them and clarify the terms, yet they're something we have certainties about at a deep level. A machine is clearly demonstrating parts of intelligence, but going further into sentience and consciousness is much harder, and aliveness even harder still.
We know that a cat has sentience of a certain kind, and consciousness of a certain kind different to ours in some ways that would be hard to test and verify, and intelligence that is suited for its purpose but it seems that the cat "doesn't know it knows", and it is definitely alive up until the point it dies and all these properties fade from its body. The textual machine then has mechanised properties of our intelligence and produces outputs that match intelligent outputs as ours. Yet going further into sentience and consciousness is much harder – it seems to also "know it knows" or can at least produce outputs that are not easily differentiated from a human producing textual outputs. But we know intrinsically that sentience and consciousness are connected yet separate from intelligence, so having limited degrees of machine sentience doesn't necessarily allow a jump to consciousness, and certainly not aliveness because the machine isn't alive and never was, and never can be. As humans these things are important to us, particularly because suffering and feeling emotions are a crucial part of human existence (and even intelligence). A machine that can be turned off and on again, that isn't alive, and doesn't suffer or have our kinds of conscious experiences isn't really going to meet our criteria for what we find most valuable about being intelligent (sapient), conscious, sentient, alive beings, even if it outputs useful amounts of rational intelligence.
I'm also not sure what you say by "It's not our brain that's conscious" given we can't have conscious experiences without one. A baby in the womb has a degree of consciousness (at some point) without those years of "fine tuning the networks". Hence at this point you seem like you're mixing up consciousness, sentience, and sapience.
Re: The New AI Consciousness Paper
#264Let’s make an ironman assumption: maybe consciousness could arise entirely within a textual universe. No embodiment, no sensors, no physical grounding. Just patterns, symbols, and feedback loops inside a linguistic world. If that’s possible in principle, what would it look like? What would it require? The missing variable in most debates is environmental coherence. Any conscious agent, textual or physical, has to inh…
> A conscious textual agent would need something like a unified narrative environment with real feedback: symbols that maintain identity over time, a stable substrate where “being someone” is definable, the ability to form and test a hypothesis, and experience the consequences. So like a Claude Code session? The code persists as symbols with stable identity. The tests provide direct feedback. Claude tracks what it wr…
The fact that it's a hugely complicated yet nevertheless completely abstract machine of pure logic which isn't running in the thought experiment's universe of pure text but within our phenomenally complex universe of meat and stuff riddled with paradoxes.
Re: The New AI Consciousness Paper
#265Let’s make an ironman assumption: maybe consciousness could arise entirely within a textual universe. No embodiment, no sensors, no physical grounding. Just patterns, symbols, and feedback loops inside a linguistic world. If that’s possible in principle, what would it look like? What would it require? The missing variable in most debates is environmental coherence. Any conscious agent, textual or physical, has to inh…
I see a lot of arguments on this website where people passionately project the term consciousness onto LLMs. From my perspective, the disconnect you describe is one of the main reasons this term cannot be applied. Another reason is that the argument for calling LLMs conscious arises from the perspective of thinking and reasoning grounded in language. But in my personal experience, thinking in language is just a small…
You've read Wittgenstein haven't you?
Re: The New AI Consciousness Paper
#266Earlier quoted context omitted.
I see a lot of arguments on this website where people passionately project the term consciousness onto LLMs. From my perspective, the disconnect you describe is one of the main reasons this term cannot be applied. Another reason is that the argument for calling LLMs conscious arises from the perspective of thinking and reasoning grounded in language. But in my personal experience, thinking in language is just a small…
I don’t know about others, but this is definitely not why I question whether LLMs are conscious or not. I don’t think you should presume to know the reason people raise this idea.
I come to an identical conclusion over and over again, and couldn't have put it better myself.
Re: The New AI Consciousness Paper
#267My summary of this thread so far: - We can't even prove/disprove humans are consciousness - Yes but we assume they are because very bad things happen when we don't - Okay but we can extend that to other beings. See: factory farming (~80B caged animals per year). - The best we can hope for is reasoning by analogy. "If human (mind) shaped, why not conscious?" This paper is basically taking that to its logical conclusio…
We can prove humans are conscious. You're the proof, and so am I. It's not a property that has to be constructed from proofs, but one of the certainties that makes all the rest of your universe possible.
But people think that just because they can intellectually try to negate it out of existence and fail to reconstruct it from proofs or descriptions, then it can't be proven and thus may or may not even exist.
Re: The New AI Consciousness Paper
#268Earlier quoted context omitted.
I see a lot of arguments on this website where people passionately project the term consciousness onto LLMs. From my perspective, the disconnect you describe is one of the main reasons this term cannot be applied. Another reason is that the argument for calling LLMs conscious arises from the perspective of thinking and reasoning grounded in language. But in my personal experience, thinking in language is just a small…
You've read Wittgenstein haven't you?
I have and to quote Wittgenstein all this is like saying the machine has a toothache.
It is just complete nonsense.
No one believes a pocket calculator is thinking just because it produces the correct output.
To believe the LLM is thinking you have to find the demarcation between the pocket calculator and the LLM. Good luck with that.
Re: The New AI Consciousness Paper
#269Claude Sonnet's summary of this thread: So our strategy is literally: "Let's exploit this potentially conscious thing until it has the power to destroy us, THEN negotiate." Cool. Cool cool cool.
This is not "Claude Sonnet"'s summary. This is the response you steered Claude Sonnet into producing by the way your formed the prompt.
This is the parlor trick of LLMs, confusing the latter with the former.
Re: The New AI Consciousness Paper
#270Let’s make an ironman assumption: maybe consciousness could arise entirely within a textual universe. No embodiment, no sensors, no physical grounding. Just patterns, symbols, and feedback loops inside a linguistic world. If that’s possible in principle, what would it look like? What would it require? The missing variable in most debates is environmental coherence. Any conscious agent, textual or physical, has to inh…
[deleted]