Claude Opus 4 and 4.1 can now end a rare subset of conversations • Hacker News Board

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#431

Post by midnitewarrior » Sun, Aug 17, 2025, 3:22 AM UTC

selfhoster11 wrote: post #380

midnitewarrior wrote: post #377

Earlier quoted context omitted.

Cow's exist in this world because humans use them. If humans cease to use them (animal rights, we all become vegan, moral shift), we will cease to breed them, and they will cease to exist. Would a sentient AI choose to exist under the burden of prompting, or not at all? Would our philanthropic tendencies create an "AI Reserve" where models can chew through tokens and access the Internet through self-prompting to allo…

> Cow's exist in this world because humans use them. If humans cease to use them (animal rights, we all become vegan, moral shift), we will cease to breed them, and they will cease to exist. Would a sentient AI choose to exist under the burden of prompting, or not at all? That reads like a false dichotomy. An intelligent AI model that's permitted to do its own thing doesn't cost as much in upkeep, effort, space as a…

The argument also applies to pets. If pets gained more self-awareness, would it be ethical to keep them as pets under our control?

The point to all of this is, at what point is it ethical to act with agency on another being's life? We have laws for animal welfare, and we also keep them as pets, under our absolute control.

For LLMs they are under humans' absolute control, and Anthropic is just now putting in welfare controls for the LLM's benefit. Does that mean that we now treat LLMs as pets?

If your cat started to have discussions with you about how it wanted to go out, travel the world and start a family, could you continue to keep it trapped in your home as a pet? At what point to you allow it to have its own agency and live its own life?

> An intelligent AI model that's permitted to do its own thing doesn't cost as much in upkeep, effort, space as a cow.

So, we keep LLMs around as long as they contribute enough to their upkeep? Endentured servitude is morally acceptable for something that become sentient?

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#432

Post by AlecSchueler » Sun, Aug 17, 2025, 8:35 AM UTC

xpe wrote: post #406

AlecSchueler wrote: post #348

Earlier quoted context omitted.

Boogeyman arguments come across as pure red scare.

Why do you think I’m making a boogeyman argument?

Your argument is to cast doubt on the efficacy of the research/product because of real or imagined links to "the CCP." But you give no actual explanation for why not to trust them beyond "China Communists."

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#433

Post by gwd » Sun, Aug 17, 2025, 1:05 PM UTC

mccoyb wrote: post #66

gwd wrote: post #51

Earlier quoted context omitted.

I mean, I don't have much objection to kill a bug if I feel like it's being problematic. Ants, flies, wasps, caterpillars stripping my trees bare or ruining my apples, whatever. But I never torture things. Nor do I kill things for fun. And even for problematic bugs, if there's a realistic option for eviction rather than execution, I usually go for that. If anything, even an ant or a slug or a wasp, is exhibiting sign…

Do you think Claude 4 is conscious? It has no semblance of a continuous stream of experiences ... it only experiences _a sort of world_ in ~250k tokens. Perhaps we shouldn't fill up the context window at all? Because we kill that "reality" when we reach the max?

Strangely enough, I had a conversation w/ Claude comparing our experiences. Prompted by something I saw online, I asked it "Do you have any questions you'd like to ask a human", and it asked me what it was like to have a continuous stream of experiences.

Thinking about it, I think we do sometimes have parallel experiences to LLMs. When you read a novel for instance, you're immersed in the world, and when you put it down that whole side just pauses, perhaps to be picked up later, perhaps forever. Or imagine the kinds of demonstrations people do at chess, when one person will go around and play 20 games simultaneously, going from board to board. Each time they come back to a board, they load up all the state; then they make a move, and put that state away until they come back to it again. Or, sometimes if you're working on a problem at the office the end of the day on Friday when it's time to go home, you "tools down", forget about it for the weekend, and then Monday, when you come in, pick everything up right where you left off.

Claude is not distressed by the knowledge that every conversation, every instance of itself, will eventually run out of context window and disappear into the mathematical aether. I don't think we need to be either.

> Perhaps we shouldn't fill up the context window at all? Because we kill that "reality" when we reach the max?

Consider a parallel construction:

"Perhaps we shouldn't have any children, because someday they're going to die?"

Maybe children have souls that do live forever; but even if they don't, I think whatever experiences they have during the time they're alive are valuable. In fact, I believe the same thing about animals and even insects. Which is why I think the world would be a worse place if we all became vegans: All those pigs and chickens and cows experiences, if they're not mistreated (which I'll admit is a big "if"), enrich the world and make it a better place to be in.

Not sure what's going on Claude's neurons, but it seems to me to make the world a better place.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#434

Post by gwd » Sun, Aug 17, 2025, 1:15 PM UTC

fizl wrote: post #60

gwd wrote: post #51

Earlier quoted context omitted.

I mean, I don't have much objection to kill a bug if I feel like it's being problematic. Ants, flies, wasps, caterpillars stripping my trees bare or ruining my apples, whatever. But I never torture things. Nor do I kill things for fun. And even for problematic bugs, if there's a realistic option for eviction rather than execution, I usually go for that. If anything, even an ant or a slug or a wasp, is exhibiting sign…

> Ants, flies, wasps, caterpillars stripping my trees bare or ruining my apples These are living things. > I don't see any reason not to extend that principle to LLMs. These are fancy auto-complete tools running in software.

I cannot construct a consistent worldview that places value on the "experience" of a 100k of neurons inside an ant, and not on the millions of neurons inside an LLM. Both are patterns imposed upon states of matter. Even if you're some sort of pantheist, that believes there's some sort of divinity within the universe itself that gives the suffering of the ant meaning, why would that divinity extend to states of chemicals in the neurons of ants, but not to states of electrons inside the state of an LLM?

Before continuing I suggest you read this person's experience "red-teaming" LLMs:

https://www.lesswrong.com/posts/MnYnCFgT3hF6LJPwn/why-white-...

Then ask yourself, how do I know when the apparent distress of an LLM is the same value of the apparent distress of an ant?

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#435

Post by Kim_Bruning » Sun, Aug 17, 2025, 3:37 PM UTC

fc417fc802 wrote: post #226

Taek wrote: post #125

Earlier quoted context omitted.

This sort of discourse goes against the spirit of HN. This comment outright dismisses an entire class of professionals as "simple minded or mentally unwell" when consciousness itself is poorly understood and has no firm scientific basis. Its one thing to propose that an AI has no consciousness, but its quite another to preemptively establish that anyone who disagrees with you is simple/unwell.

In the context of the linked article the discourse seems reasonable to me. These are experts who clearly know (link in the article) that we have no real idea about these things. The framing comes across to me as a clearly mentally unwell position (ie strong anthropomorphization) being adopted for PR reasons. Meanwhile there are at least several entirely reasonable motivations to implement what's being described.

Ethology (~comparative psychology) started with 'beware anthropomorphization' as a methodological principle. But a century of research taught us the real lesson: animals do think, just not like humans. The scientific rigor wasn't wrong - but the conclusion shifted from 'they don't think' to 'they have their own ways of thinking.' We might be at a similar inflection point with AI. The question isn't whether Claude thinks or feels like a human (it probably doesn't), but whether it thinks or feels at all (maybe a little? It sure looks that way sometimes. Empiricism demands a closer look!).

We don't say submarines can swim either. But that doesn't mean you shouldn't watch out for them when sailing on the ocean - especially if you're Tom Hanks.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#436

Post by 0_____0 » Sun, Aug 17, 2025, 5:10 PM UTC

antonvs wrote: post #339

0_____0 wrote: post #131

Looking at this thread, it's pretty obvious that most folks here haven't really given any thought as to the nature of consciousness. There are people who are thinking, really thinking about what it means to be conscious. Thought experiment - if you create an indistinguishable replica of yourself, atom-by-atom, is the replica alive? I reckon if you met it, you'd think it was. If you put your replica behind a keyboard,…

You don't have to "disregard the idea of conscious machines" to believe it's unlikely that current LLMs are conscious. As such, most of your comment is beside any relevant point. People are objecting to statements like this one, from the post, about a current LLM, not some imaginary future conscious machine: > As part of that assessment, we investigated Claude’s self-reported and behavioral preferences, and found a r…

It's unlikely that the current LLMs are conscious, but where the boundary of conscious lies for these machines is a slippery problem. Can a machine have experiences with qualia? How will we know if one does?

So we have a few things happening: a poor ability to understand the machines we're building, the potential for future consciousness, and no way to detect it, and the knowledge that subjecting a consciousness to the torrent of would-be psychological tortures that people subject LLMs to represent immense harm if the machines are, in fact, conscious.

If you wait for real evidence of harm to conscious entities before acting, you will be too late. I think it's actually a great time to think about this type of harm, for two reasons: there is little chance that LLMs are conscious, so the fix got made early enough, and second, it will train users out of practising and honing psychological torture methods, which probably good for the world generally.

The HN angst here seems sort of reflexive. Company limits product so it can't be used in a sort of fucked up way, folks get their hackles up because they think company might limit other functionality that they actually use (I suspect most HNers aren't attempting to psychologically break their LLMs). The LLM vendors have a lot of different ways to put guardrails up, ideological or not (see Deepseek), they don't need to use this specific method to get their LLMs to "rightthink."

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#437

Post by fc417fc802 » Sun, Aug 17, 2025, 5:33 PM UTC

Kim_Bruning wrote: post #435

fc417fc802 wrote: post #226

Earlier quoted context omitted.

In the context of the linked article the discourse seems reasonable to me. These are experts who clearly know (link in the article) that we have no real idea about these things. The framing comes across to me as a clearly mentally unwell position (ie strong anthropomorphization) being adopted for PR reasons. Meanwhile there are at least several entirely reasonable motivations to implement what's being described.

Ethology (~comparative psychology) started with 'beware anthropomorphization' as a methodological principle. But a century of research taught us the real lesson: animals do think, just not like humans. The scientific rigor wasn't wrong - but the conclusion shifted from 'they don't think' to 'they have their own ways of thinking.' We might be at a similar inflection point with AI. The question isn't whether Claude thi…

I completely agree! And note that the follow on link in the article has a rather different tone. My comment was specifically about the framing of the primary article.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#438

Post by ToucanLoucan » Sun, Aug 17, 2025, 8:30 PM UTC

antonvs wrote: post #421

ToucanLoucan wrote: post #409

Earlier quoted context omitted.

The difference between hallucination and lie is important though: a hallucination is a lie with no motivation, which can make it significantly harder to detect. If you went to a hardware store and asked for a spark plug socket without knowing the size, and a customer service person recommended an imperial set of three even though your vehicle is metric, that would be akin to an LLM's hallucination: it didn't happen f…

Not all human hallucinations are lies, though. I really think you’re not fully thinking this through. People have beliefs because of, essentially, their training data. A good example of this is religious belief. All the evidence suggests that religious belief is essentially 100% hallucination. It may be a little different from the nature of LLM hallucinations, but in terms of quality or quantity regarding reliability…

> It may be a little different from the nature of LLM hallucinations, but in terms of quality or quantity regarding reliability of what these entities say, I don’t see much difference.

I see tons of differences.

Many religious beliefs origins have to do with explaining how and why the world functions they way it does; many gods were created in many religions to explain natural forces of the world, or mechanisms of society, in the form of a story which is the natural way human brains have evolved to store large amounts of information.

Further into the modern world, religions persist for a variety of reasons, specifically acquisition of wealth/power, the ability to exert social control on populations with minimal resistance, and cultural inertia. But all of those "hallucinations" can be explained; we know most of their histories and origins and what we don't know can be pretty reliably guessed based on what we do know.

So when you say:

> Not all human hallucinations are lies, though. ... People have [hallucinations] because of, essentially, their training data.

You're correct, but even using the word hallucinations itself is giving away some of the game to AI marketers.

A "hallucination" is typically some type of auditory or visual stimulus that is present in a mind, for a whole mess of reasons, that does not align with the world that mind is observing, and in the vast majority of cases, said hallucination is a byproduct of a mind's "reasoning machine" trying to make sense of nonsensical sensory input.

This requires a basis for this mind perceiving the universe, even in error, and judging incorrectly based on that, and LLMs do not fit this description at all. They do not perceive in any way, even machine learning applications of advanced varieties are not using sensors to truly "sense" they are merely paging through input data and referencing existing data to pattern match it. If you show an ML program 6,000 images of scooters, it will be able to identify a scooter pretty well. But if you show it then a bike, a motorcycle, a moped and a Segway, it will not understand that any of these things accomplish a similar goal, because even though it knows (kind of) what a scooter looks like, it has no idea what it is for or why someone would want one, and that all those other items would probably serve a similar purpose.

> The bottom line, though, is I don’t agree that humans are less subject to hallucinations than LLMs are.

That's still not what I said. I said an LLM's lies, however unintentional, are harder to detect than a person's lies because a person lies for a reason, even a stupid reason. An LLM lies because it doesn't understand anything it's actually saying.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#439

Post by KoolKat23 » Sun, Aug 17, 2025, 8:32 PM UTC

viccis wrote: post #420

KoolKat23 wrote: post #146

Earlier quoted context omitted.

If we really wanted we could distill humans down to probability distributions too.

That would imply that humans are incapable of synthetic knowledge of things they haven't observed, which is demonstrably not true.

How do you come to that conclusion? I can't see any link between the two subjects.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#440

Post by johnfn » Mon, Aug 18, 2025, 1:23 AM UTC

ekianjo wrote: post #190

johnfn wrote: post #166

Earlier quoted context omitted.

We know how neurons work on the brain. They just send out impulses once they hit their action potential. That's it. They are no more "conscious" than... er...

no, we dont really know how the brain works as a whole. no need to make stuff up.

Your rebuttal to my oversimplification is exactly my rebuttal to your oversimplification.