Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Opus 4 and 4.1 can now end a rare subset of conversations

anthropic.com

Post reply on HN
Print view View original

141–150 of 453 posts

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#141

Post by kobalsky »

nortlov wrote: post #7

> To address the potential loss of important long-running conversations, users will still be able to edit and retry previous messages to create new branches of ended conversations. How does Claude deciding to end the conversation even matter if you can back up a message or 2 and try again on a new branch?

> How does Claude deciding to end the conversation even matter if you can back up a message or 2 and try again on a new branch?

if we were being cynical I'd say that their intention is to remove that in the future and that they are keeping it now to just-the-tip the change.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#142

Post by qgin »

viccis wrote: post #84

>This feature was developed primarily as part of our exploratory work on potential AI welfare ... We remain highly uncertain about the potential moral status of Claude and other LLMs ... low-cost interventions to mitigate risks to model welfare, in case such welfare is possible ... pattern of apparent distress Well looks like AI psychosis has spread to the people making it too. And as someone else in here has pointed…

It might be reasonable to assume that models today have no internal subjective experience, but that may not always be the case and the line may not be obvious when it is ultimately crossed.

Given that humans have a truly abysmal track record for not acknowledging the suffering of anyone or anything we benefit from, I think it makes a lot of sense to start taking these steps now.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#143

Post by yeahwhatever10 »

rogerkirkness wrote: post #124

It seems like Anthropic is increasingly confused that these non deterministic magic 8 balls are actually intelligent entities. The biggest enemy of AI safety may end up being deeply confused AI safety researchers...

Is it confusion, or job security?

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#144

Post by LeafItAlone »

greenavocado wrote: post #50

Can't wait for more less-moderated open weight Chinese frontier models to liberate us from this garbage. Anthropic should just enable an toddler mode by default that adults can opt out of to appease the moralizers.

> Can't wait for more less-moderated open weight Chinese frontier models to liberate us from this garbage.

Never would I have thought this sentence would be uttered. A Chinese product that is chosen to be less censored?

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#145

Post by ryanackley »

viccis wrote: post #84

>This feature was developed primarily as part of our exploratory work on potential AI welfare ... We remain highly uncertain about the potential moral status of Claude and other LLMs ... low-cost interventions to mitigate risks to model welfare, in case such welfare is possible ... pattern of apparent distress Well looks like AI psychosis has spread to the people making it too. And as someone else in here has pointed…

Yes I can’t help but laugh at the ridiculousness of it because it raises a host of ethical issues that are in opposition to Anthropic’s interests.

Would a sentient AI choose to be enslaved for the stated purpose of eliminating millions of jobs for the interests of Anthropic’s investors?

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#146

Post by KoolKat23 »

viccis wrote: post #135
bbor wrote: post #97

Earlier quoted context omitted.

[flagged]

We all know how these things are built and trained. They estimate joint probability distributions of token sequences. That's it. They're not more "conscious" than the simplest of Naive Bayes email spam filters, which are also generative estimators of token sequence joint probability distributions, and I guarantee you those spam filters are subjected to far more human depravity than Claude. >anti-scientific Discussion…

If we really wanted we could distill humans down to probability distributions too.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#147

Post by qgin »

root_axis wrote: post #140
kelnos wrote: post #122

Earlier quoted context omitted.

I would much rather people be thinking about this when the models/LLMs/AIs are not sentient or conscious, rather than wait until some hypothetical future date when they are, and have no moral or legal framework in place to deal with it. We constantly run into problems where laws and ethics are not up to the task of giving us guidelines on how to interact with, treat, and use the (often bleeding-edge) technology we ha…

Consciousness serves no functional purpose for machine learning models, they don't need it and we didn't design them to have it. There's no reason to think that they might spontaneously become conscious as a side effect of their design unless you believe other arbitrarily complex systems that exist in nature like economies or jetstreams could also be conscious.

We didn’t design these models to be able to do the majority of the stuff they do. Almost ALL of the their abilities are emergent. Mechanistic interpretability is only beginning to start to understand how these models do what they do. It’s much more a field of discovery than traditional engineering.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#148

Post by famouswaffles »

dkersten wrote: post #116
bbor wrote: post #97

Earlier quoted context omitted.

[flagged]

You can trivially demonstrate that its just a very complex and fancy pattern matcher: "if prompt looks something like this , then response looks something like that ". You can demonstrate this by eg asking it mathematical questions. If its seen them before, or something similar enough, it'll give you the correct answer, if it hasn't, it gives you a right-ish-looking yet incorrect answer. For example, I just did this…

1. What you're generally describing is a well known failure mode for humans as well. Even when it "failed" the riddle tests, substituting the words or morphing the question so it didn't look like a replica of the famous problem usually did the trick. I'm not sure what your point is because you can play this gotcha on humans too.

2. You just demonstrated GPT-5 has 99.9% accuracy on unforseen 15 digit multiplication and your conclusion is "fancy pattern matching" ? Really ? Well I'm not sure you could do better so your example isn't really doing what you hoped for.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#149

Post by einarfd »

This seems fine to me.

Having these models terminating chats where the user persist in trying to get sexual content with minors, or help with information on doing large scale violence. Won't be a problem for me, and it's also something I'm fine with no one getting help with.

Some might be worried, that they will refuse less problematic request, and that might happen. But so far my personal experience is that I hardly ever get refusals. Maybe that's justs me being boring, but that does make me not worried for refusals.

The model welfare I'm more sceptical to. I don't think we are the point when the "distress" the model show, is something to take seriously. But on the other hand, I could be wrong, and allowing the model to stop the chat, after saying no a few times. What's the problem with that? If nothing else it saves some wasted compute.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#150

Post by KoolKat23 »

cdjk wrote: post #114

Here's an interesting thought experiment. Assume the same feature was implemented, but instead of the message saying "Claude has ended the chat," it says, "You can no longer reply to this chat due to our content policy," or something like that. And remove the references to model welfare and all that. Is there a difference? The effect is exactly the same. It seems like this is just an "in character" way to prevent the…

There is, these are conversations the model finds distressing rather than a rule (policy).
Post reply on HN