Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Opus 4 and 4.1 can now end a rare subset of conversations

anthropic.com

Post reply on HN
Print view View original

211–220 of 453 posts

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#211

Post by Aeolun »

faizshah wrote: post #205

This is well intended but I know from experience this is gonna result in you asking “how do you find and kill the process on port 8080” and getting a lecture + “Claude has ended the chat.” I hope they implemented this in some smarter way than just a system prompt.

Claude kept aborting my requests for my space trading game because I kept asking it about the gene therapy.

``` Looking at the trade goods list, some that might be underutilized: - BIOCOMPOSITES - probably only used in a few high-tech items - POLYNUCLEOTIDES - used in medical/biological stuff - GENE_THERAPEUT ⎿ API Error: Claude Code is unable to respond to this request, which appears to violate our Usage Policy (https://www.anthropic.com/legal/aup). Please double press esc to edit your last message or start a new session for Claude Code to assist with a different task. ```

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#212

Post by fc417fc802 »

bastawhiz wrote: post #186
victor9000 wrote: post #183

Earlier quoted context omitted.

It seems like you're anthropomorphising an algorithm, no?

Is there an important difference between the model categorizing the user behavior as persistent and in line with undesirable examples of trained scenarios that it has been told are "distressing," and the model making a decision in an anthropomorphic way? The verb here doesn't change the outcome.

The verb doesn't change the outcome but the description is nonetheless inaccurate. An accurate description of the difference is between an external content filter versus the model itself triggering a particular action. Both approaches qualify as content filtering though the implementation is materially different. Anthropomorphizing the latter actively clouds the discussion and is arguably a misrepresentation of what is really happening.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#213

Post by Aeolun »

snickerdoodle12 wrote: post #177
mhink wrote: post #168

Earlier quoted context omitted.

Even though LLMs (obviously (to me)) don't have feelings, anthropomorphization is a helluva drug, and I'd be worried about whether a system that can produce distress-like responses might reinforce, in a human, behavior which elicits that response. To put the same thing another way- whether or not you or I *think* LLMs can experience feelings isn't the important question here. The question is whether, when Joe User se…

> although I would find it extremely distasteful and frankly alarming This objection is actually anthropomorphizing the LLM. There is nothing wrong with writing books where a character experiences distress, most great stories have some of that. Why is e.g. using an LLM to help write the part of the character experiencing distress "extremely distasteful and frankly alarming"?

Claude is actually smart enough to realize when it’s asked to write stuff that it’d normally think is inappropriate. But there’s certain topics that it gets iffy about and does not want to write even in the context of a story. It’s kind of funny, because it’ll start on the message with gusto, and then after a few seconds realize what it’s doing (presumably the protection kicking in) and abort the generation.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#214

Post by Aeolun »

transcriptase wrote: post #6

“Also these chats will be retained indefinitely even when deleted by the user and either proactively forwarded to law enforcement or provided to them upon request” I assume, anyway.

I’m fairly certain there’s already a clause displayed on their dashboard that mentions chats with TOS violations will be retained indefinitely.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#215

Post by HappMacDonald »

bgwalter wrote: post #127
kelnos wrote: post #122

Earlier quoted context omitted.

I would much rather people be thinking about this when the models/LLMs/AIs are not sentient or conscious, rather than wait until some hypothetical future date when they are, and have no moral or legal framework in place to deal with it. We constantly run into problems where laws and ethics are not up to the task of giving us guidelines on how to interact with, treat, and use the (often bleeding-edge) technology we ha…

What is that hypothetical date? In theory you can run the "AI" on a Turing machine. Would you think a tape machine can get sentient?

In theory you can emulate every biochemical reaction of a human brain on a turing machine, unless you'd like to try to sweep consciousness under the rug of quantum indeterminism from whence it wouldn't be able to do anybody any good anyway.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#216

Post by katabasis »

virgildotcodes wrote: post #176
ghostly_s wrote: post #137

Earlier quoted context omitted.

This post seems to explicitly state they are doing this out of concern for the model's "well-being," not the user's.

Yeah, but my interpretation of what the user you’re replying to is saying is that these LLMs are more and more going to be teaching people how it is acceptable to communicate with others. Even if the idea that LLMs are sentient may be ridiculous atm, the concept of not normalizing abusive forms of communication with others, be they artificial or not, could be valuable for society. It’s funny because this is making me…

Yeah pretty much this. One can argue that it’s idiotic to treat chatbots like they are alive, but if a bit of misplaced empathy for machines helps to discourage antisocial behavior towards other humans (even as an unintentional side effect), that seems ok to me.

As an aside, I’m not the kind of person who gets worked up about violence in video games, because even AAA titles with excellent graphics are still obvious as games. New forms of technology are capable of blurring the lines between fantasy and reality to a greater degree. This is true of LLM chat bots to some degree, and I worry it will also become a problem as we get better VR. People who witness or participate in violent events often come away traumatized; at a certain point simulated experiences are going to be so convincing that we will need to worry about the impact on the user.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#218

Post by headinsand »

tptacek wrote: post #134

If you really cared about the welfare of LLMs, you'd pay them San Francisco scale for earlier-career developers to generate code.

Telling that this is your definition of “caring”.

“Boss makes a dollar, I make me a dime”, eh?

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#219

Post by fc417fc802 »

intotheabyss wrote: post #160
root_axis wrote: post #140

Earlier quoted context omitted.

Consciousness serves no functional purpose for machine learning models, they don't need it and we didn't design them to have it. There's no reason to think that they might spontaneously become conscious as a side effect of their design unless you believe other arbitrarily complex systems that exist in nature like economies or jetstreams could also be conscious.

Do you think this changes if we incorporate a model into a humanoid robot and give it autonomous control and context? Or will "faking it" be enough, like it is now?

You can't even prove other _people_ aren't "faking" it. To claim that it serves no functional purpose or that it isn't present because we didn't intentionally design for it is absurd. We very clearly don't know either of those things.

That said, I'm willing to assume that rocks (for example) aren't conscious. And current LLMs seem to me to (admittedly entirely subjectively) be conceptually closer to rocks than to biological brains.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#220

Post by kordlessagain »

This happened to me three times in a row on Claude after sending it a string of emojis telling the life story of Rick Astley. I think it triggers when it tries to quote the lyrics, because they are copyright? Who knows?

"Claude is unable to respond to this request, which appears to violate our Usage Policy. Please start a new chat."

> We remain highly uncertain about the potential moral status of Claude and other LLMs, now or in the future.

That's nice, but I think they should be more certain sooner than later.

Post reply on HN