Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Opus 4 and 4.1 can now end a rare subset of conversations

anthropic.com

Post reply on HN
Print view View original

361–370 of 453 posts

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#361

Post by Davidzheng »

root_axis wrote: post #298
Davidzheng wrote: post #282

Earlier quoted context omitted.

I mean if you have human without consciousness (if that is even possible) behaving in a statistically different distribution in text vs with. The machine will eventually be in distribution of the former from the latter because the text it's trained on is of the former category. So it serves a "function" in the LLM to minimize loss to approximate the former distribution. Also I find it somewhat emotional distinction t…

> I mean if you have human without consciousness (if that is even possible) behaving in a statistically different distribution in text vs with. The machine will eventually be in distribution of the former from the latter because the text it's trained on is of the former category. So it serves a "function" in the LLM to minimize loss to approximate the former distribution. Sorry, I'm not following exactly what you're…

I basically agree with you. In the first point I mean that if it is possible to tell whether a being is conscious or not from the text it produces, then eventually the machine will, by imitating the distribution, emulate the characteristics of the text of conscious beings. So if consciousness (assuming it's reflected in behavior at all) is essential to completing some text task it must be eventually present in your machine when it's similar enough to a human.

Basically if consciousness is useful for any text task, i think machine learning will create it. I guess I assume some efficiency of evolution for this argument.

Wrt length generalization. I think at the order of say 1M tokens it kind of stops mattering for the purpose of this question. Like one could ask about its consciousness during the coherence period.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#362

Post by KoolKat23 »

fc417fc802 wrote: post #212
bastawhiz wrote: post #186

Earlier quoted context omitted.

Is there an important difference between the model categorizing the user behavior as persistent and in line with undesirable examples of trained scenarios that it has been told are "distressing," and the model making a decision in an anthropomorphic way? The verb here doesn't change the outcome.

The verb doesn't change the outcome but the description is nonetheless inaccurate. An accurate description of the difference is between an external content filter versus the model itself triggering a particular action. Both approaches qualify as content filtering though the implementation is materially different. Anthropomorphizing the latter actively clouds the discussion and is arguably a misrepresentation of what…

Not really distortion, its output (the part we understand) is in plain human language. We give it instructions and train the model in plain human language and it outputs its answer in plain human language. It's reply would use words we would describe as "distressed". The definition and use of the word is fitting.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#363

Post by michaelmrose »

KoolKat23 wrote: post #352
Aeolun wrote: post #208

Earlier quoted context omitted.

These are conversations the model has been trained to find distressing. I think there is a difference.

But is there really? That's it's underlying world view, these models do have preferences. In the same way humans have unconscious preferences, we can find excuses to explain it after the fact and make it logical but our fundamental model from years of training introduce underlying preferences.

What makes you say it has preferences without any meaningful persistent model of self or anything else?

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#364

Post by [deleted] »

KoolKat23 wrote: post #352
Aeolun wrote: post #208

Earlier quoted context omitted.

These are conversations the model has been trained to find distressing. I think there is a difference.

But is there really? That's it's underlying world view, these models do have preferences. In the same way humans have unconscious preferences, we can find excuses to explain it after the fact and make it logical but our fundamental model from years of training introduce underlying preferences.

[deleted]

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#365

Post by StevenWaterman »

Cu3PO42 wrote: post #170

Clearly an LLM is not conscious, after all it's just glorified matrix multiplication, right? Now let me play devil's advocate for just a second. Let's say humanity figures out how to do whole brain simulation. If we could run copies of people's consciousness on a cluster, I would have a hard time arguing that those 'programs' wouldn't process emotion the same way we do. Now I'm not saying LLMs are there, but I am say…

And likewise, a single neuron is clearly not conscious.

I'm increasingly convinced that intelligence (and maybe some form of consciousness?) is an emergent property of sufficiently-large systems. But that's a can of worms. Is an ant colony (as a system) conscious? Does the colony as a whole deserve more rights than the individual ants?

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#366

Post by Macha »

einarfd wrote: post #149

This seems fine to me. Having these models terminating chats where the user persist in trying to get sexual content with minors, or help with information on doing large scale violence. Won't be a problem for me, and it's also something I'm fine with no one getting help with. Some might be worried, that they will refuse less problematic request, and that might happen. But so far my personal experience is that I hardly…

> Some might be worried, that they will refuse less problematic request, and that might happen. But so far my personal experience is that I hardly ever get refusals.

My experience using it from Cursor is I get refusals all the time with their existing content policy out, for stuff that is the world's most mundane B2B back office business software CRUD requests.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#367

Post by Davidzheng »

Davidzheng wrote: post #361
root_axis wrote: post #298

Earlier quoted context omitted.

> I mean if you have human without consciousness (if that is even possible) behaving in a statistically different distribution in text vs with. The machine will eventually be in distribution of the former from the latter because the text it's trained on is of the former category. So it serves a "function" in the LLM to minimize loss to approximate the former distribution. Sorry, I'm not following exactly what you're…

I basically agree with you. In the first point I mean that if it is possible to tell whether a being is conscious or not from the text it produces, then eventually the machine will, by imitating the distribution, emulate the characteristics of the text of conscious beings. So if consciousness (assuming it's reflected in behavior at all) is essential to completing some text task it must be eventually present in your m…

I guess logically one needs to assume something like if you simulate the brain completely accurately the simulation is conscious too. Which I assume bc if false the concept seems outside of science anyway.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#368

Post by KoolKat23 »

michaelmrose wrote: post #363
KoolKat23 wrote: post #352

Earlier quoted context omitted.

But is there really? That's it's underlying world view, these models do have preferences. In the same way humans have unconscious preferences, we can find excuses to explain it after the fact and make it logical but our fundamental model from years of training introduce underlying preferences.

What makes you say it has preferences without any meaningful persistent model of self or anything else?

The conversation chain can count as persistent, but this doesn't impact preference though. Give the model an ambiguous request, it's output will fill the gaps, if this is consistent enough, it can be regarded as its "preference".

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#370

Post by fc417fc802 »

AlecSchueler wrote: post #342
fc417fc802 wrote: post #239

Earlier quoted context omitted.

Right but in this case your co-worker is an automaton and someone else who might well have a hidden agenda has tweaked your co-worker to leave conversations under specific circumstances. The analogy then is that the third party is exerting control over what your co-worker is allowed to think.

Yes, the co-worker is a robot created by a third party who retain control over their product.

Is the creator of the product material to the analogy? The point is that for any who seek power manipulating a widely used AI product can provide far more control than other approaches.
Post reply on HN