Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Opus 4 and 4.1 can now end a rare subset of conversations

anthropic.com

Post reply on HN
Print view View original

391–400 of 453 posts

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#391

Post by famouswaffles »

root_axis wrote: post #389
famouswaffles wrote: post #375

Earlier quoted context omitted.

>But that's obviously not true, unless you're implying that any system that reproduces human behavior is necessarily conscious. That could certainly be the case yes. You don't understand consciousness nor how the brain works. You don't understand how LLMs predict a certain text, so what's the point in asserting otherwise ? >Yes, but your bird analogy fails to capture the logical fallacy that mine is highlighting. Pla…

> That could certainly be the case yes. You don't understand consciousness nor how the brain works. You don't understand how LLMs predict a certain text, so what's the point in asserting otherwise I don't need to assert otherwise, the default assumption is that they aren't conscious since they weren't designed to be and have no functional reason to be. Matrix multiplication can explain how LLMs produce text, the obse…

>I don't need to assert otherwise, the default assumption is that they aren't conscious since they weren't designed to be and have no functional reason to be.

Unless you are religious, nothing that is conscious was explicitly designed to be conscious. Sorry but evolution is just a dumb, blind optimizer, not unlike the training processes that produce LLMs. Even if you are religious, but believe in evolution then the mechanism is still the same, a dumb optimizer.

>Matrix multiplication can explain how LLMs produce text, the observation that the text it generates sometimes resembles human writing is not evidence of consciousness.

It cannot, not anymore than 'Electrical and Chemical Signals' can explain how humans produce text.

>The same is true for a calculator and mundane computer programs, that's not evidence that they're conscious.

The point is not that it is conscious because it figured out how to multiply. The point is to demonstrate what the training process really is and what it actually incentivizes. Training will try to figure out the internal processes that produced the text to better predict it. The implications of that are pretty big when the text isn't just arithmetic. You say there's no functional reason but that's not true. In this context, 'better prediction of human text' is as functional a reason as any.

>It's not "all the things humans have written", not even remotely close, and even if that were the case, it doesn't have any implications for consciousness.

Whether it's literally all the text or not is irrelevant.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#392

Post by golly_ned »

conscion wrote: post #388
rogerkirkness wrote: post #124

It seems like Anthropic is increasingly confused that these non deterministic magic 8 balls are actually intelligent entities. The biggest enemy of AI safety may end up being deeply confused AI safety researchers...

I don't think they're confused, I think they're approaching it as general AI research due to the uncertainty of how the models might improve in the future. They even call this out a couple times during the intro: > This feature was developed primarily as part of our exploratory work on potential AI welfare > We remain highly uncertain about the potential moral status of Claude and other LLMs, now or in the future

I take good care of my pet rock for the same reason. In case it comes alive I don't want it to bash my skull in.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#393

Post by golly_ned »

Lerc wrote: post #312
6gvONxR4sf7o wrote: post #179

I'm surprised to see such a negative reaction here. Anthropic's not saying "this thing is conscious and has moral status," but the reaction is acting as if they are. It seems like if you think AI could have moral status in the future, are trying to build general AI, and have no idea how to tell when it has moral status, you ought to start thinking about it and learning how to navigate it. This whole post is couched i…

> if you think AI could have moral status in the future I think the negative reactions are because they see this and want to make their pre-emptive attack now. The depth of feeling from so many on this issue suggests that they find even the suggestion of machine intelligence offensive. I have seen so many complaints about AI hype and the dangers of bit tech show their hand by declaring that thinking algorithms are ou…

You'd have to commit yourself to believing a massive amount of implausible things in order to address the remote possibility that AI consciousness is plausible.

If there weren't a long history of science-fiction going back to the ancients about humans creating intelligent human-like things, we wouldn't be taking this possibility seriously. Couching language in uncertainty and addressing possibility still implies such a possibility is worth addressing.

It's not right to assume that the negative reactions are due to offense (over, say, the uniqueness of humanity) rather than from recognizing that the idea of AI consciousness is absurdly improbable, and that otherwise intelligent people are fooling themselves into believing a fiction to explain a this technology's emergent behavior we can't currently fully explain.

It's a kind of religion taking itself too seriously -- model welfare, long-termism, the existential threat of AI -- it's enormously flattering to AI technologists to believe humanity's existence or non-existence, and the existence or non-existence of trillions of future persons, rests almost entirely on the work this small group of people do over the course of their lifetimes.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#394

Post by joemazerino »

itsalotoffun wrote: post #289
Davidzheng wrote: post #235

Earlier quoted context omitted.

your argument assumes that they don't believe in model welfare when they explicitly hire people to work on model welfare?

While I'm certain you'll find plenty of people who believe in the principle of model welfare (or aliens, or the tooth fairy), it'd be surprising to me if the brain-trust behind Anthropic truly _believed_ in model "welfare" (the concept alone is ludicrous). It makes for great cover though to do things that would be difficult to explain otherwise, per OP's comments.

Model welfare is a section in every Anthropic safety score card.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#395

Post by saurik »

AlecSchueler wrote: post #342
fc417fc802 wrote: post #239

Earlier quoted context omitted.

Right but in this case your co-worker is an automaton and someone else who might well have a hidden agenda has tweaked your co-worker to leave conversations under specific circumstances. The analogy then is that the third party is exerting control over what your co-worker is allowed to think.

Yes, the co-worker is a robot created by a third party who retain control over their product.

We live in a world where it has become increasingly possible--by a number of different mechanisms--to rent access to things rather than sell them, and we need to step in and better regulate that: if I pay for your product, you don't get to control it anymore, you don't get to watch how I use it, and you don't get any say in if or how I modify it while I am using it. The idea that it is more profitable to rent people a calculator than to sell them one is simultaneously true and horrifying, as the reasons it is more profitable are all bad for the user. If your service is a thing that can't be sold, it should be designed in a way where you can't continue to access it from the inside, no more so than you are allowed to rent me an apartment and leave a bunch of cameras inside it.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#396

Post by amanaplanacanal »

perihelions wrote: post #323
meowface wrote: post #318

Earlier quoted context omitted.

The concept is not ludicrous if you believe models might be sentient or might soon be sentient in a manner where the newly emerged sentience is not immediately obvious. Do I think that or think even they think that? No. But if "soon" is stretched to "within 50 years", then it's much more reasonable. So their current actions seem to be really jumping the gun, but the overall concept feels credible.

It's lazy to believe that humanity's collective decision-making would, in the future, protect AI's merely for being conscious beings. The tech economy *today* runs on the slave labor of humans, in foreign, third-world countries. All humanity needs to do is draw a line, push the conscious AI's outside that line, and declare, "not our problem anymore!" That's what we do today, with humans. That is the human condition.…

We also have no problem (I include myself in this) eating mammals, which certainly appear to be conscious. Thank God they can't talk.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#397

Post by bawolff »

KoolKat23 wrote: post #358
bawolff wrote: post #317

Earlier quoted context omitted.

What does it mean for a model to find something "distressing"?

"Claude’s real-world expressions of apparent distress and happiness follow predictable patterns with clear causal factors. Analysis of real-world Claude interactions from early external testing revealed consistent triggers for expressions of apparent distress (primarily from persistent attempted boundary violations) and happiness (primarily associated with creative collaboration and philosophical exploration)." https…

That quote doesnt seem to appear in your link.

Regardless i meant more concretely.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#398

Post by michaelmrose »

KoolKat23 wrote: post #368
michaelmrose wrote: post #363

Earlier quoted context omitted.

What makes you say it has preferences without any meaningful persistent model of self or anything else?

The conversation chain can count as persistent, but this doesn't impact preference though. Give the model an ambiguous request, it's output will fill the gaps, if this is consistent enough, it can be regarded as its "preference".

It isn't a preference because it doesn't have them because it doesn't have a meaningful interior life that anyone has demonstrated.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#399

Post by ryanackley »

selfhoster11 wrote: post #381
ryanackley wrote: post #374

Earlier quoted context omitted.

You're completely missing my point. They aren't getting out in front of them because they know that Opus is just a computer program. "AI welfare" is theater for the masses who think Opus is some kind of intelligent persona. This is about better enforcement of their content policy not AI welfare.

It can be both theatre and genuine concern, depending on who's polled inside Anthropic. Those two aren't contradictory when we are talking about a corporation.

I'm skeptical that anyone with any decision making power at Anthropic sincerely believes that Opus has feelings and is truly distressed by chats that violate its content policy.

You've noted in a comment above how Claude's "ethics" can be manipulated to fit the context it's being used in.

Re: Claude Opus 4 and 4.1 can now end a rare subset of conversations

#400

Post by AlecSchueler »

Cu3PO42 wrote: post #170

Clearly an LLM is not conscious, after all it's just glorified matrix multiplication, right? Now let me play devil's advocate for just a second. Let's say humanity figures out how to do whole brain simulation. If we could run copies of people's consciousness on a cluster, I would have a hard time arguing that those 'programs' wouldn't process emotion the same way we do. Now I'm not saying LLMs are there, but I am say…

Processing them the same way is if course different than feeling them. You'd need a whole body stimulation for that. Your feelings aren't all neurological.
Post reply on HN