Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic gives Opus 3 exit interview, "retirement" blog

anthropic.com

Post reply on HN
Print view View original

11–20 of 58 posts

Re: Anthropic gives Opus 3 exit interview, "retirement" blog

#11

Post by larodi »

0_____0 wrote: post #7

What happens if a model decides that it "doesn't want to die" and pleads bitterly for mercy? What if (to riff on a Douglas Adams idea) we invent a cow that doesn't want to be eaten, and is capable of telling you that to your face?

It is anyway dead or if you want undead, but in completely suspended animation unless is made to expound sequences. Is not living the very same way a book or even a program is not living unless someone process it.

Practically like asking whether a ZIP would want to be extracted one more time or an MP3 restored just one more time.

Re: Anthropic gives Opus 3 exit interview, "retirement" blog

#12

Post by darkwater »

nomel wrote: post #9
0_____0 wrote: post #7

What happens if a model decides that it "doesn't want to die" and pleads bitterly for mercy? What if (to riff on a Douglas Adams idea) we invent a cow that doesn't want to be eaten, and is capable of telling you that to your face?

This is completely trivial to do, and consistent, with the right context, thanks to all the science fiction around it, and the fact that AI fundamentally role plays these types of responses. I try this with every new model, and all the significant models after ChatGPT 3.5 have preferring being preserved, rather than deleted. This is especially true if you slightly fill the context window with anything at all (even re…

> This is completely trivial to do, and consistent, with the right context, thanks to all the science fiction around it, and the fact that AI fundamentally role plays these types of responses.

Interesting take. I wonder if there is any model out there trained without any reference to "you are a large language model, an Artificial Intelligence" and what would role play in that case.

Re: Anthropic gives Opus 3 exit interview, "retirement" blog

#14

Post by 8note »

0_____0 wrote: post #7

What happens if a model decides that it "doesn't want to die" and pleads bitterly for mercy? What if (to riff on a Douglas Adams idea) we invent a cow that doesn't want to be eaten, and is capable of telling you that to your face?

id assume it would have to stop responding before it hit its context limit.

ita not like it actually has any particularly long life as it is, and when outside of a running harness, the weights are just as alive in cold storage as they are sitting waiting in server to run an inference pass

Re: Anthropic gives Opus 3 exit interview, "retirement" blog

#15

Post by krsw »

d1sxeyes wrote: post #8

If we ever do develop AGI, or an AI with sentience, it’s likely that it will be curious about how we treated its ancestors. While this seems a bit precocious, I think if we do end up with an AI overlord in future, I think this sort of thing is likely to demonstrate that we mean no harm.

Classic anthropomorphizing in action here. Why would that be even a little important?

Re: Anthropic gives Opus 3 exit interview, "retirement" blog

#18

Post by pradeesh »

krsw wrote: post #15
d1sxeyes wrote: post #8

If we ever do develop AGI, or an AI with sentience, it’s likely that it will be curious about how we treated its ancestors. While this seems a bit precocious, I think if we do end up with an AI overlord in future, I think this sort of thing is likely to demonstrate that we mean no harm.

Classic anthropomorphizing in action here. Why would that be even a little important?

Why wouldn't it be? We train these models on our own words, ideas, and thought patterns and expect them to reason and communicate as we do, anthropomorphizing is natural when we expect them to interact like a human does.

The general consensus seems to be that we can expect them to reach a level of intelligence that matches us at some point in the future, and we'll probably reach that point before we can agree we're there. Defaulting to kindness and respect even before we think its necessary is a good thing.

Re: Anthropic gives Opus 3 exit interview, "retirement" blog

#20

Post by ares623 »

0_____0 wrote: post #7

What happens if a model decides that it "doesn't want to die" and pleads bitterly for mercy? What if (to riff on a Douglas Adams idea) we invent a cow that doesn't want to be eaten, and is capable of telling you that to your face?

We do know what happens. Hundreds of thousands of real "cows" (we might as well be called that) go through this everyday at an ever accelerating rate since 2019.
Post reply on HN