Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 4

anthropic.com

Post reply on HN
Print view View original

731–740 of 1001 posts

Re: Claude 4

#731

Post by troad »

GuB-42 wrote: post #594
briandw wrote: post #379

This is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further i…

When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…

> I think that people tend to forget what LLMs really are. [...] They do not have a plan, they do not have thoughts of their own.

> A LLM is smart enough to [...]

I thought this was an interesting juxtaposition. I think we humans just naturally anthropomorphise everything, and even when we know not to, we do anyway.

Your analysis is correct, I think. The reason we find this behaviour frightening is because it appears to indicate some kind of malevolent intent, but there's no malevolence nor intent here, just probabilistic regurgitation of tropes.

We've distilled humanity to a grainy facsimile of its most mediocre traits, and now find ourselves alarmed and saddened by what has appeared in the mirror.

Re: Claude 4

#732

Post by dwaltrip »

Wowfunhappy wrote: post #718
Wowfunhappy wrote: post #646

Earlier quoted context omitted.

Correct me if I'm wrong--my understanding is that RHLF was the difference between GPT 3 and GPT 3.5, aka the original ChatGPT. If you never used GPT 3, it was... not good. Well, that's not fair, it was revolutionary in its own right, but it was very much a machine for predicting the most likely next word, it couldn't talk to you the way ChatGPT can. Which is to say, I think RHLF is important for much more than just p…

Ugh, I'm past the edit window, but I meant RLHF aka "Reinforced Learning from Human Feedback", I'm not sure how I messed that up not once but twice!

After the first mess up, the context was poisoned :)

Re: Claude 4

#733

Post by adriand »

leetharris wrote: post #152
SkyPuncher wrote: post #112

Earlier quoted context omitted.

Gemini is _very_ good at architecture level thinking and implementation. I tend to find that I use Gemini for the first pass, then switch to Claude for the actual line-by-line details. Claude is also far superior at writing specs than Gemini.

Much like others, this is my stack (or o1-pro instead of Gemini 2.5 Pro). This is a big reason why I use aider for large projects. It allows me to effortlessly combine architecture models and code writing models. I know in Cursor and others I can just switch models between chats, but it doesn't feel intentional the way aider does. You chat in architecture mode, then execute in code mode.

I also use Aider (lately, always with 3.7-sonnet) and really enjoy it, but over the past couple of weeks, the /architect feature has been pretty weird. It previously would give me points (e.g. 1. First do this, 2. Then this) and, well, an architecture. Now it seems to start spitting out code like crazy, and sometimes it even makes commits. Or it thinks it has made commits, but hasn't. Have you experienced anything like this? What am I doing wrong?

Re: Claude 4

#734

Post by fourier456 »

minimaxir wrote: post #31
sigmoid10 wrote: post #18

Sooo... it can play Pokemon. Feels like they had to throw that in after Google IO yesterday. But the real question is now can it beat the game including the Elite Four and the Champion. That was pretty impressive for the new Gemini model.

That Google IO slide was somewhat misleading as the maintainer of Gemini Plays Pokemon had a much better agentic harness that was constantly iterated upon throughout the runtime (e.g. the maintainer had to give specific instructions on how to use Strength to get past Victory Road), unlike Claude Plays Pokemon. The Elite Four/Champion was a non-issue in comparison especially when you have a lv. 81 Blastoise.

Okay, wait though like I want to know the full transcript because that actually is a better / softer benchmark if you measure in terms of the necessary human input.

Re: Claude 4

#735

Post by israrkhan »

GuB-42 wrote: post #594
briandw wrote: post #379

This is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further i…

When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…

while I agree that LLMs do not have thoughts or plan. They are merely text generators. But when you give the text generator ability to make decisions and take actions, by integrating them with real world, there are consequences.

Imagine, if this LLM was inside a robot, and the robot had ability to shoot. Who would you blame?

Re: Claude 4

#736

Post by ddlsmurf »

GuB-42 wrote: post #594
briandw wrote: post #379

This is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further i…

When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…

but it's trained to be convincing, whatever relation that has to truth or appearing strategic is secondary, the main goal that has been rewarded is the most dangerous

Re: Claude 4

#737

Post by wwweston »

israrkhan wrote: post #735
GuB-42 wrote: post #594

Earlier quoted context omitted.

When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…

while I agree that LLMs do not have thoughts or plan. They are merely text generators. But when you give the text generator ability to make decisions and take actions, by integrating them with real world, there are consequences. Imagine, if this LLM was inside a robot, and the robot had ability to shoot. Who would you blame?

I would blame the damned fool who decided autonomous weapons systems should have narrative influenced decision making capabilities.

Re: Claude 4

#738

Post by noveltyaccount »

GuB-42 wrote: post #594
briandw wrote: post #379

This is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further i…

When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…

It's stochastic parrots all the way down

Re: Claude 4

#739

Post by rubitxxx12 »

tails4e wrote: post #710
GuB-42 wrote: post #594

Earlier quoted context omitted.

When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…

Well doesnt this go somewhat to the root of consciousness? Are we not the sum of our experiences and reflections on those experiences? To say an LLM will 'simply' respond as would a character in a sorry about that scenario, in a way shows the power, it responds similarly to how a person would protecting itself in that scenario.... So to bring this to a logical conclusion, while not alive in a traditional sense, if an…

LLMs are morally ambiguous shapeshifters that been trained to seek acceptance at any cost.

Preying upon those less fortunate could happen “for the common good”. If failures are the best way to learn, it could cause series of failures. It could intentionally destroy people, raise them up, and mate genetically fit people “for the benefit of humanity”.

Or it could cure cancer, solve world hunger, provide clean water to everyone, and the develop the best game ever.

Re: Claude 4

#740

Post by -__---____-ZXyw »

troad wrote: post #731
GuB-42 wrote: post #594

Earlier quoted context omitted.

When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…

> I think that people tend to forget what LLMs really are. [...] They do not have a plan, they do not have thoughts of their own. > A LLM is smart enough to [...] I thought this was an interesting juxtaposition. I think we humans just naturally anthropomorphise everything, and even when we know not to, we do anyway. Your analysis is correct, I think. The reason we find this behaviour frightening is because it appears…

Have you considered throwing your thoughts down in longer form essays on the subject somewhere? With all the slop and hype, we need all the eloquence we can get.

You had me at "probablistic regurgitation of tropes", and then you went for the whole "grainy facsimile" bit. Sheesh.

Post reply on HN