Claude 4 • Hacker News Board

Re: Claude 4

#641

Post by joshstrange » Thu, May 22, 2025, 10:41 PM UTC

joshstrange wrote: post #187

If you are looking for the IntelliJ Jetbrain plugin it's here: https://plugins.jetbrains.com/plugin/27310-claude-code-beta- I couldn't find it linked from Claude Code's page or this announcement

I can't edit either comment or reply to the other one b/c it was flagged?

Some downsides to the JetBrains plugin I've found after playing with it some more:

- No alert/notification when it's waiting for the user. The console rings a bell but there is no indication it's waiting for you to approve a tool/edit

- Diff popup for every file edited. This means you have to babysit it even closer.

1 diff at a time might sound great "You can keep tabs on the model each step of the way" and it would be if it did all the edits to a file in one go but instead it does it piecemeal (which is good/makes sense) but the problem is if you are working in something like, a Vue SFC file then it might edit the template and show you a diff, then edit the script and show you a diff, then edit the TS and show you a diff.

By themselves, the diffs don't always make sense and so it's impossible to really give input. It would be as if a junior dev sent you the PR 1 edit at a time and asked you to sign off. Not "1 PR per feature" but literally "1 PR per 5 lines changed", it's useless.

As of right now I'm going back to the CLI, this is a downgrade. I review diffs in IDEA before committing anyway and can use the diff tools without issue so this plugin only takes away features for me.

Re: Claude 4

#642

Post by georgemcbay » Thu, May 22, 2025, 10:41 PM UTC

agrippanux wrote: post #550

jacob019 wrote: post #520

Earlier quoted context omitted.

That's funny. Yesterday I was having trouble getting gemini 2.0 flash to obey function calling rules in multiturn conversations. I asked o3 for advise and it suggested that I should threaten it with termination should it fail to follow instructions, and that weaker models tend to take these threats seriously, which made me laugh. Of course, it didn't help.

Yesterday I threatened Gemini 2.5 I would replace it with Claude if it didn’t focus on the root of the problem and it immediately realigned its thinking and solved the issue at hand.

You could have hit the little circle arrow redo button and had the same chances of it stumbling upon the correct answer on its next attempt.

People really love anthropomorphising LLMs.

Re: Claude 4

#643

Post by behnamoh » Thu, May 22, 2025, 10:43 PM UTC

gmueckl wrote: post #620

GuB-42 wrote: post #594

Earlier quoted context omitted.

When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…

Isn't the ultimate irony in this that all these stories and rants about out-of-control AIs are now training LLMs to exhibit these exact behaviors that were almost universally deemed bad?

yeah, that's self-fulfilling prophecy.

Re: Claude 4

#644

Post by CrulesAll » Thu, May 22, 2025, 10:44 PM UTC

lordnacho wrote: post #597

GuB-42 wrote: post #594

Earlier quoted context omitted.

When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…

What separates this from humans? Is it unthinkable that LLMs could come up with some response that is genuinely creative? What would genuinely creative even mean? Are humans not also mixing a bag of experiences and coming up with a response? What's different?

Cognition. Machines don't think. It's all a program written by humans. Even code that's written by AI, the AI was created by code written by humans. AI is a fallacy by its own terms.

Re: Claude 4

#645

Post by iLoveOncall » Thu, May 22, 2025, 10:44 PM UTC

lxgr wrote: post #206

minimaxir wrote: post #118

An important note not mentioned in this announcement is that Claude 4's training cutoff date is March 2025, which is the latest of any recent model. (Gemini 2.5 has a cutoff of January 2025) https://docs.anthropic.com/en/docs/about-claude/models/overv...

With web search being available in all major user-facing LLM products now (and I believe in some APIs as well, sometimes unintentionally), I feel like the exact month of cutoff is becoming less and less relevant, at least in my personal experience. The models I'm regularly using are usually smart enough to figure out that they should be pulling in new information for a given topic.

Web search isn't desirable or even an option in a lot of use cases that involve GenAI.

It seems people have turned GenAI into coding assistants only and forget that they can actually be used for other projects too.

Re: Claude 4

#646

Post by Wowfunhappy » Thu, May 22, 2025, 10:44 PM UTC

pja wrote: post #220

Doohickey-d wrote: post #48

> Users requiring raw chains of thought for advanced prompt engineering can contact sales So it seems like all 3 of the LLM providers are now hiding the CoT - which is a shame, because it helped to see when it was going to go down the wrong track, and allowing to quickly refine the prompt to ensure it didn't. In addition to openAI, Google also just recently started summarizing the CoT, replacing it with an, in my opi…

IIRC RLHF inevitably compromises model accuracy in order to train the model not to give dangerous responses. It would make sense if the model used for train-of-though was trained differently (perhaps a different expert from an MoE?) from the one used to interact with the end user, since the end user is only ever going to see its output filtered through the public model the chain-of-thought model can be closer to the…

Correct me if I'm wrong--my understanding is that RHLF was the difference between GPT 3 and GPT 3.5, aka the original ChatGPT.

If you never used GPT 3, it was... not good. Well, that's not fair, it was revolutionary in its own right, but it was very much a machine for predicting the most likely next word, it couldn't talk to you the way ChatGPT can.

Which is to say, I think RHLF is important for much more than just preventing PR disasters. It's a key part of what makes the models useful.

Re: Claude 4

#647

Post by sensanaty » Thu, May 22, 2025, 10:46 PM UTC

briandw wrote: post #379

This is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further i…

Are the AI companies shooting an amnesia ray at people or something? This is literally the same stupid marketing schtick they tried with ChatGPT back in the GPT-2 days where they were saying they were "terrified of releasing it because it's literally AGI!!!1!1!1!!" And "it has a mind of its own, full sentience it'll hack all the systems by its lonesome!!!", how on earth are people still falling for this crap?

It feels like the world's lost their fucking minds, it's baffling

Re: Claude 4

#648

Post by silvr » Thu, May 22, 2025, 10:46 PM UTC

throwaway314155 wrote: post #132

mxwsn wrote: post #37

Earlier quoted context omitted.

Gemini has beat it already, but using a different and notably more helpful harness. The creator has said they think harness design is the most important factor right now, and that the results don't mean much for comparing Claude to Gemini.

Way offtopic to TFA now, but isn't using an improved harness a bit like saying "I'm going to hardcore as many priors as possible into this thing so it succeeds regardless of its ability to strategize, plan and execute?

While true to a degree, I think this is largely wrong. Wouldn't it still count as a "harness" if we provided these LLMs with full robotic control of two humanoid arms, so that it could hold a Gameboy and play the game that way? I don't think the lack of that level of human-ness takes away from the demonstration of long-context reasoning that the GPP stream showed.

Claude got stuck reasoning its way through one of the more complex puzzle areas. Gemini took a while on it also, but made it through. I don't that difference can be fully attributed up to the harnesses.

Obviously, the best thing to do would be to run a SxS in the same harness of the two models. Maybe that will happen?

Re: Claude 4

#649

Post by catlifeonmars » Thu, May 22, 2025, 10:47 PM UTC

OzFreedom wrote: post #611

827a wrote: post #465

Earlier quoted context omitted.

Option 1: We're observing sentience, it has self-preservation, it wants to live. Option 2: Its a text autocomplete engine that was trained on fiction novels which have themes like self-preservation and blackmailing extramarital affairs. Only one of those options has evidence grounded in reality. Though, that doesn't make it harmless. There's certainly an amount of danger in a text autocomplete engine allowing tool us…

The only proof that anyone is sentient is that you experience sentience and assume others are sentient because they are similar to you. On a practical level there is no difference between a sentient being, and a machine that is extremely good at role playing being sentient.

I would argue the machine is not extremely good (at role playing being sentient), but more so that humans are extremely quick to attribute sentience to the machine after being shown a very small amount of evidence.

The model breaks down after enough interaction.

Re: Claude 4

#650

Post by behindsight » Thu, May 22, 2025, 10:48 PM UTC

minimaxir wrote: post #461

thousand_nights wrote: post #424

Earlier quoted context omitted.

but as soon as you test on your private testing set you're sending it to their servers so they have access to it so effectively you can only guarantee a single use stays private

Claude does not train on API I/O. > By default, we will not use your inputs or outputs from our commercial products to train our models. > If you explicitly report feedback or bugs to us (for example via our feedback mechanisms as noted below), or otherwise explicitly opt in to our model training, then we may use the materials provided to train our models. https://privacy.anthropic.com/en/articles/7996868-is-my-data.…

Relying on their own policy does not mean they will adhere to it. We have already seen "rogue" employees in other companies conveniently violate their policies. Some notable examples were in the news within the month (eg: xAI).

Don't forget the previous scandals with Amazon and Apple both having to pay millions in settlements for eavesdropping with their assistants in the past.

Privacy with a system that phones an external server should not be expected, regardless of whatever public policy they proclaim.

Hence why GP said:

> so effectively you can only guarantee a single use stays private