Claude 4 • Hacker News Board

Re: Claude 4

#681

Post by gr_norm » Thu, May 22, 2025, 11:54 PM UTC

layoric wrote: post #636

a2128 wrote: post #580

> Finally, we've introduced thinking summaries for Claude 4 models that use a smaller model to condense lengthy thought processes. This summarization is only needed about 5% of the time—most thought processes are short enough to display in full. Users requiring raw chains of thought for advanced prompt engineering can contact sales about our new Developer Mode to retain full access. I don't want to see a "summary" of…

There are several papers pointing towards 'thinking' output is meaningless to the final output, and using dots, or pause tokens enabling the same additional rounds of throughput result in similar improvements. So in a lot of regards the 'thinking' is mostly marketing. - "Think before you speak: Training Language Models With Pause Tokens" - https://arxiv.org/abs/2310.02226 - "Let's Think Dot by Dot: Hidden Computation…

Wow, my first ever video on AI! I'm rather disappointed. That was devoid of meaningful content save for the two minutes where they went over the Anthropic blog post on how LLMs (don't) do addition. Importantly, they didn't remotely approach what those other papers are about, or why thinking tokens aren't important for chain-of-thought. Is all AI content this kind of slop? Sorry, no offense to the above comment, it was just a total waste of 10 minutes that I'm not used to.

So, to anyone more knowledgeable than the proprietor of that channel: can you outline why it's possible to replace thinking tokens with garbage without a decline in output quality?

edit: Section J of the first paper seems to offer some succint explanations.

Re: Claude 4

#682

Post by jackdeansmith » Thu, May 22, 2025, 11:56 PM UTC

sensanaty wrote: post #647

briandw wrote: post #379

This is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further i…

Are the AI companies shooting an amnesia ray at people or something? This is literally the same stupid marketing schtick they tried with ChatGPT back in the GPT-2 days where they were saying they were "terrified of releasing it because it's literally AGI!!!1!1!1!!" And "it has a mind of its own, full sentience it'll hack all the systems by its lonesome!!!", how on earth are people still falling for this crap? It feel…

What would convince you that a system is too dangerous to release?

Re: Claude 4

#683

Post by benoittravers » Thu, May 22, 2025, 11:58 PM UTC

oofbaroomf wrote: post #36

Interesting how Sonnet has a higher SWE-bench Verified score than Opus. Maybe says something about scaling laws.

Do you have the link to that benchmark? Can’t see where Sonnet is highlighted.

Re: Claude 4

#684

Post by einrealist » Fri, May 23, 2025, 12:00 AM UTC

briandw wrote: post #379

This is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further i…

And how many times did this scenario not occur?

Guess, people want to focus on this particular scenario. Does it confirm biases? How strong is the influence of Science Fiction in this urge to discuss this scenario and infer some sort of intelligence?

Re: Claude 4

#685

Post by ls-a » Fri, May 23, 2025, 12:00 AM UTC

briandw wrote: post #379

This is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further i…

Based on my life experience with real humans, this is exactly what most humans would do

Re: Claude 4

#686

Post by 999900000999 » Fri, May 23, 2025, 12:01 AM UTC

Question:

Should I ask it to update an existing project largely written in 3.7 or ask it to start from scratch?

I keep running into an issue where an LLM will get like 75% of a solution working and then the last 25% is somehow impossible to get right.

I don’t expect perfection, but I’ve wasted so much time vibe coding this thing I guess I’d do better to actually program

Re: Claude 4

#687

Post by efitz » Fri, May 23, 2025, 12:03 AM UTC

GuB-42 wrote: post #594

briandw wrote: post #379

This is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further i…

When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…

Only now we are going to connect it to the real world through agents so it can blissfully but uncomprehendingly act out its blackmail story.

Re: Claude 4

#688

Post by lawrenceyan » Fri, May 23, 2025, 12:04 AM UTC

Claude is Buddhist! I’m extremely bullish.

Re: Claude 4

#689

Post by jackdeansmith » Fri, May 23, 2025, 12:04 AM UTC

boh wrote: post #40

Can't wait to hear how it breaks all the benchmarks but have any differences be entirely imperceivable in practice.

In my opinion most Anthropic models are the opposite, scoring well on benchmarks but not always way on top, but quietly excellent when you actually try to use them for stuff.

Re: Claude 4

#690

Post by Workaccount2 » Fri, May 23, 2025, 12:06 AM UTC

DubiousPusher wrote: post #654

gmueckl wrote: post #620

Earlier quoted context omitted.

Isn't the ultimate irony in this that all these stories and rants about out-of-control AIs are now training LLMs to exhibit these exact behaviors that were almost universally deemed bad?

This is a phenomenon I call cinetrope. Films influence the world which in turn influences film and so on creating a feedback effect. For example, we have certain films to thank for an escalation in the tactics used by bank robbers which influenced the creation of SWAT which in turn influenced films like Heat and so on.

What about the cinetrope that human emotion is a magical transcendent power that no machine can ever understand...