Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 4

anthropic.com

Post reply on HN
Print view View original

171–180 of 1001 posts

Re: Claude 4

#171

Post by swyx »

wrsh07 wrote: post #162
KaoruAoiShiho wrote: post #61

Is this really worthy of a claude 4 label? Was there a new pre-training run? Cause this feels like 3.8... only swe went up significantly, and that as we all understand by now is done by cramming on specific post training data and doesn't generalize to intelligence. The agentic tooluse didn't improve and this says to me that it's not really smarter.

My understanding for the original OpenAI and anthropic labels was essentially: gpt2 was 100x more compute than gpt1. Same for 2 to 3. Same for 3 to 4. Thus, gpt 4.5 was 10x more compute^ If anthropic is doing the same thing, then 3.5 would be 10x more compute vs 3. 3.7 might be 3x more than 3.5. and 4 might be another ~3x. ^ I think this maybe involves words like "effective compute", so yeah it might not be a full pr…

beyond 4 thats no longer true - marketing took over from the research

Re: Claude 4

#172

Post by martinsnow »

archon1410 wrote: post #121

The naming scheme used to be "Claude [number] [size]", but now it is "Claude [size] [number]". The new models should have been named Claude 4 Opus and Claude 4 Sonnet, but they changed it, and even retconned Claude 3.7 Sonnet into Claude Sonnet 3.7. Annoying.

It seems like investors have bought into the idea that llms has to improve no matter what. I see it in the company I'm currently at. No matter what we have to work with whatever bullshit these models can output. I am however looking at more responsible companies for new employment.

Re: Claude 4

#173

Post by ashirviskas »

miroljub wrote: post #78
anal_reactor wrote: post #56

Earlier quoted context omitted.

I've been initially fascinated by Claude, but then I found myself drawn to Deepseek. My use case is different though, I want someone to talk to.

I also use DeepSeek R1 as a daily driver. Combined with Qwen3 when I need better tool usage. Now that both Google and Claude are out, I expect to see DeepSeek R2 released very soon. It would be funny to watch an actual open source model getting close to the commercial competition.

Have you compared R1 with V3-0324?

Re: Claude 4

#174

Post by esafak »

jbellis wrote: post #94
tptacek wrote: post #83

Have they documented the context window changes for Claude 4 anywhere? My (barely informed) understanding was one of the reasons Gemini 2.5 has been so useful is that it can handle huge amounts of context --- 50-70kloc?

not sure wym, it's in the headline of the article that Opus 4 has 200k context (same as sonnet 3.7 with the beta header)

There's the nominal context length, and the effective one. You need a benchmark like the needle-in-a-haystack or RULER to determine the latter.

https://github.com/NVIDIA/RULER

Re: Claude 4

#175

Post by douglasisshiny »

candiddevmike wrote: post #68
uludag wrote: post #46

I'm curious what are others priors when reading benchmark scores. Obviously with immense funding at stakes, companies have every incentive to game the benchmarks, and the loss of goodwill from gaming the system doesn't appear to have much consequences. Obviously trying the model for your use cases more and more lets you narrow in on actually utility, but I'm wondering how others interpret reported benchmarks these da…

People's interpretation of benchmarks will largely depend on whether they believe they will be better or worse off by GenAI taking over SWE jobs. Think you'd need someone outside the industry to weigh in to have a real, unbiased view.

Or someone who has been a developer for a decade plus trying to use these models on actual existing code bases, solving specific problems. In my experience, they waste time and money.

Re: Claude 4

#176

Post by m3kw9 »

minimaxir wrote: post #118

An important note not mentioned in this announcement is that Claude 4's training cutoff date is March 2025, which is the latest of any recent model. (Gemini 2.5 has a cutoff of January 2025) https://docs.anthropic.com/en/docs/about-claude/models/overv...

Even that, we don’t know what got updated and what didn’t. Can we assume everything that can be updated is updated?

Re: Claude 4

#178

Post by rco8786 »

jasonthorsness wrote: post #16

“GitHub says Claude Sonnet 4 soars in agentic scenarios and will introduce it as the base model for the new coding agent in GitHub Copilot.” Maybe this model will push the “Assign to CoPilot” closer to the dream of having package upgrades and other mostly-mechanical stuff handled automatically. This tech could lead to a huge revival of older projects as the maintenance burden falls.

It could be! But that's also what people said about all the models before it!

Re: Claude 4

#179

Post by minimaxir »

KaoruAoiShiho wrote: post #61

Is this really worthy of a claude 4 label? Was there a new pre-training run? Cause this feels like 3.8... only swe went up significantly, and that as we all understand by now is done by cramming on specific post training data and doesn't generalize to intelligence. The agentic tooluse didn't improve and this says to me that it's not really smarter.

So I decided to try Claude 4 Sonnet against my "Given a list of 1 million random integers between 1 and 100,000, find the difference between the smallest and the largest numbers whose digits sum up to 30." benchmark I tested against Claude 3.5 Sonnet: https://news.ycombinator.com/item?id=42584400

The results are here (https://gist.github.com/minimaxir/1bad26f0f000562b1418754d67... ) and it utterly crushed the problem with the relevant microoptimizations commented in that HN discussion (oddly in the second pass it a) regresses from a vectorized approach to a linear approach and b) generates and iterates on three different iterations instead of one final iteration), although it's possible Claude 4 was trained on that discussion lol.

EDIT: "utterly crushed" may have been hyperbole.

Re: Claude 4

#180

Post by simlevesque »

m3kw9 wrote: post #176
minimaxir wrote: post #118

An important note not mentioned in this announcement is that Claude 4's training cutoff date is March 2025, which is the latest of any recent model. (Gemini 2.5 has a cutoff of January 2025) https://docs.anthropic.com/en/docs/about-claude/models/overv...

Even that, we don’t know what got updated and what didn’t. Can we assume everything that can be updated is updated?

You might be able to ask it what it knows.
Post reply on HN