211–220 of 1001 posts
Anyone know if this is usable with Claude Code? If so, how? I've not seen the ability to configure the backend for Claude Code, hmm
Interesting how Sonnet has a higher SWE-bench Verified score than Opus. Maybe says something about scaling laws.
BEFORE: claude-3-7-sonnet
AFTER: claude-sonnet-4
Have they documented the context window changes for Claude 4 anywhere? My (barely informed) understanding was one of the reasons Gemini 2.5 has been so useful is that it can handle huge amounts of context --- 50-70kloc?
Other tools may drop some prior context, or use RAG to help but they don't force you to start a new chat without warning.
Is this really worthy of a claude 4 label? Was there a new pre-training run? Cause this feels like 3.8... only swe went up significantly, and that as we all understand by now is done by cramming on specific post training data and doesn't generalize to intelligence. The agentic tooluse didn't improve and this says to me that it's not really smarter.
So I decided to try Claude 4 Sonnet against my "Given a list of 1 million random integers between 1 and 100,000, find the difference between the smallest and the largest numbers whose digits sum up to 30." benchmark I tested against Claude 3.5 Sonnet: https://news.ycombinator.com/item?id=42584400 The results are here ( https://gist.github.com/minimaxir/1bad26f0f000562b1418754d67... ) and it utterly crushed the proble…
But have you checked with some other number than 30? Does it screw up the upper and lower bounds?
Claude 3.8 wrote me some code this morning, and I was running into a bug. I switched to 4 and gave it its own code. It pointed out the bug right away and fixed it. So an upgrade for me :-)
> Users requiring raw chains of thought for advanced prompt engineering can contact sales So it seems like all 3 of the LLM providers are now hiding the CoT - which is a shame, because it helped to see when it was going to go down the wrong track, and allowing to quickly refine the prompt to ensure it didn't. In addition to openAI, Google also just recently started summarizing the CoT, replacing it with an, in my opi…
It would make sense if the model used for train-of-though was trained differently (perhaps a different expert from an MoE?) from the one used to interact with the end user, since the end user is only ever going to see its output filtered through the public model the chain-of-thought model can be closer to the original, more pre-rlhf version without risking the reputation of the company.
This way you can get the full performance of the original model whilst still maintaining the necessary filtering required to prevent actual harm (or terrible PR disasters).