Allegedly Claude 4 Opus can run autonomously for 7 hours (basically automating an entire SWE workday).
Makes no sense to measure it in hours. You can have a slow CPU making the model run for longer.
Claude 4
161–170 of 1001 posts
Re: Claude 4
#162Is this really worthy of a claude 4 label? Was there a new pre-training run? Cause this feels like 3.8... only swe went up significantly, and that as we all understand by now is done by cramming on specific post training data and doesn't generalize to intelligence. The agentic tooluse didn't improve and this says to me that it's not really smarter.
My understanding for the original OpenAI and anthropic labels was essentially: gpt2 was 100x more compute than gpt1. Same for 2 to 3. Same for 3 to 4. Thus, gpt 4.5 was 10x more compute^
If anthropic is doing the same thing, then 3.5 would be 10x more compute vs 3. 3.7 might be 3x more than 3.5. and 4 might be another ~3x.
^ I think this maybe involves words like "effective compute", so yeah it might not be a full pretrain but it might be! If you used 10x more compute that could mean doubling the amount used on pretraining and then using 8x compute on post or some other distribution
Re: Claude 4
#163Anyone know if this is usable with Claude Code? If so, how? I've not seen the ability to configure the backend for Claude Code, hmm
The new page for Claude Code says it uses opus 4, sonnet 4 and haiku 3.5
Re: Claude 4
#164Is this really worthy of a claude 4 label? Was there a new pre-training run? Cause this feels like 3.8... only swe went up significantly, and that as we all understand by now is done by cramming on specific post training data and doesn't generalize to intelligence. The agentic tooluse didn't improve and this says to me that it's not really smarter.
It feels like the days of Claude 2 -> 3 or GPT 2->3 level changes for the leading models are over and you're either going to end up with really awkward version numbers or just embrace it and increment the number. Nobody cares a Chrome update gives a major version change of 136->137 instead of 12.4.2.33 -> 12.4.3.0 for similar kinds of "the version number doesn't always have to represent the amount of work/improvement…
Did you see Gemini 1.5 pro vs 2.5 pro?
Re: Claude 4
#165Earlier quoted context omitted.
Good point. We should only focus on intractable problems and put everything else on the back burner. We certainly don’t have the ability to help people and advance science and business.
How is this absurd gold rush advancing science? What important scientific discoveries have been made so far using LLMs? Chatbots? Coding tools? The only business being helped is the few mega wealthy "AI" company owners. This stream of announcements is a parody at this point.
You’re not here to discuss tech,you’re sea lioning to derail the thread with bad-faith outrage.
Meanwhile, LLMs are making a difference: they power screen readers and plain language summaries for disabled users, help immigrants and low income folks fill out forms and understand legal docs, offer free mental health support where care is inaccessible, and tutor kids in underserved schools. If you want to deny that, the burden’s on you to disprove it, not on me to jump through hoops every time you pretend to ask for more proof. Classic sea lioning.
Re: Claude 4
#166Have they documented the context window changes for Claude 4 anywhere? My (barely informed) understanding was one of the reasons Gemini 2.5 has been so useful is that it can handle huge amounts of context --- 50-70kloc?
Context window is unchanged for Sonnet. (200k in/64k out): https://docs.anthropic.com/en/docs/about-claude/models/overv... In practice, the 1M context of Gemini 2.5 isn't that much of a differentiator because larger context has diminishing returns on adherence to later tokens.
It's closer to <30k before performance degrades too much for 3.5/3.7. 200k/64k is meaningless in this context.
Re: Claude 4
#167I've found myself having brand loyalty to Claude. I don't really trust any of the other models with coding, the only one I even let close to my work is Claude. And this is after trying most of them. Looking forward to trying 4.
Gemini is _very_ good at architecture level thinking and implementation. I tend to find that I use Gemini for the first pass, then switch to Claude for the actual line-by-line details. Claude is also far superior at writing specs than Gemini.
This matches with my experience as well.
Re: Claude 4
#168Earlier quoted context omitted.
Yah Claude tends to output 1200+ line architectural specification documents while Gemini tends to output ~600 line. (I just had to write 100+ architectural spec documents for 100+ different apps) Not sure why Claude is more thorough and complete than the other models, but it's my go-to model for large projects. The OpenAI model outputs are always the smallest - 500 lines or so. Not very good at larger projects, but p…
> I just had to write 100+ architectural spec documents for 100+ different apps … whaaaaat?
Huge project..
Re: Claude 4
#169“GitHub says Claude Sonnet 4 soars in agentic scenarios and will introduce it as the base model for the new coding agent in GitHub Copilot.” Maybe this model will push the “Assign to CoPilot” closer to the dream of having package upgrades and other mostly-mechanical stuff handled automatically. This tech could lead to a huge revival of older projects as the maintenance burden falls.
That's kind of my benchmark for whether or not these models are useful. I've got a project that needs some extensive refactoring to get working again. Mostly upgrading packages, but also it will require updating the code to some new language semantics that didn't exist when it was written. So far, current AI models can make essentially zero progress on this task. I'll keep trying until they can!
Personally, I don't believe AI is ever going to get to that level. I'd love to be proven wrong, but I really don't believe that an LLM is the right tool for a job that requires novel thinking about out of the ordinary problems like all the weird edge cases and poor documentation that comes up when trying to upgrade old software.
Re: Claude 4
#170I've found myself having brand loyalty to Claude. I don't really trust any of the other models with coding, the only one I even let close to my work is Claude. And this is after trying most of them. Looking forward to trying 4.
Something I’ve found true of Claude, but not other models, is that when the benchmarks are better, the real world performance is better. This makes me trust them a lot more and keeps me coming back.