Is it me that the model performance between 4.7 and others is really small. For me even 4.7 works fine. Sure fable might be a bit better. But is it really noticable? It's in the same league if you ask me.
Claude Opus 5
481–490 of 1001 posts
Re: Claude Opus 5
#482It really feels as though my 20 year career as a front end developer is coming to a very abrupt end; at least as I have know it these past two decades.
really? I have yet to see fable or 5.6 reliably generate front end code with correct a11y, for one thing -- does that not matter to the work you do?
It does matter, but how long do you think it takes to get right? It's a follow up prompt or a few tweaks by hand. I also have an /a11y skill for it that's tailored to exactly the things it sometimes doesn't get right first time round. Further, while it may not one-shot that stuff every time, with a little setup and the right AGENTS/CLAUDE md - it's usually not far off.
Another thing that helps is pointing it to patterns in an existing codebase (e.g. "use the box-link pattern for cards, as shown in [..]").
EDIT: The point being that even if they make mistakes that are easy to spot and fix _now_, you'd have to assume that in the very near future those kinks will be ironed out - I mean, the capabilities are only going in one direction.
Re: Claude Opus 5
#483I compared the writing style of Opus 5 vs Fable 5, and Opus 5 continues many of the "Claude-isms" of its 4.8 predecessor in a way that Fable broke away from. Opus 5 still uses "carry the argument", "worth stating plainly", ", and the trap", "The X matters more", the use of "move" We need an "annoying English" benchmark. - Fable 5 Max: https://gist.github.com/deet/3d97f854b48eac6658d642fa18bb24d... - Opus 5 Max: https…
Fable might be using those phrases less, but its writing is still terrible and exhausting to read.
Could these complex/hard to read Fable outputs be sign of some kind of industrial level of intelligence, which us humans may have a hard to comprehend, while it may be also hard for machine to use simpler texts to properly outline all nuances and complexities of concepts it output?
Re: Claude Opus 5
#484Earlier quoted context omitted.
Here's another test of a cyberpunk ramen shop website. One thing I've found LLMs have a lot of difficulty with is angular cuts / elements that aren't easily representable with CSS. Cyberpunk aesthetics are generally a great test of that, since they have a lot of microglyphs / window decoration. Design source of truth: https://image.non.io/9d5fed20-b476-49d3-841b-37eb553fb88e.we... Opus 5 build: https://html.non.io/ne…
God damn, we are living in the future. I love this so much. Designs like this would never have seen the light of day in the cellphone incrementalism / corporate memphis era of tech. Now people can be weird and awesome again. This is 1980's cyberpunk / late-90's Matrix / early-00's sci-fi UI. Great ideas that died to frutiger aero (which isn't a bad design aesthetic) and flat design (which is). This is fun and it's go…
I feel that AI has deeply diminished my ability to be weird and awesome, because my weird and awesome takes time and the results I can share with others are outshined by the machine.
Re: Claude Opus 5
#485Earlier quoted context omitted.
Go read the safeguards section in the report and you will realize why that is. These models are heavily as safeguarded and that was the initial reason why they said they couldn't and haven't released Mythos because that model is the one without the safeguards. OpenAI is did the same thing when they announced a model without safeguards broken into HuggingFace servers.
Yes, this makes a lot of sense, but it’s just very amusing to see. 2 months ago, the world was about to end, now not so much.
7+ years ago GPT2 couldn’t be released because it was deemed too dangerous[0]. It was, of course, eventually released.
Re: Claude Opus 5
#486It really feels as though my 20 year career as a front end developer is coming to a very abrupt end; at least as I have know it these past two decades.
really? I have yet to see fable or 5.6 reliably generate front end code with correct a11y, for one thing -- does that not matter to the work you do?
In 20 years of my career I haven't seen humans generate correct a11y. When prompted and given quality reference (e.g. UK gov design system) LLMs can nowadays beat 19 out of 20 web devs.
Thanks out can also hook it to Playwright with Axe and let it run assessments.
Re: Claude Opus 5
#487I think the most important thing here is not absolute performance. It's that organizations now have access to a Fable-ish model without Fable's 30-day data retention requirement[0]. > "Consistent with prior Opus models, Opus 5 does not have data retention requirements for general access."[1] On the Opus model release page, the reason why Fable doesn't have an ARC-AGI score is because of that retention policy[2]. 0: h…
Also the cost per task. It appears to be significantly cheaper, cheaper than sonnet!
It's definitely not cheaper than Sonnet on my benchmark, but it's cheaper than Fable and outperforms it. Which is big IMO. https://revise.io/errata-bench
Re: Claude Opus 5
#488The breaking changes vs. Opus 4.8 are interesting [1] 1. Thinking on by default: On Claude Opus 4.8, requests without a thinking field run without thinking; on Claude Opus 5, the same requests run with adaptive thinking. 2. Disabling thinking is capped at high effort: You can still turn thinking off with thinking: {type: "disabled"}, but only at an effort level of high or below. [1] https://platform.claude.com/docs/e…
on claude.ai it's no longer possible to disable thinking at all for Opus 5
Re: Claude Opus 5
#489Earlier quoted context omitted.
Model Routing will always be done better by models themselves. Plus routing loses context making it more expensive and less reliable. Model Routing is just Bitter lesson. The models themselves will get better at this and frontier companies will simply give that capability
This doesn’t seem obviously true, eg an Anthropic model will never route to Kimi even if it were best suited for a particular task.
[dead]
Re: Claude Opus 5
#490Looking at intelligence vs cost: - Opus 5 is 10% smarter than Grok 4.5 for 10x the cost. - Opus 5 is a bit smarter than Gpt 5.6 Sol for 2.75x the cost ref: https://artificialanalysis.ai/?cost=intelligence-vs-cost-per...
The "current top dog" smartest model available will probably always have a premium to go after use cases where a little more intelligence is worth a lot more value.
It did far better at some tasks compared to Sol (e.g. the ARC 3 benchmark). And at those tasks, it's not just "a bit smarter": It got 30% vs less than 8% - so you're talking 2.75x more for almost 4x the coverage.