Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Fable 5.1 and Claude Mythos 5.1

anthropic.com

Post reply on HN
Print view View original

221–230 of 1001 posts

Re: Claude Fable 5.1 and Claude Mythos 5.1

#221

Post by rot256 »

PedroBatista wrote: post #125
felixrieseberg wrote: post #69

(I work at Anthropic) Beyond all the benchmarks, I think Fable 5.1 is a big improvement in writing style. It sounds a lot less stereotypically like other Claude models, has (imho) a much more natural style, and responds to my style instructions more reliably. More work to be done (and we will!) but reading better prose makes me so much happier. Another point I expect not to get much attention until it all happens at…

This post and comment makes me believe "science" is the new "code" for Anthropic now that the code advantage is mostly gone and lost for OpenAI, ie. they got much better and Claude become significantly worse over these months.

I write a lot of Rust and Lean, Fable 5 is in my experience better at both. Cost/performance is a different story.

Re: Claude Fable 5.1 and Claude Mythos 5.1

#222

Post by enraged_camel »

jdgoesmarching wrote: post #186
skiing_crawling wrote: post #103

All the benchmarks in the world don't matter if the model just straight up refuses to do mundane things. Claude has too much of an attitude.

All the benchmarks in the world don’t matter if the subscription forces you into a walled garden of slopcoded apps. I’ll stick with Codex and, increasingly, open source SOTA models.

Great, thanks for sharing.

Re: Claude Fable 5.1 and Claude Mythos 5.1

#223

Post by seaurchinzee »

According to the FrontierCode Extended benchmarks in the system "card" (page 169-170), Fable 5.1 apparently does best on the medium effort level for this benchmark: "[...] at higher efforts, Fable 5.1 occasionally adds more small, unrequested changes [...]" Though Fable 5.1's medium is also lower than Fable 5's best score on the same benchmark, which uses xhigh.

Re: Claude Fable 5.1 and Claude Mythos 5.1

#224

Post by enraged_camel »

simonw wrote: post #144

Pelicans for thinking effort low, medium, high and xhigh (that xhigh one is pretty good): https://tools.simonwillison.net/markdown-svg-renderer#url=ht... I'm still waiting for effort max to finish. EDIT: I fixed a bug in my tooling so it now records summarized reasoning traces - here's that max pelican, which is a significant improvement: https://tools.simonwillison.net/markdown-svg-renderer#url=ht... Took just under…

In a way, this is the only benchmark I care about now. :)

Re: Claude Fable 5.1 and Claude Mythos 5.1

#225

Post by comex »

exabrial wrote: post #194
felixrieseberg wrote: post #69

(I work at Anthropic) Beyond all the benchmarks, I think Fable 5.1 is a big improvement in writing style. It sounds a lot less stereotypically like other Claude models, has (imho) a much more natural style, and responds to my style instructions more reliably. More work to be done (and we will!) but reading better prose makes me so much happier. Another point I expect not to get much attention until it all happens at…

Fable is useless. Me: "Find my security problems in my own code. This is code I own. I'm doing this under authorization of the CEO/CTO of our company." Fable: "yeah, no."

Fable 5.1 apparently changes this policy.

Re: Claude Fable 5.1 and Claude Mythos 5.1

#226

Post by eigenblake »

I am absolutely thrilled that they reset weekly limits. I have been experimenting with highly autonomous work (5+ hours continuous) and fable seems excellent at this, especially when using subagents. I ran out of Fable capacity and was bummed out that my experiment would take longer to complete. Now I'm super happy I get to continue it

Re: Claude Fable 5.1 and Claude Mythos 5.1

#227

Post by jtrn »

felixrieseberg wrote: post #69

(I work at Anthropic) Beyond all the benchmarks, I think Fable 5.1 is a big improvement in writing style. It sounds a lot less stereotypically like other Claude models, has (imho) a much more natural style, and responds to my style instructions more reliably. More work to be done (and we will!) but reading better prose makes me so much happier. Another point I expect not to get much attention until it all happens at…

My initial impression is one of massive disappointment. The main issue was that Fable was unpredictable and prone to false positives by the safeguards. In my brief testing, it still seems completely unable to understand its own guardrails and will readily reason itself into triggering them. It claims it won't do so beforehand, and insists that the topic in question is perfectly OK. Regardless of how good the car is, I'm not comfortable buying or driving it when I know it can randomly and unpredictably explodes. So yea might be good, but you never know when it refuses to help… still.

Re: Claude Fable 5.1 and Claude Mythos 5.1

#228

Post by Exoristos »

velcrovan wrote: post #121
felixrieseberg wrote: post #69

(I work at Anthropic) Beyond all the benchmarks, I think Fable 5.1 is a big improvement in writing style. It sounds a lot less stereotypically like other Claude models, has (imho) a much more natural style, and responds to my style instructions more reliably. More work to be done (and we will!) but reading better prose makes me so much happier. Another point I expect not to get much attention until it all happens at…

I have a pet theory that the Opus prose style/smell we all have grown weary of is due at least in part to the models writing more for themselves and each other than for humans. They're packing lots of signal into fewer words and they don't care if it sounds cringe because it works better as glue in long-running tasks. I'm also thinking of the 2017 novel "Void Star" where AIs who operate everything have long since lef…

> I'm also thinking of the 2017 novel "Void Star" where AIs who operate everything have long since left ceased bothering with human languages, and it takes a rare sort of direct matrix-gazing savant to be able to try and horse-whisper them into doing or revealing anything they didn't already plan to do.

This sounds irrelevant to LLMs as we know them, which are trained on human language--it's almost their machine code, in a way--while what you're citing, in stark contrast, sounds like machine code in the classic sense.

Re: Claude Fable 5.1 and Claude Mythos 5.1

#229

Post by crisnoble »

dmix wrote: post #214

I use Claude Design heavily, I wish these charts show "10% better at picking a color" or laying out an app. Maybe it's hard to build a good visual design test. Claude's good at layouts but not the colors or smaller design details.

It confuses "small details" with "tiny fonts".
Post reply on HN