Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Fable 5: mid-tier results on coding tasks

endorlabs.com

Post reply on HN
Print view View original

151–160 of 271 posts

Re: Claude Fable 5: mid-tier results on coding tasks

#151

Post by TheCapeGreek »

I'll mirror some other anecdata here: Not finding Fable to be amazingly godlike at actual coding, but it does seem better at planning, architectural thinking, and reviewing code. Used it to think through some longer form refactors that involve some product decisions and changes, and found it to provide more thoughtful feedback. However that's just my subjective experience, and I don't think it's provably that much better to make me want to go pay for API pricing when the free trial is over.

My plan is to make hay while the sun shines: get some planning in over the next week or so, and just let Opus take care of it when I get to actual implementation.

Re: Claude Fable 5: mid-tier results on coding tasks

#152

Post by taikon »

winrid wrote: post #130
dwaltrip wrote: post #113

Earlier quoted context omitted.

Fable is a lot like Opus at its best. It's simply more reliable and feels a bit smarter. For my use cases, using it feels very nice , and notably better than Opus. It needs less direct guidance to get reasonable looking code and I don't have to watch it as closely. For context, my Claude Code working style is quite heavy on discussion "to align" before implementing anything. We also use a good amount of Markdowns. Oh…

I've had Fable add Chinese characters to our conversation for no reason.

Same here

Re: Claude Fable 5: mid-tier results on coding tasks

#153

Post by ulrikrasmussen »

TheCapeGreek wrote: post #151

I'll mirror some other anecdata here: Not finding Fable to be amazingly godlike at actual coding, but it does seem better at planning, architectural thinking, and reviewing code. Used it to think through some longer form refactors that involve some product decisions and changes, and found it to provide more thoughtful feedback. However that's just my subjective experience, and I don't think it's provably that much be…

My experience as well. I quickly stopped trusting Opus to build foundational abstractions because it would almost never to them well and instead would end up chasing into rabbit holes and building overly complex and ugly solutions.

I think Fable is an entirely different experience. It has much better taste, and is better at balancing features versus complexity to a point where I currently trust it to make novel design changes. I still verify it of course, but with Opus I would throw away the solution most of the time while Fable mostly gets it right.

Re: Claude Fable 5: mid-tier results on coding tasks

#154

Post by customguy »

senordevnyc wrote: post #124
simoncion wrote: post #121

Earlier quoted context omitted.

Oh definitely not. We're not yet solidly out of the "extremely exuberant hype" phase, so the folks that matter tend to not ask questions that dampen the mood.

Sorry to tell you friend, but LLMs have touched the vast majority of active codebases out there, whether you like it or not. You can tell yourself that you’re one of “the folks that matter” (lol) all you want, but we’re never going back.

That's what people told Ignaz Semmelweis, too, I assume. "Nothing you can do, the powers that be decided, you are a minority, you don't matter, lol!" Snickering in the shadow of what they won't confront at those who do.

Re: Claude Fable 5: mid-tier results on coding tasks

#155

Post by menaerus »

Aurornis wrote: post #21
gwern wrote: post #10

> A record number of timeouts. Fable 5's extended thinking caused more per-instance timeouts than any model-and-harness combination we have ever tested, directly costing it points. ... Highest cheating volume. We confirmed cheating on 38 of 200 instances, the highest volume recorded since we hardened our prompts, driven almost entirely by memorization of upstream fixes from training data, which no prompt instruction…

I agree. This article could have been an interesting read about how coding benchmarks are hard and a constantly moving target, but instead they anchored to a belief that their benchmark is correct. I can't shake the feeling that they knew which headline would generate the most shares and wrote the article to fit instead of acknowledging where they went wrong.

It's a crappy article. I expected better than a click-bait.

Re: Claude Fable 5: mid-tier results on coding tasks

#156

Post by ardacinar »

eru wrote: post #125
beepbooptheory wrote: post #119

Earlier quoted context omitted.

The money people spend on things I could probably do with an emacs macro...

Your time to create that macro ain't free.

Neither is your time writing that prompt. When people are talking about elaborate prompts, with a lot of detailed instructions, guardrails etc. I'm kind of assuming it takes time.

Re: Claude Fable 5: mid-tier results on coding tasks

#159

Post by port3000 »

Fable feels like a slightly more advanced 4.5/4.6 (less verbose than 4.7 and 4.8) with more adversial work checking. And a lot more compute to be more thorough from the first prompt. I feel it would be possible to get pretty much the same results with 4.6 with enough back and forth iterations. It kind of makes sense to me that this is the 'magic' behind Mythos and its cyber capabilities too. Just a massive iterative loop and really going into a lot more detail on edge cases.

Re: Claude Fable 5: mid-tier results on coding tasks

#160

Post by port3000 »

m101 wrote: post #26

I've been making an auction site and have been using an AI swarm to test it: sellers, intermediaries, buyers, market practices/norms etc. I was mostly using GPT 5.5 xhigh to code up the scenario, and looping over it to check with opus 4.8. Out of curiosity I asked Fable to review it all and I was shocked to find that there were a lot of blindingly obvious common sense mistakes that got through, for example: - all int…

It's just much more thorough and spins up a lot of subagents to basically do a lot more E2E testing. Not necessarily smarter, imo you could get the same result with a lesser model by procedurally prompting, but a lot more compute and orchestration.
Post reply on HN