Live data from Hacker News

Claude Fable 5: mid-tier results on coding tasks

endorlabs.com

141–150 of 271 posts

Re: Claude Fable 5: mid-tier results on coding tasks

#141
post #44

Earlier quoted context omitted.

I used it yesterday afternoon-night and this morning-afternoon, UK time, over a period of a few 5-hour windows. I didn't count the prompts, wall time was 1d6h, API time was 2h10m.

Strange though... I spent my window after a couple of prompts and effective API time of 13m. Out for 4 hours and a half (why that?). The next day, today, I've tried to repeat the experience - even worse: one prompt for less than 10mins... and then suspended for 8 hours and a half. WTF?

How did you get suspended for 8 hours, given a 5-hour window? Maybe you are prompting it wrong [1].

[1] https://www.wired.com/2010/06/iphone-4-holding-it-wrong/

Re: Claude Fable 5: mid-tier results on coding tasks

#142
post #9

This matches my experience. Burned $2K to see how it will perform on frontend tasks and backend tasks. Frontend did a significantly better job than Opus on toy-scale wireframe projects by using gimmicks like fluid dynamics. Then when given medium to big tasks like multi-page web app where layouts and aesthetics must be decided by model itself, results by Fable and Opus scored indistinguishable score from human judges…

Fable is a lot like Opus at its best. It's simply more reliable and feels a bit smarter. For my use cases, using it feels very nice , and notably better than Opus. It needs less direct guidance to get reasonable looking code and I don't have to watch it as closely. For context, my Claude Code working style is quite heavy on discussion "to align" before implementing anything. We also use a good amount of Markdowns. Oh…

How did you straighten it out?

I am drowning in gating propagating semantic mismatches...

Re: Claude Fable 5: mid-tier results on coding tasks

#143
post #67

Earlier quoted context omitted.

Session paused Fable 5 has safety measures that flag messages on most cybersecurity or biology topics. They may flag safe, normal content as well. These measures let us bring you Mythos-level capability in other areas sooner, and we're working to refine them. Send feedback with /feedback or learn more 1. Switch to Opus 4.8 2. Edit prompt and retry with Fable 5

Biology? Why?

they're worried about people creating bioweapons

Re: Claude Fable 5: mid-tier results on coding tasks

#145

Earlier quoted context omitted.

Fable is a lot like Opus at its best. It's simply more reliable and feels a bit smarter. For my use cases, using it feels very nice , and notably better than Opus. It needs less direct guidance to get reasonable looking code and I don't have to watch it as closely. For context, my Claude Code working style is quite heavy on discussion "to align" before implementing anything. We also use a good amount of Markdowns. Oh…

How did you straighten it out? I am drowning in gating propagating semantic mismatches...

Hah, yeah... I added this to my global CLAUDE.md (~/.claude/CLAUDE.md):

## Writing voice — plain, factual, calibrated to the evidence

Write docs, session notes, commit messages, and findings plainly and factually — and calibrate every claim you assert, in chat as much as in writing. This guards against a known LLM tendency to inflate: toward punchy phrasing and claims that read as more settled than the work supports. Same spirit as the Read-Clean Check above, and composes with it — that rule governs journey-framing, this one governs tone and certainty.

*Plain over punchy.* Skip decorative metaphors and dramatic verbs when a plain word is clearer — call a fix "the change", not "the hammer"; logging "flags" a problem rather than being "radar"; numbers "grow", they don't "explode". Plain phrasing reads as engineering; flourish reads as marketing.

*Calibrated confidence.* Everything stated should be well-reasoned and defensible, with the strength of the wording matched to the strength of the evidence. Prefer "found" / "appears" / "points to" over "proved" / "clearly" / "obviously". Name the confounds and what's still unverified. Don't let a bold lead-in pre-announce a conclusion the work hasn't reached.

*Hypotheses stay labeled as hypotheses.* Speculation and educated guesses are useful — when brainstorming or investigating, surface them, and sharing a strong view is welcome. But conviction is not evidence: until there is clear evidence, a claim is a hypothesis and is stated as one — explicitly, even when it's highly compelling. The failure mode is asserting a hunch as settled fact, where it then propagates unchallenged into later docs and summaries. Back a claim with its evidence in the same breath, or mark it as not-yet-backed.

*Factual and forward-looking.* Separate what was measured from what was inferred, and stay pragmatic about what's true, what's still open, and what's next. On next steps specifically, resist the strong LLM pull to converge prematurely:

- A plausible next step is not a decided one. Don't present one or two plausible tasks as the one path we should now follow — that lock-on is a frequent failure mode. - Lay out the real options and their trade-offs. Saying which you'd lean toward and why is welcome and useful — but keep the space open and leave the choice to the user. - Premature certainty about what to do next is as much a miscalibration as premature certainty about what's true.

Re: Claude Fable 5: mid-tier results on coding tasks

#146
post #9

This matches my experience. Burned $2K to see how it will perform on frontend tasks and backend tasks. Frontend did a significantly better job than Opus on toy-scale wireframe projects by using gimmicks like fluid dynamics. Then when given medium to big tasks like multi-page web app where layouts and aesthetics must be decided by model itself, results by Fable and Opus scored indistinguishable score from human judges…

Fable is a lot like Opus at its best. It's simply more reliable and feels a bit smarter. For my use cases, using it feels very nice , and notably better than Opus. It needs less direct guidance to get reasonable looking code and I don't have to watch it as closely. For context, my Claude Code working style is quite heavy on discussion "to align" before implementing anything. We also use a good amount of Markdowns. Oh…

Yeah same here, it's a huge step up for me. Curious why people are having such different experiences. Is it just to do with what they're working on? Specific prompt styles (eg overfitting on opus)?

Re: Claude Fable 5: mid-tier results on coding tasks

#147
post #7

Earlier quoted context omitted.

The other "cheating" examples are even worse. It's wild to me that people keep designing benchmarks where the answer is lying around on disk or in the git history. "Hardening" the benchmark with strongly worded prompt instructions is bizarre. There are so many agent sandbox solutions. Why not use one and give it only access to the code it should see? And I'm not sure how they can rule out other solutions also benefit…

100%… the fact that they're just using prompting to discourage the agent from looking ahead in the Git history is wild.

Every day I am more and more convinced that AI labs can't code.

Re: Claude Fable 5: mid-tier results on coding tasks

#148
I set Fable onto a couple of intermittent bugs in my React Native app that Opus had failed to solve. It came up with novel approaches for both that squashed the bugs further up the pipeline, killing baby Hitler before he could become problem. Then Fable came up with 3 more edge case bugs, and 4 code cleanups.

This matches my experience with other model quality leaps, it's greater understanding gives it more bug blasting firepower.

Perhaps setting a new model off on a 2-4 hour tasks and expecting perfect results just isn't a great test. Chunking the problem is always a better test of abilities.

Re: Claude Fable 5: mid-tier results on coding tasks

#150
post #29

Earlier quoted context omitted.

I had almost the opposite experience. I'm building a compiler for a language without a tracing GC, so a big chunk of the work is around memory management: functional in-place update, reuse analysis, and a Perceus-style reference-counting strategy similar to what Koka uses. The hard part was that my use case wasn't exactly covered by the Koka/Perceus paper. The prior art got me maybe 75% of the way there, but the rema…

'by the way, your previous attempts have these structural problems." Just to be clear, it did not have access to any previous work that opus did? Because they are pretty good at digging out relevant tmp files and making use of whatever is out there. With my fable adventures I caught it hallucinating something and stating it as a fact in CLI twice. And it was something that I did not see opus do in such way, opus obvi…

If you have some spare time, I'd be interested in knowing what kind of questions you use to test models on understanding of Chinese culture.
Post reply on HN