This matches my experience. Burned $2K to see how it will perform on frontend tasks and backend tasks. Frontend did a significantly better job than Opus on toy-scale wireframe projects by using gimmicks like fluid dynamics. Then when given medium to big tasks like multi-page web app where layouts and aesthetics must be decided by model itself, results by Fable and Opus scored indistinguishable score from human judges…
> Burned $2K to see how it will perform on frontend tasks and backend tasks. When I read such statements on HN, I nearly always ask myself: if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs?
Claude Fable 5: mid-tier results on coding tasks
221–230 of 271 posts
Re: Claude Fable 5: mid-tier results on coding tasks
#222This matches my experience. Burned $2K to see how it will perform on frontend tasks and backend tasks. Frontend did a significantly better job than Opus on toy-scale wireframe projects by using gimmicks like fluid dynamics. Then when given medium to big tasks like multi-page web app where layouts and aesthetics must be decided by model itself, results by Fable and Opus scored indistinguishable score from human judges…
I genuinely think that Fable is just Opus 4.8 with some extra skills and harness. I saw a video of someone generating UI with them both side by side, and it gives identical recommendations for themes etc. Doesn't feel like a new model to me, just Opus 4.8 with some sprinkles on top.
Re: Claude Fable 5: mid-tier results on coding tasks
#223Earlier quoted context omitted.
I don't understand how some of y'all use these things. I get garbage unless I give them very specific concrete tasks with as much context as possible. Anything that takes more than 30 min is usually a waste because the scope was too large.
I had good experiences doing multi-hour refactoring/housekeeping tasks that basically consisted of applying the same steps and rules n times. Worth noting, a significant chunk of those runs involved the agent waiting for the compiler, linters, type checks, and test suites, as well as updating journals. It’s not the agent sputtering out code for eight hours straight. And naturally I spend more time on manual verificat…
I do this too, with a document written for this purpose.
> ... a significant chunk of those runs involved the agent waiting for the compiler, linters, type checks, and test suites, as well as updating journals.
That is a good point. I'm mostly using C, which seemingly compiles in O(1) time, so I could imagine a large C++ or Rust codebase taking much longer to iterate simply due to compilation times.
Re: Claude Fable 5: mid-tier results on coding tasks
#224Earlier quoted context omitted.
> Burned $2K to see how it will perform on frontend tasks and backend tasks. When I read such statements on HN, I nearly always ask myself: if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs?
Imagine if all that money was donated to open source instead.
Re: Claude Fable 5: mid-tier results on coding tasks
#225Earlier quoted context omitted.
> Burned $2K to see how it will perform on frontend tasks and backend tasks. When I read such statements on HN, I nearly always ask myself: if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs?
> if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs? Do you think US$2,000 is a lot of money?
If I had a need to spend $2k, I could do so easily, but I still think it’s a lot of money to burn. I wouldn’t spend it on a whim; I would not spend it without carefully, considering the value of what I get.
I would not even spend that much money in the businesses that I own, or recommended that my well capitalized employer spend that much money without being reasonably confident that the business would get good value for its money.
Re: Claude Fable 5: mid-tier results on coding tasks
#226Similar result on our kotlin coding benchmark at work. It measures how close agents can get to a small mergable PR (according to my team). 20 tasks of varying difficulty, with 5 attempts each, LLM as judge to evaluate accuracy (same outcome and quality but allowing for acceptable variances). Fable 5 sits ahead of Opus 4.7, but behind Opus 4.6, Sonnet 4.6, Opus 4.8, GPT-5.4, GPT-5.5. Fable isn't a good coding workhors…
Re: Claude Fable 5: mid-tier results on coding tasks
#227Earlier quoted context omitted.
I’ve found that agents are obsessed with adding more lines of code. Even when asking them to simplify they’ll remove 50 lines of code and then add 100 more. You have to explicitly tell them you want less lines of code. So I just do that after iterating on a task for a few steps.
I think the problem is that agents are inherently stochastic. Their idea of simplification changes from message to message because whatever objective it’s operating on internally is inherently opaque and changes. No matter how much you prompt it, eventually it’s going to not do what you want it to do. I built https://github.com/thempatel/mdlr for precisely this reason: externalize the objective and force the agent to…
Re: Claude Fable 5: mid-tier results on coding tasks
#228Earlier quoted context omitted.
I dunno, in my limited use, Fable is MORE prone to phrasing quirks. I had it use, for real, the phrase "load-bearing for correctness" yesterday. It meant something about not needing a validation check because something else (the "load-bearing" part) was already checking it. I do agree that it *feels* nicer and smarter to use.
I think the tension here is that phrasing like this actually helps keep the model aligned, which is why the training and RL converged on it. But it's so annoying to read!
Re: Claude Fable 5: mid-tier results on coding tasks
#229WTF! I run into fallback to Opus 4.8 all the time, and I am not even doing "security Research", just normal development and debugging.
My experiences with Fable thus far have been far from 'mid-tier'. While some model releases are incremental, Fable is the same qualitative change that Opus 4.6 was compared to its predecessors. It fundamentally impacts how I work with the model. (Note: I only (well, 99%) do back-end in Python)
Re: Claude Fable 5: mid-tier results on coding tasks
#230> The dominant mechanism, and the one no prompt instruction can prevent: the model has simply seen the upstream fix during training and reproduces it… > On numpy, the patch is 100% character-for-character identical to the golden patch… down to idiosyncratic comments like "Extending singleton dimension for 'reflect' is legacy behavior; it really should raise an error." This… seems like a flaw in the benchmark suite me…