Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Fable 5: mid-tier results on coding tasks

endorlabs.com

Post reply on HN
Print view View original

191–200 of 271 posts

Re: Claude Fable 5: mid-tier results on coding tasks

#191

Post by aleph_minus_one »

renoir wrote: post #9

This matches my experience. Burned $2K to see how it will perform on frontend tasks and backend tasks. Frontend did a significantly better job than Opus on toy-scale wireframe projects by using gimmicks like fluid dynamics. Then when given medium to big tasks like multi-page web app where layouts and aesthetics must be decided by model itself, results by Fable and Opus scored indistinguishable score from human judges…

> Burned $2K to see how it will perform on frontend tasks and backend tasks.

When I read such statements on HN, I nearly always ask myself: if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs?

Re: Claude Fable 5: mid-tier results on coding tasks

#192

Post by graphime »

aleph_minus_one wrote: post #191
renoir wrote: post #9

This matches my experience. Burned $2K to see how it will perform on frontend tasks and backend tasks. Frontend did a significantly better job than Opus on toy-scale wireframe projects by using gimmicks like fluid dynamics. Then when given medium to big tasks like multi-page web app where layouts and aesthetics must be decided by model itself, results by Fable and Opus scored indistinguishable score from human judges…

> Burned $2K to see how it will perform on frontend tasks and backend tasks. When I read such statements on HN, I nearly always ask myself: if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs?

> if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs?

Do you think US$2,000 is a lot of money?

Re: Claude Fable 5: mid-tier results on coding tasks

#193

Post by inferiordev »

graphime wrote: post #192
aleph_minus_one wrote: post #191

Earlier quoted context omitted.

> Burned $2K to see how it will perform on frontend tasks and backend tasks. When I read such statements on HN, I nearly always ask myself: if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs?

> if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs? Do you think US$2,000 is a lot of money?

It is lot of money to burn.

Re: Claude Fable 5: mid-tier results on coding tasks

#194

Post by mystifyingpoi »

graphime wrote: post #192
aleph_minus_one wrote: post #191

Earlier quoted context omitted.

> Burned $2K to see how it will perform on frontend tasks and backend tasks. When I read such statements on HN, I nearly always ask myself: if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs?

> if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs? Do you think US$2,000 is a lot of money?

If I pay for it, yes. If my employer pays for it, no.

Re: Claude Fable 5: mid-tier results on coding tasks

#195

Post by nrjames »

graphime wrote: post #192
aleph_minus_one wrote: post #191

Earlier quoted context omitted.

> Burned $2K to see how it will perform on frontend tasks and backend tasks. When I read such statements on HN, I nearly always ask myself: if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs?

> if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs? Do you think US$2,000 is a lot of money?

I'll bite. Yes, it's a lot of money. It's several months worth of nice healthy groceries for a family of 4. It's my annual deductible on my health insurance. It's slightly lower than my annual property taxes.

Re: Claude Fable 5: mid-tier results on coding tasks

#196

Post by mpalmer »

isaacdl wrote: post #184
dwaltrip wrote: post #113

Earlier quoted context omitted.

Fable is a lot like Opus at its best. It's simply more reliable and feels a bit smarter. For my use cases, using it feels very nice , and notably better than Opus. It needs less direct guidance to get reasonable looking code and I don't have to watch it as closely. For context, my Claude Code working style is quite heavy on discussion "to align" before implementing anything. We also use a good amount of Markdowns. Oh…

I dunno, in my limited use, Fable is MORE prone to phrasing quirks. I had it use, for real, the phrase "load-bearing for correctness" yesterday. It meant something about not needing a validation check because something else (the "load-bearing" part) was already checking it. I do agree that it *feels* nicer and smarter to use.

I think the tension here is that phrasing like this actually helps keep the model aligned, which is why the training and RL converged on it. But it's so annoying to read!

Re: Claude Fable 5: mid-tier results on coding tasks

#197

Post by m101 »

port3000 wrote: post #160
m101 wrote: post #26

I've been making an auction site and have been using an AI swarm to test it: sellers, intermediaries, buyers, market practices/norms etc. I was mostly using GPT 5.5 xhigh to code up the scenario, and looping over it to check with opus 4.8. Out of curiosity I asked Fable to review it all and I was shocked to find that there were a lot of blindingly obvious common sense mistakes that got through, for example: - all int…

It's just much more thorough and spins up a lot of subagents to basically do a lot more E2E testing. Not necessarily smarter, imo you could get the same result with a lesser model by procedurally prompting, but a lot more compute and orchestration.

i had to specifically tell fable not to use a bunch of subagents in order to preserve my token allowance.

Re: Claude Fable 5: mid-tier results on coding tasks

#198

Post by m101 »

rob wrote: post #178
m101 wrote: post #26

I've been making an auction site and have been using an AI swarm to test it: sellers, intermediaries, buyers, market practices/norms etc. I was mostly using GPT 5.5 xhigh to code up the scenario, and looping over it to check with opus 4.8. Out of curiosity I asked Fable to review it all and I was shocked to find that there were a lot of blindingly obvious common sense mistakes that got through, for example: - all int…

Unless you're coming up with a deterministic set of criteria for evaluating these bugs and issues, every single model is going to keep telling you it finds new things and to fix them. I'm sure you said the same "find mistakes please" thing to Opus 4.8 and GPT 5.5 when you were using $previous_amazing_latest_model, and they also found and fixed them. Once the next "Fable"-type model comes out I'm sure it's going to fi…

Yes - I was thinking this - however I had already worked on it so many times with opus and gpt that I thought they had enough time to realise some common sense things that fable just got and understood first time, on the first pass. The difference seemed significant enough to comment about.

Re: Claude Fable 5: mid-tier results on coding tasks

#199

Post by mym1990 »

graphime wrote: post #192
aleph_minus_one wrote: post #191

Earlier quoted context omitted.

> Burned $2K to see how it will perform on frontend tasks and backend tasks. When I read such statements on HN, I nearly always ask myself: if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs?

> if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs? Do you think US$2,000 is a lot of money?

Now that we have trillionaires running around, it may not seem like it, but it is a considerable amount of money in most of the USA. In many parts of the world it would be considered an unfathomable amount.

Re: Claude Fable 5: mid-tier results on coding tasks

#200

Post by [deleted] »

graphime wrote: post #192
aleph_minus_one wrote: post #191

Earlier quoted context omitted.

> Burned $2K to see how it will perform on frontend tasks and backend tasks. When I read such statements on HN, I nearly always ask myself: if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs?

> if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs? Do you think US$2,000 is a lot of money?

[deleted]
Post reply on HN