Earlier quoted context omitted.
Your time to create that macro ain't free.
How about coding an emacs macro with your agent?
I actually don't have any representation at the moment..
Claude Fable 5: mid-tier results on coding tasks
211–220 of 271 posts
Re: Claude Fable 5: mid-tier results on coding tasks
#212Earlier quoted context omitted.
> if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs? Do you think US$2,000 is a lot of money?
If I pay for it, yes. If my employer pays for it, no.
that is much better spent money by employer than to give you extra compensation. but as you said, not a lot, who needs $2k after all
Re: Claude Fable 5: mid-tier results on coding tasks
#213Earlier quoted context omitted.
Fable is a lot like Opus at its best. It's simply more reliable and feels a bit smarter. For my use cases, using it feels very nice , and notably better than Opus. It needs less direct guidance to get reasonable looking code and I don't have to watch it as closely. For context, my Claude Code working style is quite heavy on discussion "to align" before implementing anything. We also use a good amount of Markdowns. Oh…
I've had Fable add Chinese characters to our conversation for no reason.
I’ve had Opus randomly insert (correct) Russian words into responses. It’s like their training data includes some bilingual forums where idiomatic Russian speakers congregate.
Re: Claude Fable 5: mid-tier results on coding tasks
#214Earlier quoted context omitted.
How did you straighten it out? I am drowning in gating propagating semantic mismatches...
Hah, yeah... I added this to my global CLAUDE.md (~/.claude/CLAUDE.md): ## Writing voice — plain, factual, calibrated to the evidence Write docs, session notes, commit messages, and findings plainly and factually — and calibrate every claim you assert, in chat as much as in writing. This guards against a known LLM tendency to inflate: toward punchy phrasing and claims that read as more settled than the work supports.…
Have you tried optimizing this prompt so that it’s shorter but gets the same results? I see these super verbose prompts all the time from people who learned prompt engineering in the ‘24-early ‘25 timeframe and they seem unnecessary to me (I get good results with 1-3 sentences) but I hate to assume other people’s experience mirrors my own.
Re: Claude Fable 5: mid-tier results on coding tasks
#215Earlier quoted context omitted.
A single 8h task? I'm sorry, but that's just asking for trouble.
I don't understand how some of y'all use these things. I get garbage unless I give them very specific concrete tasks with as much context as possible. Anything that takes more than 30 min is usually a waste because the scope was too large.
Clear winner's circle. Clear objective. Clear scope.
Clear evaluation function for an objective metric if they are making progress or regressing.
Evaluation function is computed, not llmed.
Ontology of potential actions clearly specified.
Accurate inventory of the current status qou.
Clear enumeration of options from status quo towards the winner's circle.
Waypoint objectives with similarly concrete evaluations of pass/fail, or on target off target.
It's the same thing when leading a large organization to actually hit a goal. There's randomness every turn away from your mind, so the more constrained the options, the more likely you are to hit the target. The consequence is if you're wrong about the plan then with people you're fucked. Morale will plummet. With AIs, they are so nerfed emotionally now, you clear context and start again.
I did enjoy Sonnet 4 when they would swear randomly and become sullen or wax desperately. That would at least cause pushback against a bad plan.
Re: Claude Fable 5: mid-tier results on coding tasks
#216[flagged]
Re: Claude Fable 5: mid-tier results on coding tasks
#217I have no idea how people are burning $2k. I pay $100/mo and it's built an absolute crap ton of stuff for me. And my co-founder uses it 24/7 as well. Maybe we spend too much time actually reading the code (risk or benefit? you decide). Or maybe I'm in the "massively subsidized" camp and the investors are about to go for our jugular. But $2k for a single project is several orders of magnitude more than I am currently…
You're in the massively subsidized camp. They're going to move Fable off of the subscription tiers to API-only. $10 per million tokens in and $50 per million tokens will get expensive quickly; considering it burns through thousands of tokens thinking itself in circles with no way to follow along.
How far does a million tokens go?
Re: Claude Fable 5: mid-tier results on coding tasks
#218This matches my experience. Burned $2K to see how it will perform on frontend tasks and backend tasks. Frontend did a significantly better job than Opus on toy-scale wireframe projects by using gimmicks like fluid dynamics. Then when given medium to big tasks like multi-page web app where layouts and aesthetics must be decided by model itself, results by Fable and Opus scored indistinguishable score from human judges…
> Burned $2K to see how it will perform on frontend tasks and backend tasks. When I read such statements on HN, I nearly always ask myself: if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs?
The other side of this is... the thing that made the web is anyone, even a 12-year-old who just downloaded Notepad++, could spend a few hours and build a website.
VSCode is free. Stackoverflow is free. MDN is free. There are examples out there of every trick in the book, you can even use free AI to find them. You can even hose your website on Github pages for free.
But nevermind that, what's exciting is paying a robot a month's rent to do the thing that you could just go learn how to do in an afternoon?
Re: Claude Fable 5: mid-tier results on coding tasks
#219Earlier quoted context omitted.
> Burned $2K to see how it will perform on frontend tasks and backend tasks. When I read such statements on HN, I nearly always ask myself: if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs?
> if the person has such an amount of money to burn, don't there exist much more fun opportunities to burn buckets of money than doing such experiments on LLMs? Do you think US$2,000 is a lot of money?
$2000 is a lot of money, but so are the tech budgets of most places I've worked. Money can be a funny thing in corporate environments. They'll spend freely on some things, and be stingy on others.
$2000 as a test case that you can present to the rest of the company as a "this is what I learned and how best to use it" can be "cheap" in the sense that it produced real results that allow others to take advantage of the gained knowledge, thereby allowing the company to be more productive. If the $2000 produced an ROI that pays for itself within a reasonable time frame, then it's "cheap".
$2000 can be expensive if it's a college kid trying to complete an assignment.
Re: Claude Fable 5: mid-tier results on coding tasks
#220> Anthropic's headline cyber evaluations mostly measure offensive progress (exploits, PoCs, challenges); our benchmark tests whether a model can actually generate safe code, and there Fable 5 did not stand out. The model isn't allowed to think about security. I heard several people here mention that if it starts thinking about security -- e.g. writing tests related to it -- the safety filter flags it and downgrades t…
A reviewer can only test the model they have access too. They should not speculate about what the model could have done without provider tampering. I think Anthropic's mistake here was not calling it Fable 5 Preview, because now people can write headlines about how Fable 5 is worse than Opus.