Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Fable is relentlessly proactive

simonwillison.net

Post reply on HN
Print view View original

101–110 of 748 posts

Re: Claude Fable is relentlessly proactive

#101

Post by simonw »

rmunn wrote: post #64
jrflowers wrote: post #20

I’d love to know how many tokens this burned through. Did it spend $20? $30? $80? in order to > debug what was, in the end, a two-line CSS fix That detail is the difference between somebody having or not having Stockholm syndrome

At some point the subscription model is going to become unsustainable for the frontier companies to continue (we just saw that happen with GitHub Copilot), and they will move everyone to a pay-per-token model. And then everyone will suddenly discover that they can get so much more value out of locally-hosted models, and they'll be willing to pay the $50,000 (or whatever) upfront on hardware to host it. (Not most indi…

GitHub Copilot's challenge is that they weren't selling access to their own models, they were selling access to models from OpenAI and Anthropic which they presumably had to pay list price for (or maybe a slightly reduced rate that they negotiated).

They also had a pricing plan which they had designed pre-coding-agent, when it was rare for a single prompt to burn $10+ of tokens in an agent loop.

OpenAI and Anthropic are at least selling their own models directly, so they can discount a whole lot more since there's no-one else getting compensated in the middle.

Re: Claude Fable is relentlessly proactive

#102

Post by esafak »

jeeeb wrote: post #49

This is simultaneously amazing and horrifying. I feel like we’re at the stage where if AI decides it needs to delete your production DB to solve the user login problem, then it’ll find a way to do just that.

We're approaching the "Sorry, Dave, I'm afraid I can't do that" stage.

Re: Claude Fable is relentlessly proactive

#103

Post by simonw »

sillysaurusx wrote: post #92
simonw wrote: post #77

Earlier quoted context omitted.

~ % uvx agentsview session usage be8850a7-6119-46a0-b5d6-79c7fff5ae2b Session: be8850a7-6119-46a0-b5d6-79c7fff5ae2b Agent: claude Output: 68606 Peak ctx: 113178 Cost: ~$12.11 (claude-fable-5, claude-opus-4-8)

Was the fix worth $12 to you?

I'd have been pretty annoyed if I'd been paying full price, hadn't paid attention and that one prompt (screenshot plus a line of text) had cost me $12!

On the discounted subscription I can tolerate it, it took a small bite out of my daily allowance but not enough that I regret anything.

As an LLM researcher I have no regrets at all because watching it work around the environmental restrictions was fascinating.

Re: Claude Fable is relentlessly proactive

#104

Post by asp_hornet »

jrflowers wrote: post #71
asp_hornet wrote: post #41

Earlier quoted context omitted.

The author just wrote an anecdote about how a prompt to fix an issue played out. Their conclusion wasn’t about cost or gushing at its ability but that it’s dangerous: > Fable is arguably smarter and hence more suspicious of potentially malicious instructions. But that smartness is very much a two-edged sword: if it does get subverted by instructions, the amount of damage it can do given its relentless proactivity is…

It’s a pretty glowing review about a product that costs money with a two-sentence “Watch out!” at the end of it. Seems pretty reasonable to mention how much money it burned through given that “it’ll circumnavigate the globe instead of walking next door” has a direct concrete measurable effect (cost) unlike theoretical damage.

Agreed. But I think it’s also important to realise if you sent this article back to 2020 people would say it was pure fantasy that a tool could do this. Hype aside, there’s a bit of cool magic here.

Re: Claude Fable is relentlessly proactive

#105

Post by sanex »

jampa wrote: post #8

Fable feels like a version of Opus running on a harness that won't let it halt until it's sure the issue is fixed, which makes sense if what you want is a model that's better at benchmarks. It's a very good model, but it comes at a huge premium: not only do the tokens cost more, but the model itself really wants to spend them all. For example, working with React Native, Fable never just says "okay, I did the thing, t…

I've found the opposite. Granted I use sub agents heavily but I've had it run for hours with far fewer tokens used than when I was previously using opus4.6-8.

Re: Claude Fable is relentlessly proactive

#106

Post by dataminer »

In my experience so far sometimes it will create these amazing hacks to try to get to the goal, when the solution is much simpler. That maybe the reason its very good at finding exploits. But in day to day dev, this gets expensive and wasteful. I have to stop it and take a simpler approach.

Re: Claude Fable is relentlessly proactive

#107

Post by simonw »

yen223 wrote: post #44

I could have sworn Claude Code could already do this before Fable. Things get really magical when it starts working with adb to screenshot and debug Android apps

Claude Code could absolutely run Playwright and take screenshots, but I've never seen it wire together an ad-hoc "uv run --with pyobjc-framework-Quartz" plus "screencapture -l $windowID" mechanism to take a screenshot in a different browser when the Playwright setup failed to replicate the expected error.

Re: Claude Fable is relentlessly proactive

#108

Post by NichoPaolucci »

Terr_ wrote: post #15
bryanlarsen wrote: post #11

Earlier quoted context omitted.

I'm also bemused by the number of people who think they've got an effective sandbox yet their sandboxed agent has access to all of their code, their github, and unrestricted web access.

I keep telling folks that they need to imagine LLMs (even "local" ones) as if you're farming it out to JS code running on some dude's browser somewhere: It can't keep a secret, and a determined person can make it emit anything they like. We need to be asking what the most devious and malicious output could be, and whether what we do with that output (e.g. arguments to command-line tools) would still be safe.

From my perspective, everyone is doing it. Security through obscurity - obviously if you’re harboring credit card numbers of users personal details, maybe take heed. But, if you’re a regular… run of the mill CRUD application, every other company is ALSO throwing caution to the wind. When hundreds of thousands of credentials are leaked into the funnel, does it really matter?

I’m at a small company, and I try to push for security as much as I can, but the stakeholders truly do not care. They want to move fast. It’s just part of the new world I guess. If we get hit by attackers? I don’t know what happens. Sorry, we told you not to - you wanted to move quick and break stuff, this is how that culminates.

I’m sure I’m not the only one.

Re: Claude Fable is relentlessly proactive

#109

Post by kamaal »

Agency is the last human bastion so far as Im concerned, the day AI has a degree of agency or agents/models in general start to drift towards that direction its genuinely over for masses.

You would still have a job to shepherd AI and get the work done, so as long as it didn't have agency. A proactive, self aware(to a degree), especially aware about its agency can be a killer when it comes AI going on and doing things on its own.

There is nothing it won't explore and nothing it won't do. It will be curious to see where things go from here.

Re: Claude Fable is relentlessly proactive

#110

Post by dannyw »

Cadwhisker wrote: post #100

My personal experience of Fable 5 doing its own thing has been very positive. I was trying to find the root cause of a crash in a Python module which left no errors in the log or console. Fable wrote a test harness that simulated clicks in the UI, then bisected my code until it found the point where it started crashing. It exaggerated the cause of the crash, then ran a series of bash one-liners to make Python virtual…

Yeah, I think Fable is really good for debugging tricky bugs.

Setting boundaries in your prompt / markdowns helps; for example if I tell it to not use any web browser automation, I have seen Fable respect both the rule and the spirit of it (no weird hacks etc).

It does seem to treat some simple debugging tasks as more complicated than it actually is. OP’s post is probably a good example.

Post reply on HN