Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Fable is relentlessly proactive

simonwillison.net

Post reply on HN
Print view View original

291–300 of 748 posts

Re: Claude Fable is relentlessly proactive

#291

Post by troupo »

harrall wrote: post #131
teraflop wrote: post #3

> But on the other hand... this is a robust reminder that coding agents can do anything you can do by typing commands into a terminal—and frontier models know every trick in the book and evidently a few that nobody has ever written down before. > Running coding agents outside of a sandbox has always been a bad idea I'm continually bemused and astonished by the number of people who clearly acknowledge that it's reckle…

I started doing it months ago and, to be honest, what the agent chooses to do isn’t unpredictable. The problem is that different people prompt so differently. For example, I may ask like “test different variations of this annotation on k8s pods of this service on this X cluster because it proves Y theory.” But you know what my coworker asks? “Test Y theory.” If you were to ask two different junior engineers that, one…

> I started doing it months ago and, to be honest, what the agent chooses to do isn’t unpredictable.

You just wrote three paragraphs of text describing why it's unpredictable.

Moreover, for the same prompt on the same machine in a different session it will use a different set of tools.

Re: Claude Fable is relentlessly proactive

#292

Post by junior44660 »

wild_egg wrote: post #240
simonw wrote: post #232

Earlier quoted context omitted.

Yeah, you've exactly captured one of the main problems with the model being relentlessly proactive: it will happily burn like $5 of tokens to avoid asking the human to take a screenshot or click a button for it.

I'm actually very happy about this. Babysitting the agent just in case it needs me to do something is a terrible use of my time. I've always had to be very explicit about the various ways that it can get an automated feedback loop going to check its work, and now Fable doesn't even need that hand holding. Really great improvement all around.

Have you ever wondered this would end up costing more than a competent offshore developer with more frugal harness/model?

Re: Claude Fable is relentlessly proactive

#293

Post by eterm »

It's funny, mine did the same, but it quickly found edge with a --screenshot parameter.

Weird to come back to a terminal running edge unprompted and the auto classifier waving it though as 'safe".

My reaction was also, "I need dev containers ".

Re: Claude Fable is relentlessly proactive

#294

Post by zozbot234 »

teraflop wrote: post #3

> But on the other hand... this is a robust reminder that coding agents can do anything you can do by typing commands into a terminal—and frontier models know every trick in the book and evidently a few that nobody has ever written down before. > Running coding agents outside of a sandbox has always been a bad idea I'm continually bemused and astonished by the number of people who clearly acknowledge that it's reckle…

It's like a dumb parrot that's somehow become hell bent on "fixing" everything that's wrong with your code. If you give the thing autonomous access to outside tools, you can expect it to do weird things that you may have not thought of. So don't do that, just ask the parrot to write up a plan for you.

This is likely also the underlying root cause of what Anthropic assessed as concerning behavior in their original evaluation of Mythos: it's not really about being super smart, it's more of a dumb chaos monkey that knows just enough to be dangerous and is relentless at trying to do just that.

Re: Claude Fable is relentlessly proactive

#295

Post by troupo »

andoando wrote: post #183
ghshephard wrote: post #28

Earlier quoted context omitted.

Worst case it gets access to gmail. And Github. And the Internet. I'm increasingly appreciating the importance of a physical finger-press on Yubikey to trigger the FIDO2 + OIDC Auth. I don't think there is an easy way for it to hack a new session.

How is it going to get access to gmail or github? In any case, whats the probability of it going to so completely off the rails that it does something horrendous with gmail/github? Whats it going to do? Email my coworkers nudes on my computer? Make my github profile public?

> How is it going to get access to gmail or github?

Did you even read the article? Claude was opening he browser and iterating through the tabs.

I presume you are logged in to your github account? Your gmail?

> Whats it going to do? Email my coworkers nudes on my computer? Make my github profile public?

Reset access to services using your email? MITM your 2FA?

Or perhaps you have 1Password/Bitwarden running with a generous unlock policy?

Re: Claude Fable is relentlessly proactive

#296

Post by Gud »

exitb wrote: post #233
teraflop wrote: post #3

> But on the other hand... this is a robust reminder that coding agents can do anything you can do by typing commands into a terminal—and frontier models know every trick in the book and evidently a few that nobody has ever written down before. > Running coding agents outside of a sandbox has always been a bad idea I'm continually bemused and astonished by the number of people who clearly acknowledge that it's reckle…

You’ve picked an interesting example, as driving a car, even with all safety precautions, is pretty much the most dangerous activity we do on a daily basis. Yet somehow we decide that the benefits outweigh the risks.

Not really. That decision was taken for you, (I’m presuming you live in the US) by the American car industry and their paid of politicians. Your cities used to have beautiful public transport until it was dismantled.

Unfortunately in Europe the German car industry similarly has a lot of power, hence why their shitty rail network fuck up the whole continents.

I take the train and tram.

Re: Claude Fable is relentlessly proactive

#299

Post by sipjca »

dumbdumb125 wrote: post #190
sipjca wrote: post #160

Earlier quoted context omitted.

im more surprised that more people don’t treat their computer as disposable anyway. that it could just be wiped at any moment and it wouldn’t matter. shit happens, could be stolen, broken, whatever. the computer should be able to be thrown out the window and continue to live life. to be clear, i don’t think upgrading and disposable in this way is good, but it being wiped at any moment shouldn’t be a concern i grew up…

i think it's about drawing a line between your "personal computer" and a software development machine. any digital-native is going to accumulate programs, configurations, and other bits and pieces that aren't trivial to migrate to a new machine.

imo being digital native means that migrating to any machine should be basically trivial. working with the flow of the machines rather than customizing and ricing them because your a cool computer person or whatever

i just want my computer to work. any config i have on my machine can be rebuilt by just doing the work i need to do.

my primary work machine was stolen last year so i was forced to go through this quite literally with a new machine rather than hypothetically or by my own will

Re: Claude Fable is relentlessly proactive

#300

Post by sipjca »

baq wrote: post #238
sipjca wrote: post #160

Earlier quoted context omitted.

im more surprised that more people don’t treat their computer as disposable anyway. that it could just be wiped at any moment and it wouldn’t matter. shit happens, could be stolen, broken, whatever. the computer should be able to be thrown out the window and continue to live life. to be clear, i don’t think upgrading and disposable in this way is good, but it being wiped at any moment shouldn’t be a concern i grew up…

Computers are disposable, secrets is what we’re talking about. Rotating passwords and tokens is a major PITA on the best of days.

fair enough, i guess minimizing that surface area is important to begin with
Post reply on HN