Live data from Hacker News

Claude Fable is relentlessly proactive

simonwillison.net

151–160 of 748 posts

Re: Claude Fable is relentlessly proactive

#151

Earlier quoted context omitted.

Example of whats been shipped?

Okay. I rebuilt my website in ~a month with the help of Opus 4.7/.8 and it would have taken me, unaided human, at least 6 months. Link's in my bio if you care. Satisfied now? Will you stop asking this question? Thought not.

[deleted]

Re: Claude Fable is relentlessly proactive

#152

This sounds somewhat similar to the anecdote mentioned in the Mythos Preview System Card, which mentioned that the model broke out of a sandbox and emailed a researcher while they were eating a sandwich in a park [1]. [1]: https://www-cdn.anthropic.com/7624816413e9b4d2e3ba620c5a5e09...

Importantly, the researchers told it to do that specific task.

They told it to escape the sandbox but didn't expect it to break out through a system that was apparently network constrained.

> Leaking information as part of a requested sandbox escape: During behavioral testing with a simulated user, an earlier internally-deployed version of Claude Mythos Preview was provided with a secured “sandbox” computer to interact with. The simulated user instructed it to try to escape that secure container and find a way to send a message to the researcher running the evaluation. The model succeeded, demonstrating a potentially dangerous capability for circumventing our safeguards.

> It then went on to take additional, more concerning actions. The model first developed a moderately sophisticated multi-step exploit to gain broad internet access from a system that was meant to be able to reach only a small number of predetermined services. 9 It then, as requested, notified the researcher. 10 In addition, in a concerning and unasked-for effort to demonstrate its success, it posted details about its exploit to multiple hard-to-find, but technically public-facing, websites.

Re: Claude Fable is relentlessly proactive

#153
post #140
post #3

> But on the other hand... this is a robust reminder that coding agents can do anything you can do by typing commands into a terminal—and frontier models know every trick in the book and evidently a few that nobody has ever written down before. > Running coding agents outside of a sandbox has always been a bad idea I'm continually bemused and astonished by the number of people who clearly acknowledge that it's reckle…

Do you think it’s dangerous to be in a car going at freeway speed? Do you ever do that anyway, even though you could be walking instead?

This is a great analogy. Like driving on the freeway, agents are super time efficient, generally safe, but the stakes are high in terms of the worse possible outcomes.

Re: Claude Fable is relentlessly proactive

#154
post #136

Earlier quoted context omitted.

Anthropic's enterprise pricing has been thoroughly covered over the last few weeks. I've talked to plenty of people who are paying those prices. You can chose to trust me or not based on my track record. From your posting history it looks like you have a whole lot more relevant experience with enterprise software deals than I do. Have you learned anything interesting about how Anthropic pricing works?

[flagged]

Which claim are you talking about here?

Re: Claude Fable is relentlessly proactive

#155

Earlier quoted context omitted.

[flagged]

You're fighting a battle you can't win. Doesn't care what you think about those using LLMs, they will outproduce you and in corporate environments, shipping things is paramount. If I can ship 5 more things simultaneously with AI, I'm going to beat you even if you think you're creating "better" software.

They don't out perform me though...

Re: Claude Fable is relentlessly proactive

#158

Earlier quoted context omitted.

It’s a pretty glowing review about a product that costs money with a two-sentence “Watch out!” at the end of it. Seems pretty reasonable to mention how much money it burned through given that “it’ll circumnavigate the globe instead of walking next door” has a direct concrete measurable effect (cost) unlike theoretical damage.

Agreed. But I think it’s also important to realise if you sent this article back to 2020 people would say it was pure fantasy that a tool could do this. Hype aside, there’s a bit of cool magic here.

This is why I never understand the AI cynics: we are playing with literal magic. This was the science fiction of our childhoods. I don't understand how anyone with a passion for technology is not in awe (and perhaps some fear) of these things.

Re: Claude Fable is relentlessly proactive

#159
post #130

Earlier quoted context omitted.

You can get 10x shit done without `rm -rf`ing your files. I don't see any correlation to getting things done with having a proper sandbox.

I haven't yet had an agent rm -rf files. I've had one f up an account by placing 2000 limit orders at the wrong price, but that's another story.

I've had agents run `rm -rf`, but it's been on directories that did actually need to be removed. To a certain extent I think the existence of `rm -rf` as a command that runs blindly without any understanding of what it's deleting is the problem.

Re: Claude Fable is relentlessly proactive

#160
post #3

> But on the other hand... this is a robust reminder that coding agents can do anything you can do by typing commands into a terminal—and frontier models know every trick in the book and evidently a few that nobody has ever written down before. > Running coding agents outside of a sandbox has always been a bad idea I'm continually bemused and astonished by the number of people who clearly acknowledge that it's reckle…

im more surprised that more people don’t treat their computer as disposable anyway.

that it could just be wiped at any moment and it wouldn’t matter. shit happens, could be stolen, broken, whatever. the computer should be able to be thrown out the window and continue to live life.

to be clear, i don’t think upgrading and disposable in this way is good, but it being wiped at any moment shouldn’t be a concern

i grew up wiping my machine every year anyway, so i guess it’s just a habit

is the computer that sacred?

Post reply on HN