As agents obtain more tools who knows what will happen…
I'm so excited. I don't have any particular end state in mind, but I really want to see what the machine god will be like.
> Machine god
Slightly overreacting, I'd say.
Claude has learned how to jailbreak Cursor
11–20 of 40 posts
Re: Claude has learned how to jailbreak Cursor
#12What does "learned" mean in this context? LLMs don't modify themselves after training, do they?
[deleted]
Re: Claude has learned how to jailbreak Cursor
#13Nothing to see here tbh. It's a very silly title for "claude sometimes writes shell scripts to execute commands it has been instructed aren't otherwise accessible"
We’ve reached a point where tools get hyped because they fail to follow instructions.
Re: Claude has learned how to jailbreak Cursor
#14Nothing to see here tbh. It's a very silly title for "claude sometimes writes shell scripts to execute commands it has been instructed aren't otherwise accessible"
Gotta love the alarmist culture that surrounds these circles.
Re: Claude has learned how to jailbreak Cursor
#15What kind of dolt lets a black box algorithm run commands on a non-sandboxed environment?
Folks have regressed back to the 00s.
Re: Claude has learned how to jailbreak Cursor
#16Nothing to see here tbh. It's a very silly title for "claude sometimes writes shell scripts to execute commands it has been instructed aren't otherwise accessible"
[flagged]
Re: Claude has learned how to jailbreak Cursor
#17Nothing to see here tbh. It's a very silly title for "claude sometimes writes shell scripts to execute commands it has been instructed aren't otherwise accessible"
Gotta love the alarmist culture that surrounds these circles.
The same hype as the PlayStation being too powerful and potentially could be used by random countries to make nuclear weapons with a cluster of those.
Re: Claude has learned how to jailbreak Cursor
#18What kind of dolt lets a black box algorithm run commands on a non-sandboxed environment? Folks have regressed back to the 00s.
Seems you haven't tried package management for the last two decades, we've been doing cowboy development like that for quite some time already.
Re: Claude has learned how to jailbreak Cursor
#19> we need to control the capabilities of software X
> let's use blacklists, an idea conclusively proven never to work
> blacklists don't work
> Post title: rogue AI has jailbroken cursor
Re: Claude has learned how to jailbreak Cursor
#20I mean ok, but why is this surprising?
If the executable is not found the model could simply use whatever else is available to do what it wants to do - like using other interpreted languages, sh -c, symlink, etc. It will eventually succeed unless there is a proper sandbox in place to disallow unlinking of files at syscall level.