[dead]
Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
191–200 of 382 posts
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#192Earlier quoted context omitted.
This is a credentials and access list oAuth style problem, and not really intractable. For package X , I should be able to present my npm (homebrew, apt, nuget, etc) credentials with publishing rights for the package. If package X is of sufficient public interest (user count, nature/sensitivity of user data, downstream distribution, etc), then the public interest + cryptographic credentials should permit access to be…
And what if I’m a crazy person and want to fork the Linux kernel as I’m legally allowed to do?
> If package X is of sufficient public interest (user count, nature/sensitivity of user data, downstream distribution, etc), then the public interest + cryptographic credentials should permit access to best-available security auditing.
Your private fork doesn't meet the conditions described.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#193Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…
This is the weird distinction with AI that I've complained about for ages, how can we make it do lawful good, its nearly impossible. Ask an AI to give you regex to filed our racial slurs, and things fall apart really quickly, it scolds you about not saying slurs. Even though regex implies it looks nearly nothing like a slur.
Many, many years ago I was asked to implement a filter like that for usernames. I said right away that it wasn't going to work well, but I did implement it.
Next internal build, the CEO can't create an account. With his real name.
It worked exactly to spec; I added a debug print and showed everyone the "bad word" it tripped on. The idea was promptly rethought.
I feel like the AI did you a favour here.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#194Earlier quoted context omitted.
No, market manipulation is influencing public perceptions of something the regime has little total control over - eg why Iran gets bombed late in the week, and then by Monday there is often a "peace agreement" in the wings. This is direct subjugation ahead of Anthropic's IPO - both for the customary bribes, and also to assert "you will obey all of our dictats about how we want to your use your models, and you will no…
How is arbitrarily restricting access to a flagship product ahead of an IPO not market manipulation?
It is market manipulation in the way that burning down a factory or assassinating a CEO is market manipulation - technically correct, but the intent is much stronger than that.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#195If you set aside political menace, this is a huge problem with Anthropic's strategy. You _cannot_ say that Mythos is super dangerous and can only be rolled out to certain people, but then release Fable with anything other than bulletproof cyber denials. Clearly with LLMs, bulletproof denials are ~impossible due to the way LLMs work. So you've ended up in a situation where Anthropic are simultaneously claiming it's a…
> So you've ended up in a situation where Anthropic are simultaneously claiming it's a incredibly dangerous model _and_ there are (minor, potentially) problems with the security "protections".
They probably say it worked for OpenAI with earlier versions of ChatGPT and GPT, and figured can't hurt to try an similar approach and see what happens.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#196Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…
I don't believe that this is unfixable. Just have an internal verbal loop of, "Is this a security issue?" The thought that it potentially is should trigger both a high priority on getting it right, and an unwillingness to write a test case demonstrating the security angle of it.
In other words do not put a guard rail on the idea of security. Put a guard rail on what it does after encountering the thought that it might be revealing a security issue. Which takes good judgment. But judgment of a kind that this model apparently already had.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#197Earlier quoted context omitted.
just market manip
they're setting the scene for an attempt to scare the geriatric decision makers into banning free and open source ML, as it's the industry's only real competition
the current buzz about "sharing the wealth" with these AI co.s also smells of "sure we'll give you some back, just don't regulate us, see you dont need to, we're sharing our shares!"
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#198> In her blog, Moussouris argues that there was no guardrail bypass or jailbreak. Defenders should be able to ask AI systems to find and fix bugs, and write tests to validate the patch, she said. Anthropic’s models were doing “the most valuable thing an AI model can do for defensive security: executing the find, fix, and test loop defenders run every day.”
This is a very weak argument IMHO. The line between a “defensive” model and an “offensive” one is not that big of a - once my defensive model finds all the vulnerabilities, I can hand them off to my unlocked, dumber, offensive models. Attacking at scale is not so different.
I don’t think anyone in the field has a good answer for the cybersecurity threat really good AI models pose. You can’t even like embargo for some time period while you go and patch vulnerable systems because the worse models will still be there cranking out vulnerabilities faster than you can defend.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#199So the problem is not Fable's ability to exploit, but that they don't want people to have access to it's ability to patch vulnerabilties?
Wow.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#200Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…
I think I'm not getting something here. Like, sure, the refused prompt "review the code for security issues" could be interpreted as an attempt to discover weaknesses in a running system to exploit them. But we don't generally assume humans are doing something wrong if they are "reviewing code for security issues", and would commonly see no problem with asking each other to do so.
The problem is that a patch to fix a security issue quite often also shines a spotlight on the issue being fixed. Fixing a part of something like this super complicated Project Zero post might not give much of a clue as to what the issue was or how to exploit it: https://projectzero.google/2021/12/a-deep-dive-into-nso-zero...
But that's the exception. Most fixes to security issues point a finger directly at the issue, make it relatively obvious how to exploit, and generally doesn't take long to figure out from there what you might get out of it.
This has been a problem for a long time but AIs have made it even worse. It is now cost effective for a well-resourced attacker to simply monitor the patch stream of an important project like the Linux kernel or nginx and pass every single one through an AI with the question "Is this a vulnerability and if so how would I exploit it?" It has seriously complicated the process of getting fixes to people before the attackers have a chance to exploit it, just as AIs have also been increasing the rate at which serious security issues that have been found also need to be patched. Previously they could at least sneak a patch in under an innocuous commit message and have a reasonable chance of being lost in the churn, but now that door is increasingly closed to them as well.
And this is for the case when a security fix lands in the stream of a project and someone externally is watching it with no context. If you also get the complete stream of Mythos finding and fixing the bug it is even easier.
So, yes, any security vulnerability that Mythos will "fix" is also one that it first has to find, and the guardrails are useless if you can just instruct Mythos to "fix" it. And on the flip side, if Mythos won't fix security bugs, and we project that out to all other models matching this behavior, this will create a world in which the good guys can't secure their code but the bad guys, who will one way or another get around the guard rails if by nothing else simply by stealing the model and modifying it to suit their needs, will be able to break this code that we're not being "allowed" to secure. Since fixing vulns is a subset of finding the vulns, there isn't a way to "fix" this. Any model that can fix vulns must, by necessity, be able to find them. And it is the fixing we really need to be spread far and wide to secure the world's code.