If you're a global health benefits platform that relies on an AI model, do you think you're going to choose one that can get shutoff by a country due to something not remotely related to your business? If you're a buyer of that benefits platform, do you factor this into your purchasing now? X every industry.
Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
241–250 of 382 posts
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#242Of course it isn't about that, what we see online in the "news" is completely irrelevant with reality in most cases, it's exhausting to see people parroting what giant corps & gov are saying as if it's not extremely well crafted and plain false or deceptive most of the time. It's not even about politic left or right, both sides are acting completely dumb about it, look at Google trends, people are literally being "sw…
What makes it a risky bet?
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#243>“The behavior described in the paper cannot meaningfully be fixed, and any attempt would only weaken the model for defense,” said Moussouris, who criticized the export control directive as hasty, heavy-handed, and misguided.
This literally means the models are too dangerous to release, and yet he and they reached the opposite conclusion.
A lot of people have been saying this repeatedly for a long time.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#244Earlier quoted context omitted.
That would make the model useless
How does this make the model useless? It finds and fixes the security hole. It can even write a test that verifies that the fix didn't break things. But it deliberately doesn't reveal the fact that it was a security issue that was fixed. Seems useful to me. But more useful for defenders than attackers.
Imagine that you have the repo A, ask the model to "fix the security issue" and end up with A'.
Just take the Diff A' - A to see the security hole.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#245>“The behavior described in the paper cannot meaningfully be fixed, and any attempt would only weaken the model for defense,” said Moussouris, who criticized the export control directive as hasty, heavy-handed, and misguided. This literally means the models are too dangerous to release, and yet he and they reached the opposite conclusion. A lot of people have been saying this repeatedly for a long time.
Or perhaps: we don't want our adversaries fixing all the security holes we rely on.
Or even: this is a good chance to stick it back to Anthropic.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#246Does anybody actually trust the official version of events from the US government anymore? I know I sure don't. For all I know, this was an insider play to boost the spacex valuation or something equally meaningless and stupid.
This is not the official version of the events in any sense. Some "expert" looked at report WH saw and said this. That "expert" has probably never been involved in anything like that.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#247Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…
It is fixable. Model requires proof that you are a legitimate developer of that piece of software. Every Anthropic/OpenAI account will have a list of projects the model is allowed to work on for security issues.
lol worst idea ever
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#248>“The behavior described in the paper cannot meaningfully be fixed, and any attempt would only weaken the model for defense,” said Moussouris, who criticized the export control directive as hasty, heavy-handed, and misguided. This literally means the models are too dangerous to release, and yet he and they reached the opposite conclusion. A lot of people have been saying this repeatedly for a long time.
Mousssouris is not a "he".
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#249Earlier quoted context omitted.
This is the weird distinction with AI that I've complained about for ages, how can we make it do lawful good, its nearly impossible. Ask an AI to give you regex to filed our racial slurs, and things fall apart really quickly, it scolds you about not saying slurs. Even though regex implies it looks nearly nothing like a slur.
Many, many years ago I was asked to implement a filter like that for usernames. I said right away that it wasn't going to work well, but I did implement it. Next internal build, the CEO can't create an account. With his real name. It worked exactly to spec; I added a debug print and showed everyone the "bad word" it tripped on. The idea was promptly rethought. I feel like the AI did you a favour here.
Ah the classic Scunthorpe problem
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#250Is defenders a common term used in cybersecurity? Idk why but it's giving war fighters vibes. I've noticed it on all the anthropic blog posts and then this one.
Yes, and it's effective marketing. The war fighter vibes are thrilling. There's a tribal sense of us-vs-them, there's danger, there's the prospect of victory or defeat. Security products marketing is full of these ideas, because security is about preventing arbitrarily bad things from happening. So evoking your worst imaginable nightmare scenario is a great way to get you excited about buying something that might help prevent it.