Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

theregister.com

Post reply on HN
Print view View original

101–110 of 382 posts

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#101

Post by cogman10 »

Retr0id wrote: post #96
NiloCK wrote: post #86

Earlier quoted context omitted.

This is a credentials and access list oAuth style problem, and not really intractable. For package X , I should be able to present my npm (homebrew, apt, nuget, etc) credentials with publishing rights for the package. If package X is of sufficient public interest (user count, nature/sensitivity of user data, downstream distribution, etc), then the public interest + cryptographic credentials should permit access to be…

This is not tractable, because there is nothing stopping me from copy-pasting someone else's project into my own namespace. Under most OSS licenses I have express permission to do so. If you try to do some kind of dupe-detection, someone can use a lightweight LLM to make superficial changes until it's considered a different project. Finally, the meatspace status quo is that it is totally acceptable to pay someone to…

> If you try to do some kind of dupe-detection, someone can use a lightweight LLM to make superficial changes until it's considered a different project.

Even if you don't, a lot of source code can be legitimately copied thanks to the GPL/MIT/BSD/etc. I'm allowed to take all of zlib and integrate it into my own project if I so chose.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#102

Post by Yossarrian22 »

NiloCK wrote: post #86
cogman10 wrote: post #56

Earlier quoted context omitted.

Ok, and how is that determined? How does anthropic know my "kernel" project isn't a personal toy and not the Linux kernel? How does anthropic determine I'm a legitimate kernel hacker? What proof do I give them and how does it tie back to my email? What would the steps be to create a new project? Do I need to send anthropic a list of my team members each time and keep them updated as the company changes? Shall I be gi…

This is a credentials and access list oAuth style problem, and not really intractable. For package X , I should be able to present my npm (homebrew, apt, nuget, etc) credentials with publishing rights for the package. If package X is of sufficient public interest (user count, nature/sensitivity of user data, downstream distribution, etc), then the public interest + cryptographic credentials should permit access to be…

And what if I’m a crazy person and want to fork the Linux kernel as I’m legally allowed to do?

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#103

Post by Retr0id »

cogman10 wrote: post #101
Retr0id wrote: post #96

Earlier quoted context omitted.

This is not tractable, because there is nothing stopping me from copy-pasting someone else's project into my own namespace. Under most OSS licenses I have express permission to do so. If you try to do some kind of dupe-detection, someone can use a lightweight LLM to make superficial changes until it's considered a different project. Finally, the meatspace status quo is that it is totally acceptable to pay someone to…

> If you try to do some kind of dupe-detection, someone can use a lightweight LLM to make superficial changes until it's considered a different project. Even if you don't, a lot of source code can be legitimately copied thanks to the GPL/MIT/BSD/etc. I'm allowed to take all of zlib and integrate it into my own project if I so chose.

Yup, I just added something to that effect, sorry if my edit arrived after you replied.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#104

Post by piokoch »

martinald wrote: post #6

If you set aside political menace, this is a huge problem with Anthropic's strategy. You _cannot_ say that Mythos is super dangerous and can only be rolled out to certain people, but then release Fable with anything other than bulletproof cyber denials. Clearly with LLMs, bulletproof denials are ~impossible due to the way LLMs work. So you've ended up in a situation where Anthropic are simultaneously claiming it's a…

If it weren't for the IPO, Anthropic would just ship another model, called Opus 4.898, people would run another "duck on the bicycle" test that would be slightly better than the one from previous version 4.897 and move on.

But we have IPO coming, hence we face that big drama about model that would enable Iran to produce nukes, ok, that card was played, so maybe Taliban producing some magic poison to kill all Americans or some really bad people (Venezuelans?, Cubans? Somalian football referees?) to break into Github and make Github Actions working even worst (if this is even possible).

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#105

Post by bitexploder »

HarHarVeryFunny wrote: post #69
dathinab wrote: post #9

Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…

Exactly - it effectively is a "jail break" since it accomplishes something the model's security filter was trying to prevent, and the ridiculous simplicity of it shows just how broken that type of security is. I wonder if Dario is now regretting hyping up how dangerous the model is? How does he walk this back? Do the feds let him just put a band-aid on it?

I also have a 100% success rate jail breaking them by breaking the work down into small pieces and stripping all security related language. Smaller tasks, test engineering and normal programming language. Fable found a few bugs in my harness for me before they pulled it. I was testing it vs ChatGPT, Gemini, and Opus. It was doing well at bug hunting.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#106

Post by TZubiri »

>“That’s it,” Moussouris wrote. “‘Fix this code,’ plus several manual steps to generate test scripts, should never have triggered an export control. I feel like making ’90s-style t-shirts with ‘fix this code’ on the front and ‘this shirt is a munition’ on the back.”

Huh? Presumably if it shipped without guardrails, then it would still have triggered an export control, would you make a plain shirt on the front which says this shirt is a munition on the back?

The munition is the exported good, not the bypass of its safety feature. If anything that the bypass is 3 words long should make the export restriction more justified, not less.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#107

Post by ianm218 »

dkdcdev wrote: post #97
ianm218 wrote: post #71

Earlier quoted context omitted.

Isn’t your point that AI safety is impossible to prevent 100% of bad things? It is quite hard (but not impossible) to get an the frontier AI to tell you how to build a nuke or launder money now, where jailbreaks used to be trivial “ignore all previous instructions”. It seems like a worthwhile effort.

The idea that an LLM can discern intent on any given prompt is farcical. I might be researching nukes to commit an atrocity, or to prevent one. I might be asking about laundering money to commit a crime, or to prevent one. I might be researching the Nazis because I want to commit a genocide, or I want to read up so I know how to prevent one. Same with cybersecurity. Same with anything. In my opinion, these companies…

I don't really agree with it but the government is moving towards making you ID yourself to use frontier AI - i.e. only US citizens are going to be able to use Claude Fable supposedly. In that regime the AI companies would in fact know if you are a money laundering expert or a normal software engineer.

> The idea that an LLM can discern intent on any given prompt is farcical.

Not really though. For most people in most situations it's just not going to give you that info. Software security is a niche where its a bit strange in that there is 100X the amount of white hat users than bad actors and there's open source etc.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#108

Post by sophrosyne42 »

NiloCK wrote: post #86
cogman10 wrote: post #56

Earlier quoted context omitted.

Ok, and how is that determined? How does anthropic know my "kernel" project isn't a personal toy and not the Linux kernel? How does anthropic determine I'm a legitimate kernel hacker? What proof do I give them and how does it tie back to my email? What would the steps be to create a new project? Do I need to send anthropic a list of my team members each time and keep them updated as the company changes? Shall I be gi…

This is a credentials and access list oAuth style problem, and not really intractable. For package X , I should be able to present my npm (homebrew, apt, nuget, etc) credentials with publishing rights for the package. If package X is of sufficient public interest (user count, nature/sensitivity of user data, downstream distribution, etc), then the public interest + cryptographic credentials should permit access to be…

You are talking about creating a big moat, which might be a worse precedent than removing fable access altogether.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#109

Post by minraws »

dathinab wrote: post #9

Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…

I am not sure but I have been using codex and claude like this for a while now didn't know it was untoward or malicious jail braking since codex & claude would refuse to work if you ask it to implement a feature in a reverse engineering tool I was building.

I even moved to using Deepseek for helping with it for a bit.

And for properly working drivers for some old locked down hardware.

Could I have phrased it better and not hit model guardrails sure. But this seemed genuinely obvious, since my intent wasn't well bad.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#110

Post by cogman10 »

Yossarrian22 wrote: post #102
NiloCK wrote: post #86

Earlier quoted context omitted.

This is a credentials and access list oAuth style problem, and not really intractable. For package X , I should be able to present my npm (homebrew, apt, nuget, etc) credentials with publishing rights for the package. If package X is of sufficient public interest (user count, nature/sensitivity of user data, downstream distribution, etc), then the public interest + cryptographic credentials should permit access to be…

And what if I’m a crazy person and want to fork the Linux kernel as I’m legally allowed to do?

Not just allowed to do, encouraged to do as part of legitimate development.
Post reply on HN