Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

theregister.com

Post reply on HN
Print view View original

121–130 of 382 posts

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#121

Post by giancarlostoro »

dathinab wrote: post #9

Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…

This is the weird distinction with AI that I've complained about for ages, how can we make it do lawful good, its nearly impossible. Ask an AI to give you regex to filed our racial slurs, and things fall apart really quickly, it scolds you about not saying slurs. Even though regex implies it looks nearly nothing like a slur.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#122

Post by roenxi »

Retr0id wrote: post #50
zipy124 wrote: post #29

Earlier quoted context omitted.

What's surprising to me is that anyone who has a CS education thinking that jailbreaks are not trivial. It is as simple as normal algorithmic reduction [1], e.g can I transform a dangerous task into a not-dangerous task that the LLM will agree to solve, and then re-transform back. [1]: https://en.wikipedia.org/wiki/Reduction_(complexity)

Something being possible doesn't mean it's easy. Transforming a problem from a forbidden shape into an allowed shape could well be harder than just solving the original problem.

I think the article just proved that aggressive exploitation is equivalent to normal bugfixing, so it seems like there are some large and important classes of transform that are easy.

It took me a minute of thinking to understand how this could even be considered a jailbreak; if Anthropic are going to turn out models that can't handle "find and develop regression test scripts for bugs in this program" as a prompt then it is going to take serious model crippling. To be able to prompt the model someone will need to already understand secure programming - the model itself won't be able to independently detect security problems without active guidance.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#123

Post by KronisLV »

cbg0 wrote: post #114
KronisLV wrote: post #93

Earlier quoted context omitted.

> What proof do I give them and how does it tie back to my email? Presumably your ID so that feds may pay you a visit when they feel like it, your email need not apply. I’m surprised that there’s even enough pushback against ID verification to matter, all the corpos are probably salivating at the idea of having fully accurate profiles of everyone, think of the ad and product targeting. The govt. would also love that,…

How will the "feds" pay you a visit in Albania or China?

Simple - you wouldn’t be given access to those models, and probably all VPN access would be blocked too. Since this is a hypothetical, throw in a social credit score as well to require a proven “track record”, but maybe that’s too exaggerated (although credit scores already exist for different purposes).

It’s not too hard to imagine a future where you can only use certain things only with the govt. mandated spyware installed - bank apps already often don’t work on rooted Android phones (and you’re expected to use those apps to confirm payments) and all sorts of certification exam software is basically that already if you take a test remotely.

It follows that the same principle would just get pushed further, like what Discord wanted to do etc. Same with how Apple requires your documents for a developer account, Hetzner for a hosting account or Twitch for getting paid by them and tax stuff.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#124

Post by [deleted] »

dathinab wrote: post #9

Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…

[deleted]

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#125

Post by contravariant »

bloppe wrote: post #119
ianm218 wrote: post #107

Earlier quoted context omitted.

I don't really agree with it but the government is moving towards making you ID yourself to use frontier AI - i.e. only US citizens are going to be able to use Claude Fable supposedly. In that regime the AI companies would in fact know if you are a money laundering expert or a normal software engineer. > The idea that an LLM can discern intent on any given prompt is farcical. Not really though. For most people in mos…

The idea that checking for a US ID could possibly stop actual foreign bad actors from using it is also farcical. Millions of stolen identity documents can be bought on the dark web for relatively cheap. North Koreans have been hiring real American citizens for years to infiltrate tons of US tech companies as employees. And ya, it's pretty easy to hide your intent once you have access.

Even that is overselling the effort. Last time I checked you could find IDs with a simple image search.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#126

Post by MPSimmons »

HarHarVeryFunny wrote: post #69
dathinab wrote: post #9

Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…

Exactly - it effectively is a "jail break" since it accomplishes something the model's security filter was trying to prevent, and the ridiculous simplicity of it shows just how broken that type of security is. I wonder if Dario is now regretting hyping up how dangerous the model is? How does he walk this back? Do the feds let him just put a band-aid on it?

I think it's a side effect of the Transformer architecture. The worldview where all input is equally trusted, and there's no concept of "the other", makes it hard to build effective guardrails where some input is trusted and other input is not trusted.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#127

Post by cogman10 »

godwinson__4-8 wrote: post #95
cpburns2009 wrote: post #75

Earlier quoted context omitted.

No, it's regulatory capture. Anthropic is the current leader and they want to ensure their position by forcing regulation to stamp out the Chinese competition.

How does this achieve that goal?

That's what's not clear to me. About the only way this works is if we create the "Great US firewall", or if china decides to also put in export controls around usage of their models (unlikely).

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#128

Post by Supermancho »

cpburns2009 wrote: post #75
jpcompartir wrote: post #8

They weren't freaked by anything, it's a retaliatory shakedown after ideological differences and Anthropic not doing exactly what they're told/what the Admin wants them to do.

No, it's regulatory capture. Anthropic is the current leader and they want to ensure their position by forcing regulation to stamp out the Chinese competition.

> Anthropic is the current leader

How's that determined?

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#129

Post by piokoch »

dathinab wrote: post #9

Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…

There are big theories already born out of that glitch (like https://archive.ph/2OWwO#selection-1373.278-1377.12). The Doom is Coming!

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#130

Post by Retr0id »

roenxi wrote: post #122
Retr0id wrote: post #50

Earlier quoted context omitted.

Something being possible doesn't mean it's easy. Transforming a problem from a forbidden shape into an allowed shape could well be harder than just solving the original problem.

I think the article just proved that aggressive exploitation is equivalent to normal bugfixing, so it seems like there are some large and important classes of transform that are easy. It took me a minute of thinking to understand how this could even be considered a jailbreak; if Anthropic are going to turn out models that can't handle "find and develop regression test scripts for bugs in this program" as a prompt the…

> aggressive exploitation is equivalent to normal bugfixing

It isn't, though. The venn diagram has overlap for sure, and the "normal bugfixing" flows may yield results that are useful for offensive security, but a more targeted prompt asking for a specific security objective would be more effective, if allowed.

If the guardrails can be bypassed at, say 50x token cost (due to the agent also pursuing things you don't care about), then it's still pretty effective as a safeguard, because at that cost you might as well hire humans instead.

And, having to "babysit" a model while you re-prompt to work around guardrails strongly limits how much you can scale up your work.

Post reply on HN