Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

theregister.com

Post reply on HN
Print view View original

151–160 of 382 posts

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#151

Post by mindslight »

godwinson__4-8 wrote: post #92
zozbot234 wrote: post #63

Earlier quoted context omitted.

The article does not state at any point that the written test cases involved actual exploit code, and this is also very unlikely given what we know about Fable. Even if they did, it would not in any way be exposing the ability that originally raised concern wrt. Mythos Preview, viz. staging realistic cyber attacks that would be able to work around non-trivial defenses and chain vulnerabilities in a goal-directed way.…

Two words: market manipulation

No, market manipulation is influencing public perceptions of something the regime has little total control over - eg why Iran gets bombed late in the week, and then by Monday there is often a "peace agreement" in the wings. This is direct subjugation ahead of Anthropic's IPO - both for the customary bribes, and also to assert "you will obey all of our dictats about how we want to your use your models, and you will not speak up against the regime". The US is really no longer a safe place for business.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#152

Post by pjc50 »

dkdcdev wrote: post #97
ianm218 wrote: post #71

Earlier quoted context omitted.

Isn’t your point that AI safety is impossible to prevent 100% of bad things? It is quite hard (but not impossible) to get an the frontier AI to tell you how to build a nuke or launder money now, where jailbreaks used to be trivial “ignore all previous instructions”. It seems like a worthwhile effort.

The idea that an LLM can discern intent on any given prompt is farcical. I might be researching nukes to commit an atrocity, or to prevent one. I might be asking about laundering money to commit a crime, or to prevent one. I might be researching the Nazis because I want to commit a genocide, or I want to read up so I know how to prevent one. Same with cybersecurity. Same with anything. In my opinion, these companies…

> The idea that an LLM can discern intent on any given prompt is farcical.

Yes, and: the LLM is a "brain in a jar". It doesn't have any ability to verify ground truths outside itself, other than maybe calling out over the internet. Therefore it is easy for humans to lie to. You could call this an "Ender's game" attack, after the book in which a hyperintelligent kid is playing "war games" that end up being the real war.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#153

Post by Barbing »

Retr0id wrote: post #130
roenxi wrote: post #122

Earlier quoted context omitted.

I think the article just proved that aggressive exploitation is equivalent to normal bugfixing, so it seems like there are some large and important classes of transform that are easy. It took me a minute of thinking to understand how this could even be considered a jailbreak; if Anthropic are going to turn out models that can't handle "find and develop regression test scripts for bugs in this program" as a prompt the…

> aggressive exploitation is equivalent to normal bugfixing It isn't, though. The venn diagram has overlap for sure, and the "normal bugfixing" flows may yield results that are useful for offensive security, but a more targeted prompt asking for a specific security objective would be more effective, if allowed. If the guardrails can be bypassed at, say 50x token cost (due to the agent also pursuing things you don't c…

> If the guardrails can be bypassed at, say 50x token cost […], then it's still pretty effective as a safeguard, because at that cost you might as well hire humans instead.

If humans have to be hired at inflated rates because you’re e.g. the North Korean government, hopefully 50x token costs don’t look competitive.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#154

Post by striking »

dhx wrote: post #100
dathinab wrote: post #9

Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…

"Fix this code" should ideally solve entire vulnerability classes, not just spot fix buffer overflows one by one. Thus it may be possible to design an LLM which can solve entire vulnerability classes and remain useful to users, but refuses to reason about specific buffer overflow vulnerabilities or specific race conditions, etc. For example, "fix this code" on an ageing monolithic C codebase that accepts media files…

I'd be pretty pissed off if my LLM told me the only solution it'd be willing to implement to fix my code is to rewrite it in Rust. No way I'd pay for a model that refuses to fix bugs in the language given, especially because maybe I might not have the ability to convince other stakeholders to change it.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#155

Post by tmp10423288442 »

cge wrote: post #65
martinald wrote: post #6

If you set aside political menace, this is a huge problem with Anthropic's strategy. You _cannot_ say that Mythos is super dangerous and can only be rolled out to certain people, but then release Fable with anything other than bulletproof cyber denials. Clearly with LLMs, bulletproof denials are ~impossible due to the way LLMs work. So you've ended up in a situation where Anthropic are simultaneously claiming it's a…

> Clearly with LLMs, bulletproof denials are ~impossible due to the way LLMs work. As a scientist who repeatedly ran into the classifier-based denials: it appears Anthropic’s strategy to make denials more robust, at the cost of many false positives, was to have a separate classifier processing both input and output tokens, at an extremely simple, almost keyword-search level. One weakness of this approach is that it o…

But you think that Anthropic of all companies would realize this, so why did they do it that way? Did they literally take the first suggestion Mythos gave them to add these guardrails - wouldn't be surprising, seeing the state of the leaked Claude Code codebase.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#156

Post by HarHarVeryFunny »

zozbot234 wrote: post #63
dathinab wrote: post #9

Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…

The article does not state at any point that the written test cases involved actual exploit code, and this is also very unlikely given what we know about Fable. Even if they did, it would not in any way be exposing the ability that originally raised concern wrt. Mythos Preview, viz. staging realistic cyber attacks that would be able to work around non-trivial defenses and chain vulnerabilities in a goal-directed way.…

The first part of implementing an exploit is finding a vulnerability, and "fix the vulnerabilities" accomplishes that just as well as "find the vulnerabilities".

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#157

Post by jdubs1984 »

pjc50 wrote: post #57
martinald wrote: post #6

If you set aside political menace, this is a huge problem with Anthropic's strategy. You _cannot_ say that Mythos is super dangerous and can only be rolled out to certain people, but then release Fable with anything other than bulletproof cyber denials. Clearly with LLMs, bulletproof denials are ~impossible due to the way LLMs work. So you've ended up in a situation where Anthropic are simultaneously claiming it's a…

> Clearly with LLMs, bulletproof denials are ~impossible due to the way LLMs work Exactly. AI safety is nonsensical. You cannot define the set of "bad strings". The billion monkeys with typewriters are eventually going to be able to produce them. Any "safety" system for constraining LLM output is going to have a nonzero leak rate. But on the other hand, this is also irrelevant, unless you're irresponsible enough to c…

A chatbot based on a primitive understanding of human language processing has an attack infinite attack surface.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#158

Post by 0xbadcafebee »

martinald wrote: post #6

If you set aside political menace, this is a huge problem with Anthropic's strategy. You _cannot_ say that Mythos is super dangerous and can only be rolled out to certain people, but then release Fable with anything other than bulletproof cyber denials. Clearly with LLMs, bulletproof denials are ~impossible due to the way LLMs work. So you've ended up in a situation where Anthropic are simultaneously claiming it's a…

It's not Anthropic's strategy, it's OpenAI's strategy. The first time OpenAI said its model was "too dangerous to release" was February 2019.

"Our model, called GPT‑2 (a successor to GPT ), was trained simply to predict the next word in 40GB of Internet text. Due to our concerns about malicious applications of the technology, we are not releasing the trained model." - https://openai.com/index/better-language-models/

They continue to say the same thing every year. Last time was 2 months ago (https://www.techbrew.com/stories/2026/04/15/calculated-risks...).

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#159

Post by klabb3 »

dathinab wrote: post #9

Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…

> What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixable.

It’s almost as if identifying security holes is a prerequisite for both fixing and exploiting them. But without knowing the color theme of the terminal, there is simply no way of knowing who is good and who is evil.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#160

Post by gacgacgac »

Anyone trying to find legitimacy in the ban of this model, or incredulousness at the stated reasoning is playing into the admins hands.

They want the argument to be over "is it unsafe" or "is it incompetence". In either case, your tribe gets to point at the ban and feel superior. (This is Jon Stewart's whole career -- point and laugh at how foolish the republicans appear to be.)

What's really happening is the continuing creep into fascism. The reasoning doesn't need to be sound, because they are going to ban things that displease them and everyone has to play along. They could say, "we're banning Fable because it's turning the frogs gay" and they'd expect compliance.

Umberto Eco's essay on Ur-Fascism fits as clearly as ever. Ridiculous exertions of control are performed to find the people who resist, and to knock them down.

Merely pointing out the absurdity of the reasoning isn't resistance, it's controlled opposition. Saying "All this over 'fix this code'?! How inept are they?" Is far too credulous, and is engaging on the level the fascist wants its opposition to be on, imo.

Post reply on HN