Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

theregister.com

Post reply on HN
Print view View original

311–320 of 382 posts

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#311

Post by thinkindie »

bflesch wrote: post #271
thinkindie wrote: post #252

As an European, I really don't get where this strategy wants to take the USA to. It's pretty clear everyone is getting scared about changes like this that happen overnight, without clear reason and completely unpredictable. Business requires a stable environment, and Trump is making everything in his power to disrupt business stability. Ultimately, I see the rest of the world (especially Europe) relying less and less…

> Ultimately, I see the rest of the world (especially Europe) relying less and less on US tech. The long term damage is done. They know it and they try to slow it down as much as possible.

How? If anything it seems like they are accelerating some processes - not least the export control over Fable just few days ago or the erratic behavior with the war with Iran

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#312

Post by tracker1 »

dathinab wrote: post #9

Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…

Security vulnerability guardrails are kind of stupid to begin with... I would want the AI agent to be able to fix my security issues... having it obscured is just begging for more unsafe code in the world.

Oh, I'll just leave this SQL injection path in place.... etc.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#313

Post by deadbabe »

thewebguyd wrote: post #309
deadbabe wrote: post #221

Earlier quoted context omitted.

There is a solution: users must not be allowed to directly read code. Your code could be entirely hosted and edited on Anthropic servers, visible only to LLMs, and when it’s time to deploy Anthropic handles deployment for you.

I hope this is satire?

Why satire? Instead of dumping code on GitHub, you open repos on Anthropic and the details of languages and code are all abstracted away for you. You just have your application deployed and you use it as you develop and request changes. Zero code.

If you want escape hatch, Anthropic can just dump all the code for you and you download the zip.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#315

Post by michaellee8 »

neuronexmachina wrote: post #229
dathinab wrote: post #9

Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…

Also worth noting that the main touted difference with Claude Mythos isn't it's ability to find vulnerabilities, but rather chaining them together to create full useable exploits. I haven't heard of any evidence that the Claude Fable "fix this code" jailbreak could have been used to do exploit-chaining.

if you actually figure out enough pieces of bugs, even opus level model would be able to chain it together imo, and the latest china models has already been described as close to such level.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#316

Post by pembrook »

jp57 wrote: post #281

I think this brings out the cognitive dissonance around "safety" regarding cyber security: a) In order to make us safe, the LLM should help us find (and fix) the vulnerabilities in our own code. b) In order for us to be safe, the LLM should not find vulnerabilities in other people's code. I don't think this is resolvable in a way where both (a) and (b) win.

Yes, it's so wildly silly if you assume good faith on the part of both parties.

Hence why I think the real explanation lies in bad faith positions from both the US Government and Anthropic:

Anthropic's doomerism-as-marketing (in reality its like 17% better at coding) basically enabled the US Gov to plausibly take them down on an irrelevant technicality as retribution for the dept of war showdown.

Both groups (the current US Admin and Anthropic) are full of authoritarian-minded people, just on opposite ends of the political spectrum. Which is the only thing I find scary here, not the silly LLMs.

To me, OpenAI seems like the least bad option given they're a quaint old "center-left in the streets, center-right in the sheets" capitalist enterprise.

At least I know why they make the decisions they make. I trust the people building a profit-seeking enterprise more than I trust people trying to build a religion using compute.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#317

Post by ceejayoz »

catigula wrote: post #305
ceejayoz wrote: post #273

Earlier quoted context omitted.

> This literally means the models are too dangerous to release… Unless you believe Anthropic has an irreplacable wizard or genie or fairy chained up somewhere that other providers can't replicate, someone is going to release such a thing, and that someone might be a lot more cavalier about the safety of it.

Yes, this is the flawed logic Anthropic is using to do dangerous things; it's not lost on anyone.

What's flawed about the logic?

Are we gonna drone strike China's datacenters when they release a similar model?

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#318

Post by godwinson__4-8 »

HWR_14 wrote: post #301
godwinson__4-8 wrote: post #170

Earlier quoted context omitted.

How is arbitrarily restricting access to a flagship product ahead of an IPO not market manipulation?

The company hasn't IPOed so it's not on the market.

You should run for office. You'd fit in.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#319

Post by Supermancho »

dgellow wrote: post #131
Supermancho wrote: post #128

Earlier quoted context omitted.

> Anthropic is the current leader How's that determined?

API usage? They are for sure leading in the enterprise world

API usage is a poor metric, but it's a metric, for sure.

I would not say Anthropic is leading in the enterprise, depending on how you define enterprise. It's leading in marketing, to be sure.

Ofc, my sample size is a few companies and all the developers I know.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#320

Post by thewebguyd »

deadbabe wrote: post #313
thewebguyd wrote: post #309

Earlier quoted context omitted.

I hope this is satire?

Why satire? Instead of dumping code on GitHub, you open repos on Anthropic and the details of languages and code are all abstracted away for you. You just have your application deployed and you use it as you develop and request changes. Zero code. If you want escape hatch, Anthropic can just dump all the code for you and you download the zip.

> details of languages and code are all abstracted away for you

You don't see how that's a problem? You're arguing for a fully vibe coding solution to software engineering, we simply aren't there yet. Human-in-the-loop intervention is still required. I still write code, every day, and use AI heavily.

That could possibly work for simple React/TypeScript SPAs, it's probably the stack that these models excel with the most. It's a complete non starter for anyone wanting to use these tools on existing brownfield projects. Opus notably falls over trying to do anything with legacy .NET Framework & WPF/XAML, obscure hardware SDKs (ID scanners, for example, hardware I deal with at work), industrial control software.

There's no world where I can upload our codebase to Anthropic and have it just abstract everything away and make arbitrary decisions. There's no amount of prompt engineering where LLMs in their current state are going to be able to figure out an unmaintained SDK for some obscure hardware that hasn't been updated since 2008. The enterprise world is full of stuff like that.

Post reply on HN