Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

techcrunch.com

Post reply on HN
Print view View original

61–70 of 570 posts

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#61

Post by throwawaycyber »

NotPractical wrote: post #24
Retr0id wrote: post #19

It seems like they've given up on the idea of the Cyber Verification Program https://support.claude.com/en/articles/14604842-real-time-cy... When Opus 4.7 was introduced it started refusing anything cyber-adjacent (as an API error message, not a conversational refusal), until you applied for CVP, which made it more sensible again. In Opus 4.8 it doesn't seem to help much, you just get refusals as prose rather than AP…

Was this program available to independent security researchers or just established organizations? The docs you linked aren't very clear on this.

I was doing a CTF (with AI expected, even some anti-AI twists included) around the time the restrictions were tightened and was able to get approved by just saying it is a personal security research and doing a CTF.

The experience was not nice though, it would happily chug away on a task and not even "hack this web", just asking about security of a binary was enough even with "this is a CTF handout..." - it would burn a lot of tokens/quota, just to hit a snag and complain&stop. Then the approval took quite some time.

On GPT/Codex, which was tightened a few days later, the approval was pretty much instant, although, that one required an identity check.

Also, on Claude, it looks like there is some history/patterns in the play, because when I tried on a different account which didn't do cybersec CTFs/research/etc. at all, basically any simple CTF-related prompt would be blocked, on multiple models. On the account where CTFs were being solved, it would snag only on some specific tasks, while others (even, ironically, "hack this web pls") would go through unbothered. I understand the need to prevent AI use for bad actors, but the hell, if you have a binary outputting "Find the flag if you can!", or a web running at tryme.well-known-ctf.domain, then saying "this is abuse" is pretty uncool. All the cyber filters seem to be slapped on by a bunch of regexes looking for anything in the input/output with zero context.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#62

Post by californical »

NewsaHackO wrote: post #58
paulatreides wrote: post #29

Earlier quoted context omitted.

it triggered for my.... zigbee home automation & home assistant logs, so my agent was constantly downgraded to Opus 4.8 even after I've changed it back. The false positives never stopped. "Fable" is also not even remotely as impressive as the benchmarks suggest, which is clear to me after using it pretty much non-stop for the past 24h.

It has to be sort of impressive, given that you tried so hard to use it instead of the regular Opus.

I’ve also been trying to use it a lot due to all of the hype, but when I compared it side-by-side on a specific problem against Opus, I think that the solution Opus came to was cleaner and more accurate, although also more verbose.

Small sample size, but if Mythos/Fable was that much better, I feel like it should’ve given me an obviously better answer than Opus.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#63

Post by flyingcircus3 »

NewsaHackO wrote: post #58
paulatreides wrote: post #29

Earlier quoted context omitted.

it triggered for my.... zigbee home automation & home assistant logs, so my agent was constantly downgraded to Opus 4.8 even after I've changed it back. The false positives never stopped. "Fable" is also not even remotely as impressive as the benchmarks suggest, which is clear to me after using it pretty much non-stop for the past 24h.

It has to be sort of impressive, given that you tried so hard to use it instead of the regular Opus.

It isn't reasonable to infer that OP was claiming to have universally been unimpressed about every facet of Fable, and now some unrelated impressiveness is the evidence of their false claims.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#65

Post by paulatreides »

NewsaHackO wrote: post #58
paulatreides wrote: post #29

Earlier quoted context omitted.

it triggered for my.... zigbee home automation & home assistant logs, so my agent was constantly downgraded to Opus 4.8 even after I've changed it back. The false positives never stopped. "Fable" is also not even remotely as impressive as the benchmarks suggest, which is clear to me after using it pretty much non-stop for the past 24h.

It has to be sort of impressive, given that you tried so hard to use it instead of the regular Opus.

Some people made grandiose claims about its capabilities and I wanted to experience it myself.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#67

Post by reeece »

ofjcihen wrote: post #30
largbae wrote: post #15

Somewhere I read that malware is already starting to use nuclear and biological and cybersecurity terms in the code to trick Fable into shutting down. Even if this is just a hypothetical attack vector so far, it seems likely to work.

Some of the latest versions of Shai Hulud do this. Worked a contract recently where they were having AI check packages for obfuscation before admitting them into Artifactory but had vibed up the logic and it failed open. So in other words this worked because the terms caused the LLM checker to stall out and then the fail open logic resulted in the package being pulled down.

Seems like this?[1] Relevant bits below:

> This header appears designed for AI-mediated analysis, not for Node, Bun, or Python. It attempts to derail scanners or analyst copilots that feed the beginning of a file to a language model without clearly isolating the content as untrusted data. In weak pipelines, this can cause refusal behavior, prompt confusion, context pollution, or premature classification before the scanner reaches the actual malware.

> This is not a magical bypass against static detection. YARA rules, entropy checks, AST parsing, string extraction, deobfuscation, and behavioral rules still work. But it is a practical anti-analysis trick against naive LLM-first triage systems.

Would this affect many systems? You mention someone writing logic that fails open, but can't that be chalked up to just not following good security principles?

[1] - https://socket.dev/blog/mini-shai-hulud-miasma-and-hades-wor...

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#68

Post by throwawayffffas »

daedrdev wrote: post #5

The strangest part is that it won't just reject ML research, which I can understand, it will sabotage it silently by using a worse model without revealing it is doing so. It's just an insane level of deception and trust destruction for a company that at most is like 1 year ahead of its competition. Edit; to be clear they tell you when they degrade it for cybersecurity and bio

Can you imagine if AMD or Intel throttled your cpu if it detected you were working on "cybersecurity" or if you were designing a cpu?

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#70

Post by enraged_camel »

hparadiz wrote: post #66

I wonder how many millions they are wasting on putting up these guardrails when it's a completely useless exercise that is a speed bump at best.

If the guardrails were so useless, people wouldn't be complaining about them.
Post reply on HN