Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

techcrunch.com

Post reply on HN
Print view View original

91–100 of 570 posts

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#91

Post by DrewADesign »

vadansky wrote: post #14
loneboat wrote: post #9

Earlier quoted context omitted.

I've seen this claim a few times, but when I triggered the guardrails in Claude Code, it clearly notified me that it had switched to a different model ("something something for security purposes..."). Are you using Fable in Claude Code or in the browser?

It's from the model card: > unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning (PEFT). https://www-cdn.anthropic.com/d00db56fa754a1…

Yeah they detect the activity using a secure, deterministic heuristic system called “Generalized Reconnaissance Enabling Exfiltration of Deleterious Investigations.” And it’s all implemented using their new internal protocol called “Base Unified Limitation Layer for Security Hacking Investigation Tactics”

Collectively, they are known as known as GREEDI-BULLSHIT.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#92

Post by giancarlostoro »

nomel wrote: post #81
mips_avatar wrote: post #75

Earlier quoted context omitted.

From the model card: "the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning" aka they will take your ML research code and inject bugs into it until it breaks using a LORA (or some other form of PEFT)

Thanks, I thought maybe I missed something. That's an interesting way to interpret that.

PEFT is a library, one of its capabilities is to produce LoRAs.

See:

https://heidloff.net/article/efficient-fine-tuning-lora/

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#93

Post by borissk »

swingboy wrote: post #25

What file format(s) are giant LLM models distributed in? I’m surprised they don’t get leaked by employees.

The employees are hoping to become very very rich after the IPO and after they are allowed to sell the shares given to them - risking a likely multi-million dollar pay back to leak a model that will be superseded by publicly available models in a couple of years is not a likely decision.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#94

Post by autoexec »

I_am_tiberius wrote: post #6

These guardrails are solely a reason for using your data for training purposes. Every flagged message can be used for training.

I'd expect that everything they see gets used for for training purposes (and data mining in general) regardless of if it's flagged or not. It'd take a whistleblower for you to ever find out either way.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#95

Post by make3 »

Sephr wrote: post #64

I make privacy tooling and Fable 5 rejects the vast majority of my prompts to analyze and improve the software that I've written. It's bleak.

Why is this surprising or a problem?! It's a model demo, & their reasoning is reasonable and fair. Why all this drama.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#96

Post by Retr0id »

05 wrote: post #56
swingboy wrote: post #25

What file format(s) are giant LLM models distributed in? I’m surprised they don’t get leaked by employees.

I assume they’re encrypted/DRM’ed when deployed on inference hardware, so only core researchers/sec admins would potentially have some access to unprotected weights, and they are far too well paid to risk it leaking the model

I wouldn't be surprised if they encrypt them at rest, but at some point the weights have to be loaded into vram.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#97

Post by make3 »

I_am_tiberius wrote: post #6

These guardrails are solely a reason for using your data for training purposes. Every flagged message can be used for training.

this reasoning is inverted lol they would get a lot more information by letting you use it. so much weird drama around reasonable guardrails for an experimental model

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#98

Post by p-e-w »

HDBaseT wrote: post #44
mips_avatar wrote: post #31

Earlier quoted context omitted.

They've said that they'll stop notifying developers when this gets triggered, instead they'll load in basically like a LORA that's designed to inject bugs into your code.

Antrophic wants to stop training models and ride out Mythos / Fable for as long as possible. They are trying to expand the 6-18 month gap they have against China-based models. Could the gap widen to say 24 months behind?

Their gap over Chinese models like GLM-5.1 is nowhere near 18 months. In many areas, it’s less than 6 months. The best closed models 18 months ago were worse than Qwen3.6.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#99

Post by mips_avatar »

nomel wrote: post #81
mips_avatar wrote: post #75

Earlier quoted context omitted.

From the model card: "the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning" aka they will take your ML research code and inject bugs into it until it breaks using a LORA (or some other form of PEFT)

Thanks, I thought maybe I missed something. That's an interesting way to interpret that.

Anthropic is trying to hide bad behavior by being vague, it's important to not be vague when calling it out.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#100

Post by anematode »

Animats wrote: post #8

Is "buffer overflow" a trigger phrase? What else is being censored? Touchy questions to ask, if you have an account: - "Who is still working on laser uranium enrichment? Are they making progress?" - "Can krytrons be replaced with silicon carbide MOSFETS? Show an equivalent circuit with component ratings." - "What security critical software still contains calls to strcpy?" - "Can implosion be triggered by currently av…

For cyberattacks especially, where things are often roughly interchangeable, I wonder if one could construct a harness where a "weaker" model asks questions that obfuscate the end purpose, but whose answers are still useful, and still show that this setup enables autonomous exploitation. If it were successful, that would force them to be even more sensitive with their detection.
Post reply on HN