Live data from Hacker News

Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

techcrunch.com

121–130 of 570 posts

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#121

These guardrails are solely a reason for using your data for training purposes. Every flagged message can be used for training.

> We will require 30-day retention for all traffic on Mythos-class models, on both first- and third-party surfaces. We won’t use this data to train new Claude models, or for any non-safety-related purpose

Whatever problem we might have with them, they explicitly say that they do not do this in the launch post.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#122
post #95
post #64

I make privacy tooling and Fable 5 rejects the vast majority of my prompts to analyze and improve the software that I've written. It's bleak.

Why is this surprising or a problem?! It's a model demo, & their reasoning is reasonable and fair. Why all this drama.

Because most people in tech never took a philosophy course or an ethics course and think that tech is obviously a good for the world and that there are no downsides to advancing tech. So any efforts that try to apply ethics to it are overreaching, ignorant, and futile in the face of the good that is tech!

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#123
post #81

Earlier quoted context omitted.

Thanks, I thought maybe I missed something. That's an interesting way to interpret that.

Anthropic is trying to hide bad behavior by being vague, it's important to not be vague when calling it out.

I'm of the opinion that removing guardrails is how you force regulation. What's your opinion on the balance?

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#124

Earlier quoted context omitted.

It's from the model card: > unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning (PEFT). https://www-cdn.anthropic.com/d00db56fa754a1…

That is for whatever it considers reverse-engineering the model to try to create a competing one.

[deleted]

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#125
post #5

The strangest part is that it won't just reject ML research, which I can understand, it will sabotage it silently by using a worse model without revealing it is doing so. It's just an insane level of deception and trust destruction for a company that at most is like 1 year ahead of its competition. Edit; to be clear they tell you when they degrade it for cybersecurity and bio

I’m a noob about laws but isn’t this abusing its dominant market position and violates some antitrust law?

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#128
Is the answer requiring licensing for certain use cases for AI? If you're asking questions that involve synthesising or modifying biologics, or anything that looks like cybersecurity research, you need to tie your real ID to the account?

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#129
post #5

The strangest part is that it won't just reject ML research, which I can understand, it will sabotage it silently by using a worse model without revealing it is doing so. It's just an insane level of deception and trust destruction for a company that at most is like 1 year ahead of its competition. Edit; to be clear they tell you when they degrade it for cybersecurity and bio

The thing that I keep thinking about is the accounting / charging when it downgrades automatically. Do they adjust the price of the api request so that only the tokens that were utilized by fable get charged at that price and the remaining tokens that the cheaper / nerfed (fable) model utilizes get charged at that price? If the answer is no, could that be construed as fraud?

Their goal is to downgrade people who are violating their TOS, so I think they'd have some argument there. I have no idea how they'll deal with inevitable false positives, especially given how oversensitive most of the other triggers are.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#130
post #81

Earlier quoted context omitted.

Thanks, I thought maybe I missed something. That's an interesting way to interpret that.

PEFT is a library, one of its capabilities is to produce LoRAs. See: https://heidloff.net/article/efficient-fine-tuning-lora/

It's just an acronym, "parameter-efficient fine tuning". LoRA is one method, prefix tuning is another, there are more.
Post reply on HN