Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

techcrunch.com

Post reply on HN
Print view View original

491–500 of 570 posts

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#491

Post by mkl »

daedrdev wrote: post #5

The strangest part is that it won't just reject ML research, which I can understand, it will sabotage it silently by using a worse model without revealing it is doing so. It's just an insane level of deception and trust destruction for a company that at most is like 1 year ahead of its competition. Edit; to be clear they tell you when they degrade it for cybersecurity and bio

They walked that back, and now tell you they're downgrading the model: https://www.wired.com/story/anthropic-responds-to-backlash-o..., https://archive.is/yxYhU

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#492

Post by inglor_cz »

red-iron-pine wrote: post #475
inglor_cz wrote: post #438

Earlier quoted context omitted.

"Corporate America never backs down. It simply rallies and tries again later until people are too fatigued to care. " Frankly, that sounds excactly like Chat Control and similar recurring attempts to enact total surveillance here in the EU (Now shifted to heavy-handed age verification and various politicians touting bans on VPNs.) I don't want to abandon my continent of birth, though...

guess who is pushing for those anti-privacy laws? hint: they're publicly traded

I have encountered enough such people to know that the really heavy push is coming from the police and secret service circles. These are the workplaces that attract all the wannabe Stasi types.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#493

Post by conception »

saidnooneever wrote: post #479
brookst wrote: post #462

Earlier quoted context omitted.

I’ve never understood the “if I don’t enable bad behavior, someone else will, so I might as well enable bad behavior” argument. Can you elaborate? From where I sit it seems reasonable for Anthropic to not want their product used to create malware, even if they can’t solve the entire problem globally for every model. What’s wrong with that position? What should they do differently?

some context: its not about creating malware. this is already trivial and fully automated. its about finding exploits (which can be used to deploy malware), which is something both attackers and defenders benefit from. threat actors will find them anyway, LLM or not. They only need 1 so its much less work for them. defenders, they need to find them all. So for defenders, these models are more valuable than for attack…

I think your presumption is off. It’s not that threat actors won’t find them, but LLM tools rapidly increase the rate in which they can find them. It’s a bow and arrow versus a machine gun.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#495

Post by _whiteCaps_ »

micah94 wrote: post #141

I tried asking Fable 5 to identify the fungus in a picture I uploaded of one of my wife's plants. Apparently it thought I was trying to build a bioweapon. Opus answered it (yellow dog vomit fungus). Now I can spread the spores and take over the world!

It's my favourite slime mold. Just by the name you can instantly recognize it.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#496

Post by Roark66 »

This is a sign of things to come. First they sabotage your perfectly legal ML dicking around in your homelab.

Next they will be sabotaging anything that competes with them. Oh you are working on OpenCode codebase? Sorry Dave I can't allow you to do that.

How is this not illegal monopolistic practice? It is as if a maker of metalworking equipment put in the ToS you're not allowed to make your own spare parts using said equipment. Those fuckers should be banned from the EU and alternatives should get public funding.

(don't even tell me about these companies being a result of "free market". It is state level oligarchy it's clear to everyone. I don't see why we shouldn't counter them with public funding ourselves).

Just like Taiwan managed to take over advanced semiconductor production a well governed narrowly targeted state level funding will always win with oligarchs trying to do the same (they will always try to skim more and more). Of course I'm talking about things that require many dozens of billions in investment. Far too much for the free market to handle.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#497

Post by giancarlostoro »

matheusmoreira wrote: post #189
giancarlostoro wrote: post #89

Earlier quoted context omitted.

It's the dumbest thing ever, I sometimes edit code for custom AI related tooling I've built, so I run the risk of getting a worse model, and being billed for it? I'll stick to Opus, but at this point I'm about to just invest in fully local inference instead.

> at this point I'm about to just invest in fully local inference instead This is the best way forward long term. We won't have frontier performance, but at least the models will be aligned with us instead of refusing us or sabotaging us.

I think my biggest hangup is some models dont have big enough context windows, my sweet spot personally for Opus is having at least 400 to 600k tokens, if I can have a local model that can go up to that or slightly above 600k maybe 700k for some buffer, that would be perfect.

I've also debated having a frontier model for planning only, and then feeding plan to smaller offline models.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#498

Post by hparadiz »

make3 wrote: post #361
hparadiz wrote: post #90

Earlier quoted context omitted.

Well you see when a daddy H100 and a mommy H100 meet....

you don't get the model when you buy the data center, & no amount of running smaller models on a tiny 200k$ "cluster" (that's like one 4 gpus node, not even 8) will get you remotely close to Fable 5 level performance

Uh huh

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#499

Post by hootz »

Roark66 wrote: post #496

This is a sign of things to come. First they sabotage your perfectly legal ML dicking around in your homelab. Next they will be sabotaging anything that competes with them. Oh you are working on OpenCode codebase? Sorry Dave I can't allow you to do that. How is this not illegal monopolistic practice? It is as if a maker of metalworking equipment put in the ToS you're not allowed to make your own spare parts using sai…

It's how American companies have always worked, of course it is monopolistic practice, but those things are rarely illegal because the US absolutely loves their corps. Look at Google, Microsoft and the likes, this is the norm.

Re: Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable

#500

Post by worthless-trash »

conception wrote: post #493
saidnooneever wrote: post #479

Earlier quoted context omitted.

some context: its not about creating malware. this is already trivial and fully automated. its about finding exploits (which can be used to deploy malware), which is something both attackers and defenders benefit from. threat actors will find them anyway, LLM or not. They only need 1 so its much less work for them. defenders, they need to find them all. So for defenders, these models are more valuable than for attack…

I think your presumption is off. It’s not that threat actors won’t find them, but LLM tools rapidly increase the rate in which they can find them. It’s a bow and arrow versus a machine gun.

Right, but now we can't use the same tooling to find the flaw.

Its like a set of glasses that intentionally obscures the battlefield.

Post reply on HN