Suggestion: run "fix this code" on all of github before bad guys do.
I wonder what that would cost...
Perhaps less than the cost of not doing it.
Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
291–300 of 382 posts
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#292Earlier quoted context omitted.
I think it's a side effect of the Transformer architecture. The worldview where all input is equally trusted, and there's no concept of "the other", makes it hard to build effective guardrails where some input is trusted and other input is not trusted.
It seems like real robust guardrails would require some sort of "world model" or some other word to describe - AI that understands intent. Transformers are (to grossly summarize & I don't mean this as an insult) like auto-complete on steroids. So we have cat&mouse guardrails the way swear word filters and Chinese censorship work. People come up with increasingly complex miss-spelling, euphemisms & indirections to get…
>I suppose one solution would be to completely vet the training data such that nothing deemed "dangerous" exists in the data, which would be a huge effort.
I can see how this is tempting, but I suspect it would yield a naive model. I think the only way to improve this is to use a model that is legitimately advanced to support the concept of empathy, which may allow it to recognize others as being separate from itself, similar to how toddlers develop this sense (https://blog.lovevery.com/skills-stages/empathy/)
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#293I suspect we'll eventually hit a point where possession or usage of powerful open models will be criminalized.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#294>“The behavior described in the paper cannot meaningfully be fixed, and any attempt would only weaken the model for defense,” said Moussouris, who criticized the export control directive as hasty, heavy-handed, and misguided. This literally means the models are too dangerous to release, and yet he and they reached the opposite conclusion. A lot of people have been saying this repeatedly for a long time.
Mousssouris is not a "he".
ok
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#295Earlier quoted context omitted.
Amazon is a competitor to Anthropic.
Not really, they don't train their own (serious) models and they do a lot of hosting for Anthropic. iirc Anthropic trained a model on Trainium
I don't believe Anthropic trains on Trainium, only serves models on it.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#296Damn, I was hoping for another three words "make no mistakes"
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#297Earlier quoted context omitted.
> Why is requesting the model to show vulnerabilities is being blocked if fixing it not? This is how Anthropic describes Fable's behavior: "When Fable’s classifiers detect a request related to cybersecurity, biology and chemistry, or distillation, the response is automatically handled by Claude Opus 4.8 instead. Users will be informed whenever this occurs." So if you ask the model to "find security issues in this cod…
I wonder if opus 4.8 would also be able to fix the code too
It's not even clear if Anthropic care. If they genuinely think the user is trying to do something dangerous, then "OK, sure, but you're going to have to use Opus 4.8 for that" doesn't make a whole lot of sense.
Maybe this is just Anthropic pre-IPO marketing to try to convince people how much better Mythos is than Opus 4.8. There sure seemed to be a lot of shills out on release day talking about how it was a "step change" (exact phrase) in capability.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#298Earlier quoted context omitted.
I also have a 100% success rate jail breaking them by breaking the work down into small pieces and stripping all security related language. Smaller tasks, test engineering and normal programming language. Fable found a few bugs in my harness for me before they pulled it. I was testing it vs ChatGPT, Gemini, and Opus. It was doing well at bug hunting.
I took an assembler class in college. Before that, I'd been messing around with Core Wars and working my way through Peter Norton's book on assembly. So when an assignment came up, I used self modifying code to solve it. It was the shortest solution, it ran perfectly, and I submitted it. The next day, the professor caught me in the math department office (my dad worked there) and said she wanted to talk. Once we were…
Self modifying has some sneaky failure modes with modern CPUs. The modification can't be too close to it's execution or it's possible to execute the old version. And it's a nightmare to debug. I have no problem with a teacher prohibiting it. That being said, it should be understood because sometimes you don't get a choice. Borland Pascal 200mhz bug, an initializer in the library would crash. You either don't use that part of the library at all, or you put something ahead of it in the initialization that will find and overwrite the bug. (The root cause was the library calibrating the number of times to spin it's wheels to get a 1 millisecond delay. CPUs above 200mhz would cause this to produce a divide underflow.)
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#299Kind of highlights how ridiculous their notion of safety is in this case. By this measure, I guess making the model "safe" means making it play dumb and intentionally ignore security bugs that it notices in the code? And what will the eventual legality of this look like? "Yes, your honor, we allege that this AI system that was sold to us willingly and knowingly ignored a critical security vulnerability in our software system, thereby leading us to be hacked and causing our business to fold."
It's exactly the same problem as backdoors in crypto systems. Criminals will find the crypto that isn't broken and use it regardless (or make it for themselves), while the rest of us losers are stuck with the broken version that we're allowed to use.
On this issue of cyber security, it seems better if authorities just start acting like the cat is out of the bag instead of pretending like it isn't. ASI is basically here now, so what are we going to do about it? Let's not bother pretending otherwise.
On another note, I doubt this was anything other than a vindictive administration enacting revenge on a party that refused them. We all know the Trump admin's priorities.
Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
#300I think this brings out the cognitive dissonance around "safety" regarding cyber security: a) In order to make us safe, the LLM should help us find (and fix) the vulnerabilities in our own code. b) In order for us to be safe, the LLM should not find vulnerabilities in other people's code. I don't think this is resolvable in a way where both (a) and (b) win.
Exactly, it's a failure of Anthropic and others to understand cyber security. Finding security bugs in software is a good thing and not evil. It will lead to more secure software.
Defense and offense in cyber security are two sides of the same coin.