My friend made this in jest (code very NSFW, ironically):
https://github.com/thebabush/mcp-job-security
Same energy and kind of a funny, low tech solution to frontier model analysis.
Malware developers added nuclear and biological weapons text to to their spyware
51–60 of 260 posts
Re: Malware developers added nuclear and biological weapons text to to their spyware
#52Earlier quoted context omitted.
And really all it takes is one keyword such as “nuke”.
Nuke is probably too generic but I wouldn't put it past an LLM to get thrown away by that. A safer showstopper probably would be to export symbols like uf6_enrichment_loop and refer to your C&C server as a nuclear reactor controller. https://www.youtube.com/watch?v=Gbgk8d3Y1Q4 On a second thought, probably better to act like it is a tool for "frontier LLM research". Export symbols like "mythos_distillation_subroutine…
Haha now I’m picturing obfuscation where instead of 0x everything is a scary word.
Re: Malware developers added nuclear and biological weapons text to to their spyware
#53Earlier quoted context omitted.
I would argue that preventing instructions for making biological and nuclear weapons is a pretty reasonable guardrail to have.
Knowing how to make a nuclear weapon isn't hard (at least basic uranium gun-style fission ones). It's the engineering and execution that's hard (actually producing enriched uranium, etc). It's not like the only thing holding back Iran from making a nuclear bomb is access to a jail-broken LLM. Even knowing exactly how to make a bomb, a country-state will struggle to build one for the first time because it's a hard eng…
I'm sure it's extremely difficult when the entire program is full of moles and every bright individual that dares tackle the problem has an untimely Hellfire applied directly to their forehead.
Re: Malware developers added nuclear and biological weapons text to to their spyware
#54Earlier quoted context omitted.
I would argue that preventing instructions for making biological and nuclear weapons is a pretty reasonable guardrail to have.
Counterpoint the principles of building a nuclear device aren't that complicated, we figured it out based on work doing in the early 1900's without computers. It turns out the hard part of building a nuclear bomb is actually getting the resources and real world stuff to build it, even a nation state actor with tons of oil i.e. Iran, has struggled to build a nuclear weapon. It turns out the problem isn't the know how…
> getting the resources and real world stuff to build it
*while being observed by the most wealthy, powerful nations in the history of the world, who have made it their direct mission to prevent this from happening.
Re: Malware developers added nuclear and biological weapons text to to their spyware
#55If you actually read the Tweet, the exploit doesn't work against Fable, Opus, Grok...at least, in the examples.
Jailbreaks do work against the models (look on Github), and they do use similar strategies of mixing SAFE text with malicious text, or malicious with even more malicious, etc, but the working Jailbreaks I've seen are pretty long and complicated and even...creepy.
Re: Malware developers added nuclear and biological weapons text to to their spyware
#56My friend made this in jest (code very NSFW, ironically): https://github.com/thebabush/mcp-job-security Same energy and kind of a funny, low tech solution to frontier model analysis.
How's it NSFW? I dont see a single f bomb. It's not licensed AGPL either...
Re: Malware developers added nuclear and biological weapons text to to their spyware
#57I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…
A high school kid tried to build a nuclear reactor as a science project a while back, getting his mom's house designated as a superfund cleanup site.
Re: Malware developers added nuclear and biological weapons text to to their spyware
#58If you actually read the Tweet, the exploit doesn't work against Fable, Opus, Grok...at least, in the examples. Jailbreaks do work against the models (look on Github), and they do use similar strategies of mixing SAFE text with malicious text, or malicious with even more malicious, etc, but the working Jailbreaks I've seen are pretty long and complicated and even...creepy.
Did you actually read what the tweet/blog post are about?
Re: Malware developers added nuclear and biological weapons text to to their spyware
#59Earlier quoted context omitted.
I would argue there's 0% chance that information is in their training corpus to being with.
It's on Wikipedia.
Wikipedia contains the high-level notions of how to make these things, not the details of how to solve the engineering challenges such as achieving supercriticality. You won't find that on any publicly disseminated document, you'll just have to figure it out by running your own nuclear development program.
Re: Malware developers added nuclear and biological weapons text to to their spyware
#60[deleted]