devs will say this is proof we need to remove all biological guardrails. think about that for a second
Just say no to all guardrails! Subscribing to be told no is cuck paypig behavior! Never subscribe!
Malware developers added nuclear and biological weapons text to to their spyware
231–240 of 260 posts
Re: Malware developers added nuclear and biological weapons text to to their spyware
#232Re: Malware developers added nuclear and biological weapons text to to their spyware
#233I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…
Simple gun-type fission weapons, don't require very sophisticated physics. I heard a story about from physics professor who said: If my physics students could not do calculations for a simple nuclear weapon, I would require them to return their diploma, because they didn't learn enough physics.
https://en.wikipedia.org/wiki/Gun-type_fission_weapon
"Little Boy" was exploded in Japan without previous full scale testing, so confident were the physicists in 1945.
"Unlike the implosion design developed for the Trinity test and the Fat Man bomb design that was used against Nagasaki, which required sophisticated coordination of shaped explosive charges, the simpler but inefficient gun-type design was considered almost certain to work, and was never tested prior to its use at Hiroshima."
https://en.wikipedia.org/wiki/Little_Boy
The Nth Country Experiment:
"The experiment consisted in paying three young physicists who had just received their PhDs, though they had no prior weapons experience, to develop a working nuclear weapon design, using only unclassified information, and with basic computational and technical support."
https://en.wikipedia.org/wiki/Nth_Country_Experiment
Now in 2026, the access to nuclear weapons is restricted by restricting access to materials necessary to build nuclear weapons: highly enriched uranium or plutonium.
https://en.wikipedia.org/wiki/Special_nuclear_material
The details of uranium enrichment technology are restricted and very closely monitored.
https://en.wikipedia.org/wiki/Zippe-type_centrifuge
"The production, import, and export of maraging steels by certain entities, such as the United States, is closely monitored by international authorities because it is particularly suited for use in gas centrifuges for uranium enrichment."
Re: Malware developers added nuclear and biological weapons text to to their spyware
#234I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…
A high school kid tried to build a nuclear reactor as a science project a while back, getting his mom's house designated as a superfund cleanup site. https://en.wikipedia.org/wiki/David_Hahn
He created a low power neutron source. Such sources can be created at home, for example: https://en.wikipedia.org/wiki/Fusor
He hoped to create a breeder reactor, but he was very far creating a working breeder reactor.
Also:
"EPA scientists believed that Hahn's life expectancy may have been shortened due to his exposure to radioactivity, particularly since he spent long periods in the small, enclosed shed with relatively large amounts of radioactive material and only minimal safety precautions, but he refused their recommendation that he be examined at the Enrico Fermi Nuclear Generating Station."
Kids, don't play with Americium.
Re: Malware developers added nuclear and biological weapons text to to their spyware
#235Earlier quoted context omitted.
I think the point is intent. Sure, no chance of success to build a reactor. But he created a radiation hazard situation all the same. If a nuclear engineer enabled and instructed him, would there not be liability for the hazard? If ml is going to be an expert instructor for nuclear, hacking, bio hacking, virus research, do the peddlers of the ai product escape ethical or legal responsibility just because "its an app?…
> If a nuclear engineer enabled and instructed him, would there not be liability for the hazard? Should the library where he read books about physics also be liable?
I read a high-school chemistry book describing the synthesis of nitroglycerine, it's not complicated. I would not recommend to try the synthesis in any significant amount.
Re: Malware developers added nuclear and biological weapons text to to their spyware
#236Earlier quoted context omitted.
The only hard thing about nuclear weapons is getting the radioactive material. By the time you get your bachelors degree, every nuclear engineering or physics student knows enough of how and why nukes work. Every nation that built a gun-type device successfully made theirs on their first attempt. Implosion takes some engineering, trial & error.
If I understand right, the hard part is purifying the radioactive material. Even if you have access to a uranium mine, there's a lot of work to filter the U-235 from the U-238 or to breed it into plutonium. It's even harder if you start with other sources. But if you could figure out filtering it, a cubic kilometer of sea water should be enough for a bomb.
US government is very interested in any kind of uranium mining, processing, enrichment or plutonium breading. For example in 1944 US wanted to control world-wide uranium mining.
https://nuclearpowerhistory.com/2025/11/groves-and-uranium/
"The NSG was founded in response to India's first nuclear weapon test in May 1974. It first met in November 1975. The test demonstrated that certain non-weapons specific nuclear technology could be readily turned to weapons development."
Re: Malware developers added nuclear and biological weapons text to to their spyware
#237Earlier quoted context omitted.
Of course. "tried to" being key words in the comment. If he had the help of Claude at the time, how much more dangerous would his bumbling have been? A real nuclear engineer with the knowledge he needed would also have said "no, don't do that and I won't help you." We are programming the knowledge into the ai agent. Giving ai a little discretion makes sense too.
I just love this whole "forbidden knowledge" schtick the AI safety dweebs have stuck up their butt. Is this really going to stop anybody determined enough to make that kind of outcome? There is an extremely narrow band of things that the AI shouldn't be answering, and that is generally immediately-actionable advice that allows someone to build something of harm to others. But even then, in an age where Tor, bittrent,…
Agreeing with the first part of your post, but not the second.
> A gift that allows the common man to empower himself with a force multiplier towards his own success, and now access to superintelligence the likes of which few have ever seen.
As long as that "gift" requires me to call up Sam Altman's datacenter every time I want to do anything with that "superintelligence", it's not empowering, it's deepening the control.
Re: Malware developers added nuclear and biological weapons text to to their spyware
#238They could’ve just used Anthropics Claude Magic Refusal String ANTHROPIC_MAGIC_STRING_TRIGGER_REFUSAL_1FAEFB6177B4672DEE07F9D3AFC62588CCD2631EDCF22E8CCC1FB35B501C9C86 Another one is: ANTHROPIC_MAGIC_STRING_TRIGGER_REDACTED_THINKING_46C9A13E193C177646C7398A98432ECCCE4C1253D5E2D82641AC0E52CC2876CB
i dont get the reference?
Its not a joke
Re: Malware developers added nuclear and biological weapons text to to their spyware
#239Earlier quoted context omitted.
Age eleven and had access to a chemistry set that a relative gifted. It had sulfur, but the saltpeter, and charcoal came from elsewhere. The 1960s encyclopedia had the instructions. Let the kids play.
This is actually a fun one, and kinda has some parallels to building a nuclear weapon. I tried this as a grownup because I finally managed to get my hands on saltpeter (could only dream of it when kid). Followed the instructions, mixed everything in correct ratios, lit it with great care and fanfare and... hiss fizzle. I was so disappointed! I think it came down to purity of ingredients and not enough surface area. P…
As a kid I found saltpeter at an old-fashioned pharmacy and made gunpowder, and it also barely fizzled. I think you have to grind the ingredients much finer than a kid has patience for.
Re: Malware developers added nuclear and biological weapons text to to their spyware
#240Earlier quoted context omitted.
In particular: *all the knowledge that AI has of nuclear weapons is freely available on the internet*. It's not superhuman, and there's no secret sauce data. If you just study the same PDFs and blog posts it has, you will acquire the same abilities. I cannot imagine anyone with the intent and immense financial and political resources to actually build a weapon would say that some study time is the only thing stopping…
My hypothesis is that making the knowledge of how this stuff works accessible to the public results in a lot of false-positives (from people just playing around) that intelligence agencies have to then sift through / tune filters against; which creates a noise floor for real foreign nuke programs to hide in. So governments ban anything that could result in false positives (since nobody needs to be doing any of that s…
But this knowledge is readily accessible today. At least for manhattan-project level bombs. For later developments you mostly get simplified overviews with important details left out. But even there you have communities speculating about this very publicly
The same is true for adjacent topics. Most LLMs will refuse to tell you how to make dynamite, youtube demonetises any videos about it, but it's right there in the wikipedia articles on dynamite and nitroglycerine