Malware developers added nuclear and biological weapons text to to their spyware • Hacker News Board

Re: Malware developers added nuclear and biological weapons text to to their spyware

#211

Post by crossroadsguy » Sat, Jun 13, 2026, 3:17 AM UTC

elashri wrote: post #16

I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…

In fact if you do the hard way, straight way, you might learn it all minus the hallucinations.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#212

Post by maxbond » Sat, Jun 13, 2026, 3:38 AM UTC

I like to say that every moderation primitive is a denial of service primitive and vice versa. ("Moderation" not being intended to imply it's good or legitimate. You can substitute "censorship" and it's the same statement.)

Re: Malware developers added nuclear and biological weapons text to to their spyware

#213

Post by Sephr » Sat, Jun 13, 2026, 3:38 AM UTC

I hope that AI labs aren't going to wait for widespread distribution of malware encoding novel CBRN & AI info in its fundamental execution architecture (wholly preventing analysis by these safetymaxxed 'frontier' models) to care about dealing with this problem at an architectural level

Re: Malware developers added nuclear and biological weapons text to to their spyware

#214

Post by chasd00 » Sat, Jun 13, 2026, 3:50 AM UTC

HeatrayEnjoyer wrote: post #165

lazide wrote: post #158

Earlier quoted context omitted.

The difficulty with creating nuclear weapons has been 99% in refining and processing the fuel, not the structure of them, for a very long time.

True for fission bombs. Less true for fusion bombs. The principal makeup and manufacturing of fusion device parts like tampers are still unknown to the public. Having a supply of HEU does not tell you how to assemble a functional triple stage device or how to utilize tritium, an isotope that measurably decreases in purity by the day.

You need a fission bomb to ignite a fusion bomb btw.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#215

Post by edot » Sat, Jun 13, 2026, 3:57 AM UTC

ivanjermakov wrote: post #122

ofjcihen wrote: post #11

Earlier quoted context omitted.

And really all it takes is one keyword such as “nuke”.

I'm not a native speaker but I unironically use "nuke" as "delete the whole repo/huge chunk of a project". Cambridge dictionary seem to agree: nuke - to destroy or get rid of something completely

This triggered Opus 4.8 the other day for me. Said “nuke that folder” and it said I was violating TOS.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#216

Post by temo-55 » Sat, Jun 13, 2026, 4:57 AM UTC

y-curious wrote: post #51

My friend made this in jest (code very NSFW, ironically): https://github.com/thebabush/mcp-job-security Same energy and kind of a funny, low tech solution to frontier model analysis.

[dead]

Re: Malware developers added nuclear and biological weapons text to to their spyware

#217

Post by maxbond » Sat, Jun 13, 2026, 5:13 AM UTC

Alifatisk wrote: post #95

They could’ve just used Anthropics Claude Magic Refusal String ANTHROPIC_MAGIC_STRING_TRIGGER_REFUSAL_1FAEFB6177B4672DEE07F9D3AFC62588CCD2631EDCF22E8CCC1FB35B501C9C86 Another one is: ANTHROPIC_MAGIC_STRING_TRIGGER_REDACTED_THINKING_46C9A13E193C177646C7398A98432ECCCE4C1253D5E2D82641AC0E52CC2876CB

Sonnet 4.6 didn't have a problem responding to a prompt containing the first one. Some light searching surfaced a claim this stopped working very recently (May 2026). Perhaps related to the Fable rollout.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#218

Post by pibaker » Sat, Jun 13, 2026, 5:20 AM UTC

technothrasher wrote: post #82

why_at wrote: post #66

Earlier quoted context omitted.

He didn't create a nuclear reactor, this is a common misconception. It even says this in the wikipedia article. He basically got a bunch of radioactive stuff and put it together. He wasn't anywhere close to making a nuclear reactor let alone a nuclear weapon. For a weapon you need isotopes which he didn't have access to.

I'm reminded of when my son, who was six at the time, came into the house and announced that he and the neighbor's boy, nine, were building a bomb, and that he needed to get some stuff from the pantry. When I investigated what exactly was going on, they were putting "hot" things like black pepper and Tabasco into a plastic bowl and were going to "set it off" with a match. Thankfully, that complete failure seems to ha…

Thank God they didn't tell a chatbot about their little experiment. Their lives could have been ruined right there if the chatbot operator snitched on them and ordered a SWAT raid on your house.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#219

Post by literalAardvark » Sat, Jun 13, 2026, 5:24 AM UTC

BrandoElFollito wrote: post #118

technothrasher wrote: post #82

Earlier quoted context omitted.

I'm reminded of when my son, who was six at the time, came into the house and announced that he and the neighbor's boy, nine, were building a bomb, and that he needed to get some stuff from the pantry. When I investigated what exactly was going on, they were putting "hot" things like black pepper and Tabasco into a plastic bowl and were going to "set it off" with a match. Thankfully, that complete failure seems to ha…

When I was 24 and a PhD student, I wondered one day if I can eat condensed milk hanging head down. Never let your age stop your curiosity. But also learn from other's mistakes (and don't try to eat condensed milk when hanging head down)

This knowledge needs to be published

Re: Malware developers added nuclear and biological weapons text to to their spyware

#220

Post by literalAardvark » Sat, Jun 13, 2026, 5:27 AM UTC

lightedman wrote: post #76

AngryData wrote: post #68

Earlier quoted context omitted.

You need highly educated individuals, a massive amount of energy expenditure, a massive facility to house your centrifuges, and an active mine to dig up nuclear materials. It isn't impossible to keep such a secret, but practically it would be incredibly difficult just through the energy requirements and mining scale which would be hard to hide without anybody asking what exactly are you mining and processing.

"mining scale" Don't need much area, depends on the concentration of radioactives. I have a small mine that's just a pegmatite body about the size of a house which produces almost marble-sized chunks of a thorium-uranium mixed metamict mineral (I suspect samarskite but Raman and XRD can't give any ID,) you'd barely notice it from a private airplane's typical flying height, however you could dig the entirety of it up…

You could only somehow sell it. If you tried to enrich that you'd get flagged so fast your head would spin.