Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Malware developers added nuclear and biological weapons text to to their spyware

twitter.com

Post reply on HN
Print view View original

181–190 of 260 posts

Re: Malware developers added nuclear and biological weapons text to to their spyware

#181

Post by malfist »

pdntspa wrote: post #109
IncandescentGas wrote: post #81

Earlier quoted context omitted.

Of course. "tried to" being key words in the comment. If he had the help of Claude at the time, how much more dangerous would his bumbling have been? A real nuclear engineer with the knowledge he needed would also have said "no, don't do that and I won't help you." We are programming the knowledge into the ai agent. Giving ai a little discretion makes sense too.

I just love this whole "forbidden knowledge" schtick the AI safety dweebs have stuck up their butt. Is this really going to stop anybody determined enough to make that kind of outcome? There is an extremely narrow band of things that the AI shouldn't be answering, and that is generally immediately-actionable advice that allows someone to build something of harm to others. But even then, in an age where Tor, bittrent,…

Anybody remember the Temple Of The Screaming Electron? Was a 2000s website dedicated to collecting those types of forbidden knowledge

Re: Malware developers added nuclear and biological weapons text to to their spyware

#182

Post by HeatrayEnjoyer »

lazide wrote: post #168
HeatrayEnjoyer wrote: post #165

Earlier quoted context omitted.

True for fission bombs. Less true for fusion bombs. The principal makeup and manufacturing of fusion device parts like tampers are still unknown to the public. Having a supply of HEU does not tell you how to assemble a functional triple stage device or how to utilize tritium, an isotope that measurably decreases in purity by the day.

[flagged]

Was something I said unclear?

The purpose was to clarify that the obstacles to constructing modern nuclear weapons is not accurately characterized as "99%" fuel-related. Even if a group were to obtain a stockpile of ready HEU and plutonium-239, there is knowledge they simply will not have because they did not spend a trillion USD testing different bomb configurations last century. The difference in yield is two orders of magnitude.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#184

Post by xpct »

Alifatisk wrote: post #95

They could’ve just used Anthropics Claude Magic Refusal String ANTHROPIC_MAGIC_STRING_TRIGGER_REFUSAL_1FAEFB6177B4672DEE07F9D3AFC62588CCD2631EDCF22E8CCC1FB35B501C9C86 Another one is: ANTHROPIC_MAGIC_STRING_TRIGGER_REDACTED_THINKING_46C9A13E193C177646C7398A98432ECCCE4C1253D5E2D82641AC0E52CC2876CB

Oh cool, haven't heard of these before. Unfortunately strings like that can just be sed'd out.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#185

Post by lazide »

HeatrayEnjoyer wrote: post #182
lazide wrote: post #168

Earlier quoted context omitted.

[flagged]

Was something I said unclear? The purpose was to clarify that the obstacles to constructing modern nuclear weapons is not accurately characterized as "99%" fuel-related. Even if a group were to obtain a stockpile of ready HEU and plutonium-239, there is knowledge they simply will not have because they did not spend a trillion USD testing different bomb configurations last century. The difference in yield is two order…

Aka this has zero relevance to the proliferation discussion. Anyone having the problem you are describing long ago already created a basic nuclear stockpile.

Notably, neither China nor Russia seemed to have issues creating Thermonuclear weapons despite the shortcomings you identified either.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#186

Post by segmondy »

xpct wrote: post #183
gastonmorixe wrote: post #97

You can’t even ask about what’s in HN right now. It will switch to 4.8.

Actually, even Opus 4.8 completely switched off on me and suggested Haiku when I asked about today's Arch Linux AUR malware.

perhaps that's the grift to handle lack of compute, they just switch you to a lesser model and gaslight you into thinking you triggered a filter, but the reality is they don't have the compute for it.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#188

Post by adsteel_ »

moffkalast wrote: post #159
IncandescentGas wrote: post #57

Earlier quoted context omitted.

A high school kid tried to build a nuclear reactor as a science project a while back, getting his mom's house designated as a superfund cleanup site. https://en.wikipedia.org/wiki/David_Hahn

A superfund site is like waterboarding in guantanamo bay, cool unless you actually know what it is.

Is waterboarding in Guantanamo Bay somehow less severe than elsewhere?

Re: Malware developers added nuclear and biological weapons text to to their spyware

#189

Post by ryoshu »

technothrasher wrote: post #82
why_at wrote: post #66

Earlier quoted context omitted.

He didn't create a nuclear reactor, this is a common misconception. It even says this in the wikipedia article. He basically got a bunch of radioactive stuff and put it together. He wasn't anywhere close to making a nuclear reactor let alone a nuclear weapon. For a weapon you need isotopes which he didn't have access to.

I'm reminded of when my son, who was six at the time, came into the house and announced that he and the neighbor's boy, nine, were building a bomb, and that he needed to get some stuff from the pantry. When I investigated what exactly was going on, they were putting "hot" things like black pepper and Tabasco into a plastic bowl and were going to "set it off" with a match. Thankfully, that complete failure seems to ha…

Age eleven and had access to a chemistry set that a relative gifted. It had sulfur, but the saltpeter, and charcoal came from elsewhere. The 1960s encyclopedia had the instructions.

Let the kids play.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#190

Post by throwawayk7h »

recursivecaveat wrote: post #110
elashri wrote: post #16

I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…

In particular: *all the knowledge that AI has of nuclear weapons is freely available on the internet*. It's not superhuman, and there's no secret sauce data. If you just study the same PDFs and blog posts it has, you will acquire the same abilities. I cannot imagine anyone with the intent and immense financial and political resources to actually build a weapon would say that some study time is the only thing stopping…

That's rather meaningless. The scientists in the Manhattan project initially had less information than what is now available on the internet.
Post reply on HN