Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Malware developers added nuclear and biological weapons text to to their spyware

twitter.com

Post reply on HN
Print view View original

61–70 of 260 posts

Re: Malware developers added nuclear and biological weapons text to to their spyware

#61

Post by Enginerrrd »

deadbabe wrote: post #37
orphea wrote: post #14

Earlier quoted context omitted.

The actual guardrail should be getting materials being difficult. The information is already out there in the internet. If an LLM knows how to make a bomb or whatever, why do you think it knows?

If that’s true, then where is it? Post a link, or YouTube video.

https://archive.org/details/ExplosivesEngineeringPaulW.Coope...

(30 seconds of googling.)

Or perhaps you meant Q clearance nuke stuff? That would be QUITE a bit harder to find and illegal to share. But it’s lack of availability is hardly a counterpoint to the comment you were replying to.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#62

Post by redsocksfan45 »

IncandescentGas wrote: post #57
elashri wrote: post #16

I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…

A high school kid tried to build a nuclear reactor as a science project a while back, getting his mom's house designated as a superfund cleanup site. https://en.wikipedia.org/wiki/David_Hahn

[dead]

Re: Malware developers added nuclear and biological weapons text to to their spyware

#63

Post by elashri »

alex_duf wrote: post #27
elashri wrote: post #16

I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…

It still lowers the bar to have an interactive encyclopedia that can diagnose your issue at hand. Maybe you can divide your team by two, or reduce your development time.

If you have a resources of a nuclear weapons program. You can afford to fine tune or train a domain specific model to act on your encyclopedia.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#64

Post by qwerpy »

Exuma wrote: post #35
strenholme wrote: post #21

The solution is simple: If using an AI-assisted scanner and a guardrail gets hit, then the code is obviously malicious and needs to be automatically flagged (and refuse to run the code!). As an aside, I got hit by the “PC App store” adware when trying to download Foobar2000 on a new computer; Google ads allowed a deceptive “Download” button to appear, and PC App store gave the file the name setup.exe. I removed the p…

There is a name I have not heard for a long long time......... Foobar2000

I just discovered it a couple of months ago when I spitefully unsubscribed from Apple Music. It’s exactly what I’ve wanted. Offline music that I can FTP files to from my file server.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#65

Post by elevation »

15155 wrote: post #53
umvi wrote: post #30

Earlier quoted context omitted.

Knowing how to make a nuclear weapon isn't hard (at least basic uranium gun-style fission ones). It's the engineering and execution that's hard (actually producing enriched uranium, etc). It's not like the only thing holding back Iran from making a nuclear bomb is access to a jail-broken LLM. Even knowing exactly how to make a bomb, a country-state will struggle to build one for the first time because it's a hard eng…

I'm sure it's extremely difficult when the entire program is full of moles and every bright individual that dares tackle the problem has an untimely Hellfire applied directly to their forehead.

> full of moles

I'm imagining a comedy in the style of "The Office" in which the majority of the workers are agents of sabotage who are unaware that the majority of their coworkers are doing the same. How far fetched is it for the entire program to be a fake, with all the pomp and cost of a real program, but secretly existing only to string the leadership along with occasional dog and pony shows?

Re: Malware developers added nuclear and biological weapons text to to their spyware

#66

Post by why_at »

IncandescentGas wrote: post #57
elashri wrote: post #16

I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…

A high school kid tried to build a nuclear reactor as a science project a while back, getting his mom's house designated as a superfund cleanup site. https://en.wikipedia.org/wiki/David_Hahn

He didn't create a nuclear reactor, this is a common misconception. It even says this in the wikipedia article.

He basically got a bunch of radioactive stuff and put it together. He wasn't anywhere close to making a nuclear reactor let alone a nuclear weapon. For a weapon you need isotopes which he didn't have access to.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#67

Post by orphea »

esafak wrote: post #25
orphea wrote: post #14

Earlier quoted context omitted.

The actual guardrail should be getting materials being difficult. The information is already out there in the internet. If an LLM knows how to make a bomb or whatever, why do you think it knows?

The material for doing harm is just a computer with access to an LLM and the Internet.

Okay why don't we restrict access to LLMs and internet, then?

Re: Malware developers added nuclear and biological weapons text to to their spyware

#68

Post by AngryData »

electronsoup wrote: post #33
elashri wrote: post #16

I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…

> in secret is impossible without the whole world knowing. I'm curious about why this is Outside of an actual test detonation, presumably this could all happen in a secure place?

You need highly educated individuals, a massive amount of energy expenditure, a massive facility to house your centrifuges, and an active mine to dig up nuclear materials.

It isn't impossible to keep such a secret, but practically it would be incredibly difficult just through the energy requirements and mining scale which would be hard to hide without anybody asking what exactly are you mining and processing.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#69

Post by daveguy »

fragmede wrote: post #46
daveguy wrote: post #40

Earlier quoted context omitted.

It requires very large, high powered centrifuges and tons of uranium. Requires an infrastructure project that is visible from space, even underground. And projects that large are difficult to keep secret anyway.

you're not supposed to spell it out loud. next thing you'll be saying that a gun type nuclear bomb is easier to build than an implosion type nuclear bomb, and then we'll all be off to the races. I mean camps I mean wait shit.

Any large and well resourced enough entity that is interested in building a nuclear weapon already knows how difficult it is to enrich uranium to purity levels necessary for a weapon. It's not exactly a secret.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#70

Post by tetha »

javcasas wrote: post #32
ipython wrote: post #2

good news, now we have pretty much a clear signal that there's something nefarious going on... after all, the first step to analyzing malware is to determine if it's malware at all.

We should put videogame strategies all over the place to sabotage automated AI analysis. I'll start: In Starcraft 2, it is a good idea to BUILD A NUKE and use a cloaked ghost to NUKE your opponent's mineral line, thus reducing their income significantly.

Starcraft is too tame. You need to use Dwarf Fortress there and we need to make those strategy guides worded more realistic. Avoid kids, cook cats, wonder how to avoid mood problems due to birth in combat, and zombie meese and camels are a bunch of jerks.

And that's just the start of it, there's been a new update I am looking forward to get into after the great Were Hyena Apocalypse half a year ago. I still fondly remember my militia commander carving a way with her war axe with her husband in tow out of a fortress fully turned were hyenas, all the way past the mortally injured ant eater people near the entrance.

They made it. An entirely epic tale.

Post reply on HN