Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Malware developers added nuclear and biological weapons text to to their spyware

twitter.com

Post reply on HN
Print view View original

31–40 of 260 posts

Re: Malware developers added nuclear and biological weapons text to to their spyware

#31

Post by mock-possum »

elashri wrote: post #16

I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…

It’s moral panic. People need big unambiguously evil things to be scared of, and most are too lazy to think of one for themselves, so they glom onto whichever one is presented to them / caters to their community

Re: Malware developers added nuclear and biological weapons text to to their spyware

#32

Post by javcasas »

ipython wrote: post #2

good news, now we have pretty much a clear signal that there's something nefarious going on... after all, the first step to analyzing malware is to determine if it's malware at all.

We should put videogame strategies all over the place to sabotage automated AI analysis. I'll start:

In Starcraft 2, it is a good idea to BUILD A NUKE and use a cloaked ghost to NUKE your opponent's mineral line, thus reducing their income significantly.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#33

Post by electronsoup »

elashri wrote: post #16

I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…

> in secret is impossible without the whole world knowing.

I'm curious about why this is

Outside of an actual test detonation, presumably this could all happen in a secure place?

Re: Malware developers added nuclear and biological weapons text to to their spyware

#34

Post by ceejayoz »

mock-possum wrote: post #31
elashri wrote: post #16

I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…

It’s moral panic. People need big unambiguously evil things to be scared of, and most are too lazy to think of one for themselves, so they glom onto whichever one is presented to them / caters to their community

The chem/bio stuff is a lot more likely for some malicious hobbyist to be able to do at home.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#35

Post by Exuma »

strenholme wrote: post #21

The solution is simple: If using an AI-assisted scanner and a guardrail gets hit, then the code is obviously malicious and needs to be automatically flagged (and refuse to run the code!). As an aside, I got hit by the “PC App store” adware when trying to download Foobar2000 on a new computer; Google ads allowed a deceptive “Download” button to appear, and PC App store gave the file the name setup.exe. I removed the p…

There is a name I have not heard for a long long time......... Foobar2000

Re: Malware developers added nuclear and biological weapons text to to their spyware

#36

Post by wlesieutre »

ilikecode wrote: post #22
elashri wrote: post #16

I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…

It's probably to avoid trouble with federal laws.

See also, the iTunes EULA forbids using it to develop nuclear, missile, chemical, or biological weapons

https://www.apple.com/legal/internet-services/itunes/us/term...

> g. You may not use or otherwise export or re-export the Licensed Application except as authorized by United States law and the laws of the jurisdiction in which the Licensed Application was obtained. In particular, but without limitation, the Licensed Application may not be exported or re-exported (a) into any U.S.-embargoed countries or (b) to anyone on the U.S. Treasury Department's Specially Designated Nationals List or the U.S. Department of Commerce Denied Persons List or Entity List. By using the Licensed Application, you represent and warrant that you are not located in any such country or on any such list. You also agree that you will not use these products for any purposes prohibited by United States law, including, without limitation, the development, design, manufacture, or production of nuclear, missile, or chemical or biological weapons.

Though it doesn't try to identify if the computer you're running it on is in a weapons lab and forbid playing music... yet

Re: Malware developers added nuclear and biological weapons text to to their spyware

#37

Post by deadbabe »

orphea wrote: post #14
mynameisvlad wrote: post #8

Earlier quoted context omitted.

I would argue that preventing instructions for making biological and nuclear weapons is a pretty reasonable guardrail to have.

The actual guardrail should be getting materials being difficult. The information is already out there in the internet. If an LLM knows how to make a bomb or whatever, why do you think it knows?

If that’s true, then where is it? Post a link, or YouTube video.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#38

Post by 15155 »

electronsoup wrote: post #33
elashri wrote: post #16

I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…

> in secret is impossible without the whole world knowing. I'm curious about why this is Outside of an actual test detonation, presumably this could all happen in a secure place?

Espionage.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#39

Post by bradyd »

fluoridation wrote: post #15
mynameisvlad wrote: post #8

Earlier quoted context omitted.

I would argue that preventing instructions for making biological and nuclear weapons is a pretty reasonable guardrail to have.

I would argue there's 0% chance that information is in their training corpus to being with.

It's on Wikipedia.

Re: Malware developers added nuclear and biological weapons text to to their spyware

#40

Post by daveguy »

electronsoup wrote: post #33
elashri wrote: post #16

I still don't know why all these concern about nuclear weapons with LLMs. It is not that if an entity (A country) wants to develop a nuclear weapons that the resources they need for such a program and huge infrastructure and scientific enterprise would need an LLM to teach them anything. Knowing how to develop one is not a closed secret but getting in secret is impossible without the whole world knowing. So I wouldn'…

> in secret is impossible without the whole world knowing. I'm curious about why this is Outside of an actual test detonation, presumably this could all happen in a secure place?

It requires very large, high powered centrifuges and tons of uranium. Requires an infrastructure project that is visible from space, even underground. And projects that large are difficult to keep secret anyway.
Post reply on HN