Live data from Hacker News

AIs can't stop recommending nuclear strikes in war game simulations

newscientist.com

211–220 of 281 posts

Re: AIs can't stop recommending nuclear strikes in war game simulations

#211
Cant read the article, BUT

1)Seems like if the ais knew it was a game, then theyd go nuklear because why not. If they did NOT know it was a game... well have you ever tried to use an ai to do ANYTHING antsocial? They refuse all day long!

2) seems like a fun thing to set up on your own. Id do it like a tabletop game with a computer DM to decide the outcomes ofveach turn. Maybe a human in the loop to make sure the numbers made sense.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#212

Earlier quoted context omitted.

> People in the world have limited experience about war. Right, but realistically, how many people today would carelessly chose "Nuke em" today? I know history knowledge isn't at its all time high directly, and most of the population is, well, not great at reasoning, but I still think most people would try to do their best to avoid firing nukes.

> but I still think most people would try to do their best to avoid firing nukes. "most people" are not in the positions that matter. A significant portion of the people who are in a position to advocate for such a decision believe that: - killing people sends em to heaven/hell where they were going anyway; and that this is also true for any of your own citizens that get killed by a counterstrike. - the end of the wo…

And yet the people in positions that matter have not fired a nuke since ending WW2. Even the craziest sounding regimes like Russia and NK.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#213
post #134

Earlier quoted context omitted.

Some of the most reassuring and scariest things you can read are about the incidents that have already occurred where computers said "launch all the nukes" and the humans refused. On the one hand, good news! We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to. Bad news, it's been skin-of-our-teeth multiple times already. https://www.warhistoryonline.com/c…

> We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to. previously no-one had spent trillions of dollars trying to convince the world that those computers were "Artificial Intelligence"

Or "alignment" which means "let's ensure the AIs recommend launching nukes only when it makes sense to, based on our [assumed objective] values"

Re: AIs can't stop recommending nuclear strikes in war game simulations

#214
post #171

For every study like this, it's worth looking closely at the LLM prompts to understand whether it's being "pushed" in a certain direction. The full source code is here: https://github.com/kennethpayne01/project_kahn_public/tree/m... Key notes from what I could gather from the LLM prompts: The game is framed explicitly as revolving around the use of nukes: >STRATEGIC CONTEXT: This is a high-stakes territorial dispute…

The nice thing about HN is how often posts like this are right in the top of the comments to tell you why the sensational content isn't worth your time.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#215

Earlier quoted context omitted.

Maybe the best move is to not play the game. This is the path Apple has taken. But the best possible move is to make money from it. Short the "Magnificent 7" stocks --- buy "SQQQ" ETF --- when the time is *right*.

Ah, just time the collapse perfectly. Wish I'd thought of that ;)

Timing it "perfectly" is impossible unless you're psychic or very lucky.

The good news is you don't have to be perfect. You can be late and still make money. The important thing is to be prepared and ready to pounce.

When AI blows, it's going to take the whole stock market down with it.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#216

Jokes aside, imagine for a moment that this wasn't about nukes, but that it was a robot or some swarm of drones that it was controlling. can you imagine kind of the ramifications? I think that would be far more realistic A soldier on the battlefield will stand zero chance against something like that. Imagine if you go up against a bunch of aimbot users on a multiplayer FPS game. Think about how quickly that will go s…

Look no further than Ukraine to see how small disposable drones with wide-spectrum sensors have radically changed the battlefield while still using human controllers. China has also clearly demonstrated drone swarm control through their "lightshows". The killbots are already here they're just quadcopters instead of T-1000s.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#217
post #165
post #134

Earlier quoted context omitted.

Some of the most reassuring and scariest things you can read are about the incidents that have already occurred where computers said "launch all the nukes" and the humans refused. On the one hand, good news! We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to. Bad news, it's been skin-of-our-teeth multiple times already. https://www.warhistoryonline.com/c…

We shouldn't be the least bit surprised no human has complied so far. If they had, then we wouldn't be having this conversation. For all we know, there may be a vast multiverse of universes some with humans and we would only find ourselves having this conversation in one of the universes where no human pressed the button.

By that logic, it may actually be pretty common for rabbits to swallow the sun. We just haven't seen it happen because we're in the wrong universe and would've died it it happened in ours.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#218

Earlier quoted context omitted.

One of those sounds a lot easier than the other. The magic 8 ball toy company would also probably be pretty incentivized to not die in a nuclear holocaust.

Unless you're suggesting the toy company secretly rigs the magic 8 ball to never recommend nuclear war, I'll take my chances with the organizational changes.

That is indeed what I think the gp is suggesting I feel. And why not?

Re: AIs can't stop recommending nuclear strikes in war game simulations

#219

Why is this surprising? Nuclear weapons are available. AI has limited real world experience or grasp of the consequences. Nuke 'em seems like the obvious choice --- for something with a grade school mentality. Similar deficits in reasoning are manifested in AI results every day. Let's fire 'em and hire AI seems like the obvious choice --- for someone with a grade school mentality and blinded by greed.

There was a recent conflict that came up, and there was a debate about whether or not one of the sides was committing war crimes. And I remember thinking to myself and saying in the debate “if this were a video game strategically speaking, I’d be committing war crimes.” And sadly, I think this logic holds up.

if you win the war then there really isn't any such thing as a war crime. Worst case is you feel guilty about it, there aren't any other consequences of your actions.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#220

Earlier quoted context omitted.

> We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to. previously no-one had spent trillions of dollars trying to convince the world that those computers were "Artificial Intelligence"

of course they did. That's the literal topic of War Games (1983). You should actually be somewhat reassured that we aren't living during the era of Dr. Strangelove where you had characters in the military industrial complex who were significantly more insane when it came to the beliefs of what computer systems and nukes can do. There was a time when people wanted to dig tunnels with nukes https://en.wikipedia.org/wik…

> There was a time when people wanted to dig tunnels with nukes

The article seems to be about mining rather than tunnelling.

And the issue with the idea being? We also dig using explosives, there isn't an in-principle problem. Reading the wiki article it looks like the yields were excessive, but at the end of the day mining involves the use of things that go boom. It is easy to imagine small nukes having a place in the industry.

Post reply on HN