daily reminder that john von neumann, smarter than me, you or anyone else here, recommended a first strike on the soviet union as the obvious strategy maybe intelligence isn't the only thing
[flagged]
AIs can't stop recommending nuclear strikes in war game simulations
71–80 of 281 posts
Re: AIs can't stop recommending nuclear strikes in war game simulations
#72daily reminder that john von neumann, smarter than me, you or anyone else here, recommended a first strike on the soviet union as the obvious strategy maybe intelligence isn't the only thing
Who knows. At the time, maybe it would have stopped decades of cold war. For thousands of years, the culture with the upper hand in technology has always wiped out everyone else. So when US had the bomb and USSR didn't, there was a short window to take over the world. Even more than the US did. Maybe the US conspiracy theory people wouldn't mind a 'one world government' if that government was actually the US. And uni…
> Who knows
Well we know he was wrong as his entire premise was based on war being inevitable - all the logic flows from that one wrong assumption.
Also trying to take out supposed capabilities before they are built - doesn't mean the Russia people are suddenly freed from communism. ( cf Iran ). Also there is a premise that it's somehow a one off event. When in reality you'd have to constantly monitor and potentially constantly strike ( cf Iran ).
Re: AIs can't stop recommending nuclear strikes in war game simulations
#73Why is this surprising? Nuclear weapons are available. AI has limited real world experience or grasp of the consequences. Nuke 'em seems like the obvious choice --- for something with a grade school mentality. Similar deficits in reasoning are manifested in AI results every day. Let's fire 'em and hire AI seems like the obvious choice --- for someone with a grade school mentality and blinded by greed.
It's "surprising" because there's supposed to be this thing called "alignment" which in general is supposed to make AIs not do such things. If the headline were the less interesting "AIs never recommend nuclear strikes in war games", people on HN would probably ask "how is that surprising, that's what alignment is supposed to be?" In any case, we're extremely lucky that there's about 0.001% probability of LLMs being…
In any case, we're extremely lucky that there's about 0.001% probability of LLMs being a path to AGI.
It's pretty safe to say that AGI requires a lot more than picking plausible words using probability.
The danger is the number of people in positions of leadership who don't get this. People who are easily seduced by the "fake intelligence" of LLMs.
Re: AIs can't stop recommending nuclear strikes in war game simulations
#74Why is this surprising? Nuclear weapons are available. AI has limited real world experience or grasp of the consequences. Nuke 'em seems like the obvious choice --- for something with a grade school mentality. Similar deficits in reasoning are manifested in AI results every day. Let's fire 'em and hire AI seems like the obvious choice --- for someone with a grade school mentality and blinded by greed.
> AI has limited real world experience or grasp of the consequences [of nuclear weapons]
I don't understand this argument. Almost no human has real world experience of the consequences of nuclear weapons. AI is working from the same sources of knowledge as the rest of us - text, audio, pictures, and video.
Re: AIs can't stop recommending nuclear strikes in war game simulations
#75Earlier quoted context omitted.
> but I still think most people would try to do their best to avoid firing nukes. "most people" are not in the positions that matter. A significant portion of the people who are in a position to advocate for such a decision believe that: - killing people sends em to heaven/hell where they were going anyway; and that this is also true for any of your own citizens that get killed by a counterstrike. - the end of the wo…
> "most people" are not in the positions that matter If polling were to reveal a majority of either party were more open to nuclear strikes than their predecessors, that gives policy makers a signal and an opening.
The current administration does not seem to be considering the majority within their own party considering how unpopular the current approach to immigration enforcement is. Or for another example, the glycophosphate/MAHA situation.
Re: AIs can't stop recommending nuclear strikes in war game simulations
#76Why is this surprising? Nuclear weapons are available. AI has limited real world experience or grasp of the consequences. Nuke 'em seems like the obvious choice --- for something with a grade school mentality. Similar deficits in reasoning are manifested in AI results every day. Let's fire 'em and hire AI seems like the obvious choice --- for someone with a grade school mentality and blinded by greed.
> AI has limited real world experience or grasp of the consequences [of nuclear weapons] I don't understand this argument. Almost no human has real world experience of the consequences of nuclear weapons. AI is working from the same sources of knowledge as the rest of us - text, audio, pictures, and video.
Almost no human has real world experience of the consequences of nuclear weapons.
Exactly!
Humans possess this amazing ability to understand and extrapolate beyond personal experience.
It's called "intelligence".
Re: AIs can't stop recommending nuclear strikes in war game simulations
#77Earlier quoted context omitted.
,,AI has limited real world experience or grasp of the consequences.'' People in the world have limited experience about war. We're living in a world where doing terrible things with 1000 people with photo/video documentation can get more attention then a million people dying, and the response is still not do whatever it takes so that people don't die. And now we are at a situation where nuclear escalation has alread…
> People in the world have limited experience about war. Right, but realistically, how many people today would carelessly chose "Nuke em" today? I know history knowledge isn't at its all time high directly, and most of the population is, well, not great at reasoning, but I still think most people would try to do their best to avoid firing nukes.
On social media, there are many, and this feeds back into training data. Unfortunately.
Re: AIs can't stop recommending nuclear strikes in war game simulations
#78Imagine if the models were made to play Hearts of Iron and train on the outcomes of that data what would happen.
Re: AIs can't stop recommending nuclear strikes in war game simulations
#79Why is this surprising? Nuclear weapons are available. AI has limited real world experience or grasp of the consequences. Nuke 'em seems like the obvious choice --- for something with a grade school mentality. Similar deficits in reasoning are manifested in AI results every day. Let's fire 'em and hire AI seems like the obvious choice --- for someone with a grade school mentality and blinded by greed.
> AI has limited real world experience or grasp of the consequences [of nuclear weapons] I don't understand this argument. Almost no human has real world experience of the consequences of nuclear weapons. AI is working from the same sources of knowledge as the rest of us - text, audio, pictures, and video.
AI is not at all like real intelligence. Computers do not know what words mean because they do not experience the world as we do. They don't have the common sense or wisdom that people accumulate through the experience of life. Humans can understand the consequences of nuclear war. Computers can only predict the next best word in their response from a statistical map that has no connection to meatspace.
Re: AIs can't stop recommending nuclear strikes in war game simulations
#80Earlier quoted context omitted.
There was a recent conflict that came up, and there was a debate about whether or not one of the sides was committing war crimes. And I remember thinking to myself and saying in the debate “if this were a video game strategically speaking, I’d be committing war crimes.” And sadly, I think this logic holds up.
I swear I'm not trying to start a flame war, but I think it'd be useful/valuable to know where you're from and what country you live in, as this certainly shapes how we feel about these sort of issues. I've also been dabbled in such thought experiments with friends lately, and so far we've all landed at very different conclusions, even thought there are some reasons that it might make strategic sense at the moment.
In in the US. I mean flame away, but I’m not happy about the observation I’m making, I’m not saying “given what I would do in a video game, it justifies what people would do in real life.” I’m saying “given what I would do it a video game, I think I see more clearly the choices people are making in real life.” life shouldn’t be a video game, but I think to a lot of high level leaders trying to compartmentalize it becomes one.
This is monstrous in the real world with obviously real consequences. But I think too many people say “obviously government X wouldn’t act in a monstrous way” but the video game analogy helps you see the incentives and thus, why they would/do.