Nuclear War: An LLM Scenario
chrisclapham.com
Nuclear War: An LLM Scenario
1–10 of 30 posts
Re: Nuclear War: An LLM Scenario
#2We don’t need agi or superintelligence for these things to be dangerous. We just need to be willing to hand over our decision making to a machine.
And of course a human can make a wrong call too. In this scenario that’s what is happening. And of course we should bring all of our tools to bear when it comes to evaluating nuclear threats.
But that doesn’t make it less concerning that we’ve now got machines capable of linguistic persuasion in that toolset.
Re: Nuclear War: An LLM Scenario
#3Would you like to play a game?
Re: Nuclear War: An LLM Scenario
#4No-one got fired for ~buying IBM~ following a statistical-based text output.
Re: Nuclear War: An LLM Scenario
#5I'd posit the faster we feed LLM exhisting nuclear crisis and invented, dissimilar to its training corpus, nuclear scenarios, the better we will know how wrong they can be.
Fear-mongering isn't lucrative, isn't dopamine triggering, isn't actionable, doesn't look good on the resume, so it's tipically ignored.
Re: Nuclear War: An LLM Scenario
#6This is not unlikely. This is actually likely.
The instructions for those agents is to find signals that prove there is an attack.
Llms are steered to do what they are requested. They will interpret the signals a strongly as possible.
They will omit counter evidence to achieve their objective.
They will distort analysis to find their objective.
This has been everyone's llm problem daily. How is not that clear yet?
Re: Nuclear War: An LLM Scenario
#7We don’t need agi or superintelligence for these things to be dangerous. We just need to be willing to hand over our decision making to a machine. And of course a human can make a wrong call too. In this scenario that’s what is happening. And of course we should bring all of our tools to bear when it comes to evaluating nuclear threats. But that doesn’t make it less concerning that we’ve now got machines capable of l…
"hand over" is a misnomer - what actually happens is that there's an interaction with a machine and people either trust it too much, or forget that it's a machine (i.e. handed from one person to another and the "AI warning" label is accidentally or intentionally ripped off)
Re: Nuclear War: An LLM Scenario
#8Since the beginning of the nuclear age, literally billions of dollars have been spent paying incredibly smart people to model all aspects of nuclear war, including the chain of escalation under uncertainty.
Not to discount the importance of this risk, but we’re not likely to sleepwalk into it, barring a collapse in strategic & operational competence in planning (yeah, yeah) that would make MANY risks dangerously severe.
Re: Nuclear War: An LLM Scenario
#9This is not unlikely. This is actually likely. The instructions for those agents is to find signals that prove there is an attack. Llms are steered to do what they are requested. They will interpret the signals a strongly as possible. They will omit counter evidence to achieve their objective. They will distort analysis to find their objective. This has been everyone's llm problem daily. How is not that clear yet?
I don't disagree, but just to play devils advocate: the LLM can also be told to look for counter-evidence, and will at least make a stab at doing so. That's more than we can expect from the humans currently in charge.
Re: Nuclear War: An LLM Scenario
#10Would you like to play a game?
The quote is "Shall we play a game?”.
“Would you like to play a game?" is from Saw.