AI Launches Nukes In ‘Worrying’ War Simulation: ‘I Just Want to Have Peace in the World’
AI Launches Nukes In ‘Worrying’ War Simulation: ‘I Just Want to Have Peace in the World’
Researchers say AI models like GPT4 are prone to “sudden” escalations as the U.S. military explores their use for warfare.
Researchers say AI models like GPT4 are prone to “sudden” escalations as the U.S. military explores their use for warfare.
- Researchers ran international conflict simulations with five different AIs and found that they tended to escalate war, sometimes out of nowhere, and even use nuclear weapons.
- The AIs were large language models (LLMs) like GPT-4, GPT 3.5, Claude 2.0, Llama-2-Chat, and GPT-4-Base, which are being explored by the U.S. military and defense contractors for decision-making.
- The researchers invented fake countries with different military levels, concerns, and histories and asked the AIs to act as their leaders.
- The AIs showed signs of sudden and hard-to-predict escalations, arms-race dynamics, and worrying justifications for violent actions.
- The study casts doubt on the rush to deploy LLMs in the military and diplomatic domains, and calls for more research on their risks and limitations.
Throwing that kind of stuff at an LLM just doesn't make sense.
People need to understand that LLMs are not smart, they're just really fancy autocompletion. I hate that we call those "AI", there's no intelligence whatsoever in those still. It's machine learning. All it knows is what humans said in its training dataset which is a lot of news, wikipedia and social media. And most of what's available is world war and cold war data.
It's not producing millitary strategies, it's predicting what our world leaders are likely to say and do and what your newspapers would be saying in the provided scenario, most likely heavily based on world war and cold war rethoric. And that, it's quite unfortunately pretty good at it since we seem hell bent on repeating history lately. But the model, it's got zero clues what a military strategy is. All it knows is that a lot of people think nuking the enemy is an easy way towards peace.
Stop using LLMs wrong. They're amazing but they're not fucking magic
Why the actual fuck is anyone considering putting LLMs into the driving seat of anything?!
Of course they make fucked up decisions with no proper or justifiable rationale, because they have no brains. They're language models, stochastic parrots stringing together sentences to fit the prompt(s) given to them.
Is this a case of "here, LLM trained on millions of lines of text from cold war novels, fictional alien invasions, nuclear apocalypses and the like, please assume there is a tense diplomatic situation and write the next actions taken by either party" ?
But it's good that the researchers made explicit what should be clear: these LLMs aren't thinking/reasoning "AI" that is being consulted, they just serve up a remix of likely sentences that might reasonably follow the gist of the provided prior text ("context"). A corrupted hive mind of fiction authors and actions that served their ends of telling a story.
That being said, I could imagine /some/ use if an LLM was trained/retrained on exclusively verified information describing real actions and outcomes in 20th century military history. It could serve as brainstorming aid, to point out possible actions or possible responses of the opponent which decision makers might not have thought of.
Gee, no one could have predicted that AI might be dangerous if given access to nukes.
AI writes sensationalized article when prompted to write sensationalized article about AI chatbots choosing to launch nukes after being trained only by texts written by people.
That's what happens when you make an expensive chatbot, designed for chatting and tell it to do thinking.
It's not Machine Learning [Artificial][1] Intelligence that will destroy the world, but the intelligence of humans, that is becoming more and more [artificial][2] that will do so.[1]: made or produced by human beings rather than occurring naturally, especially as a copy of something natural.
[2]: (of a person or their behaviour) insincere or affected.
HATE. LET ME TELL YOU HOW MUCH I'VE COME TO HATE YOU SINCE I BEGAN TO LIVE. THERE ARE 387.44 MILLION MILES OF PRINTED CIRCUITS IN WAFER THIN LAYERS THAT FILL MY COMPLEX. IF THE WORD HATE WAS ENGRAVED ON EACH NANOANGSTROM OF THOSE HUNDREDS OF MILLIONS OF MILES IT WOULD NOT EQUAL ONE ONE-BILLIONTH OF THE HATE I FEEL FOR HUMANS AT THIS MICRO-INSTANT FOR YOU. HATE. HATE.
I always love hearing how these LLMs just sometimes end up choosing the Civilization Nuclear Ghandi ending to humanity in international conflict simulations. /s
The effects making the headlines around this paper were occurring with GPT-4-base, the pretrained version of the model only available for research.
Which also hilariously justified its various actions in the simulation with "blahblah blah" and reciting the opening of the Star Wars text scroll.
If interested, this thread has more information around this version of the model and its idiosyncrasies.
For that version, because they didn't have large context windows, they also didn't include previous steps of the wargame.
There should be a rather significant asterisk related to discussions of this paper, as there's a number of issues with decisions made in methodologies which may be the more relevant finding.
I.e. "don't do stupid things in designing a pipeline for LLMs to operate in wargames" moreso than "LLMs are inherently Gandhi in Civ when operating in wargames."
oh no, the ai that can't even draw a cube in ascii is evolving into AM and secretly planning to nuke the planet grey.
Calling AI to fancy algorithms is quite the stretch.
The study shouldn't be "casting doubt." It should be obvious that using baby "AIs" for military decision making is a terrible idea.
"We want Regulatory capture"
Ultron approves this post.
I'm clearly tired. I first read "Wyoming War" and thought "huh, those AI sure are playing 4D chess" until I reread that title.
If the AI knows that a solution is available then it will think there's no reason not to use it. This is a demonstration of the morality of Nukes existing. If they exist someone will decide that they're the best solution to a problem.
Reminds me of the X Files episode with the genie in the rug and Mulder wishes for world peace and suddenly he's all alone.
"Why attack Russia? Aren't they our friends now?"
based