AI chatbots tend to choose violence and nuclear strikes in wargames
AI chatbots tend to choose violence and nuclear strikes in wargames
As the US military begins integrating AI technology, simulated wargames show how chatbots behave unpredictably and risk nuclear escalation
Did nobody really question the usability of language models in designing war strategies?
Did nobody really question the usability of language models in designing war strategies?
Correct, people heard "AI" and went completely mad imagining things it might be able to do. And the current models act like happy dogs that are eager to give an answer to anything even if they have to make one up on the spot.
LLM are just plagiarizing bullshitting machines. It's how they are built. Plagiarism if they have the specific training data, modify the answer if they must, make it up from whole cloth as their base programming. And accidentally good enough to convince many people.
How is that structurally different from how a human answers a question? We repeat an answer we "know" if possible, assemble something from fragments of knowledge if not, and just make something up from basically nothing if needed. The main difference I see is a small degree of self reflection, the ability to estimate how 'good or bad' the answer likely is, and frankly plenty of humans are terrible at that too.
It kind of irks me how many people want to downplay this technology in this exact manner. Yes you're sort of right but in no way does that really change how it will be used and abused.
"But people think it's real AI tho!"
Okay and? Most people don't understand how most tech works and that doesn't stop it from doing a lot of good and bad things.
It kind of irks me how many people want to downplay this technology in this exact manner. Yes you're sort of right but in no way does that really change how it will be used and abused.
"But people think it's real AI tho!"
Okay and? Most people don't understand how most tech works and that doesn't stop it from doing a lot of good and bad things.
If that's really how they work, it wouldn't explain these:
https://notes.aimodels.fyi/researchers-discover-emergent-linear-strucutres-llm-truth/
https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html
Would you like to play a game?
How about a nice game of chess?
It's better than you at chess:
Here is an alternative Piped link(s):
https://piped.video/watch?v=wJzSHRNyspg
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I'm open-source; check me out at GitHub.
I see we have 5 GMs who disagree
Of course, LLM is simply copying the behavior of most people, and most people would resort to that as well.
And they probably trained it on Civ, and Gandhi was chosen as the role model.
Makes a lot of sense AI would nuke disproportionately. For an AI, if you do not set a value for something, it is worth zero. This is actually the base problem for AI: Alignment.
For a human, there's a mushy vagueness about it but our cultural upbringing says that even in war, it's bad to kill indiscriminately. And we value the future humans who do not yet exist, we recognize that after the war is over, people will want to live in the nuked place and they can't if it's radioactive. There's a self-image issue where we want to be seen as a good person by our peers and the history books. There is value there which is overlooked by programmers.
An AI will trade infinite things worth 0 for a single thing worth 1. So if nukes increase your win percentage by .1%, and they don't have the deterrence of being labeled history's greatest monster, they will nuke as many times as they can.
That explanation is obviously based on traditional chess AI. This is about role-playing with chatbots (LLMs). Think SillyTavern.
LLMs are made for text production, not tactical or strategic reasoning. The text that LLMs produce favors violence, because the text that humans produce (and want) favors violence.
Especially if its training material included comments from the early 00s. There was a lot of "nuke it from orbit" and "glass parking lot" comments about the Middle East in the wake of 911.
And with the glorified text predictors that LLMs are, you could probably adjust the wording of the question to get the opposite results. Like, "what should we do about the Middle East?" might get a "glass parking lot" response, while "should we turn the middle East into a glass parking lot?" might get a "no, nuking the middle East is a bad idea and inhumane" because that's how those conversations (using the term loosely) would go.
The text that LLMs produce favors violence, because the text that humans produce (and want) favors violence.
That's not necessarily true, there is a lot of violent fiction.
For AGI, sure, those kinds of game theory explanations are plausible. But an LLM (or any other kind of statistical model) isn't extracting concepts, forming propositions, and estimating values. It never gets beyond the realm of tokens.
Get Matthew Broderick on the horn!
AI is Civilization's Gandhi.
...how shocking
It's a WAR GAME. Emphasis on war and game. Do you chuckle fucks think wargame players should emphasize kumbaya sing dance or group therapy sessions in their games?
If the goal is to win and overwhelming force is an option, that option will always win. On the contrary, in the modern world, humans tend to try to find non-violent means in order to bring an end to wars. The point is that AI doesn't have humanity but is still being utilized by militaries (or at least that's what I think)
And a language model, absolutely unsuited for this task, just as much as a lawnmower or a float needle.
I am shocked—shocked!—to find out that a technology performs poorly when applied to a task it's completely unsuited for!
Ever heard of skynet anybody ?
How about WOPR?
DUN DUN DUN - DUN DUNN
whaaat? Robots don't just have their own inherent sense of morality for whatever reason???
Did nobody really question the usability of language models in designing war strategies?
They got some nice clickbait out of it. And that's how dumb af ideas turn into smart career moves.
I hope no one is coming away with the idea that this about something the military is actually doing.
She's just like me!
Violence in a war game?! Oh my!
Whenever we have disrupting technological advancements, DARPA looks at it to see if it can be applied to military action, and this has been true with generative AI, with LLMs and with sophisticated learning systems. They're still working on all of these.
They also get clickbait news whenever one of their test subjects does something whacky, like kill their own commander in order to expedite completing the mission parameters (in a simulation, not on the field.) The whole point is to learn how to train smart weapons to not do funny things like that.
So yes, that means on a strategic level, we're getting into the nitty of what we try to do with the tools we have. Generals typically look to minimize casualties (and to weigh factors against the expenditure of living troops) knowing that every dead soldier is a grieving family, is rhetoric against the war effort, is pressure against recruitment and so on. When we train our neural-nets, we give casualties (and risk thereof) a certain weight, so as to inform how much their respective objectives need to be worth before we throw more troopers to take them.
Fortunately, AI generals will be advisory to human generals long before they are commanding armies, themselves, or at least I'd hope so: among our DARPA scientists, military think tanks and plutocrats are a few madmen who'd gladly take over the world if they could muster a perfectly loyal robot army smart enough to fight against human opponents determined to learn and exploit any weaknesses in their logic.
AI doesn't take half measures
Do you want to play a game?
They need to be trained on the film "Wargames". Or forced to pay Noughts & Crosses against themselves.
Pull the power cord out
People who write about AI have no idea what they are talking about.
Current gen LLMs aren't magic. They are just very advanced text completion. They will do whatever they are trained on, they are not 'intelligence' and they certainly don't make 'decisions' (in the normative sense as we understand them).
tl;dr this is idiotic.
The important part of the research was that all the models had gone through 'safety' training.
That means among other things they were fine tuned to identify themselves as LLMs.
Gee - I wonder if the training data included tropes of AI launching nukes or acting unpredictably in wargames...
They really should have included evaluations of models that didn't have a specific identification or were trained to identify as human in the mix of they wanted to evaluate the underlying technology and not the specific modeled relationships between the concept of AI and the concept of strategy in wargames.
I'm sure Israel did question the use of The Gospel in developing targets. The trouble is they liked the answer.