The reason in books: muggles will oppress or kill them, falls flat in the face of their overwhelming power. And the fact that they've never had much is an issue.
I don't remember if it was a main book or one of those charity publications but she specifically decided to mention that witch burnings, when they caught a real witch, just resulted in the witch casting a spell that made fire tickle a bit.
Does it, tho? Wizards are powerful, sure, but they're severely outnumbered, and almost completely ignorant of technology. What's your Avada Kedavra, that moves slow enough that untrained school children can dodge it, gonna do against the Mother of all Bombs? Or a nuke?
The issue is that wizards can instantly wage a guerilla war. All it takes is Muggle Studies being taken seriously and integration in muggle society. Then it's impossible to identify them. Wandless magic could be a focus to be certain you're never carrying an identifying marker.
Then the wizards don't need to attack. They can destabilize the economy by duplicating money, precious resources, etc. These will deteriorate eventually but cause chaos and devalue the economy for a time. Imagine taking a $100 bill and just duplicating it to prop up your car-wash business as a front. But that's baby-level chicanery.
The real sauce is the mind-control. Imperio (who cares if it's unforgivable when you're at war?) and more importantly the Memory Charm and False Memory spells. Let's just get in there, stun a very important person (start low, work up) and insert the memories you need to get them to act accordingly. Remove the memories of that transaction and boom! Access to the most powerful people in the world and the ability to influence them. Now a wizard has their finger (remotely) on the nuclear button!
Seriously. Voldemort was an idiot. He used basic terror tactics for years and the only reason he wasn't killed quickly was because Magical Britian was written to be as stupid as possible because Rowling was writing a children's story about incompetent leaders.