The thing is, nobody knows if you write a special character or not. They might try to brute force passwords without first. But the way more important factor is length.
When I wasnt using a password manager for password creation I just did random sentences that were not allowed to have meaning. They were quite easy to memorize because they were so absurd. Usual password check would be 3 trillion trillion years or something.
You could probably write a Hashcat plugin to brute force that pretty easily. Something that tries groupings of keys that are adjacent to one another on qwerty keyboard layouts.