It wouldn't be a Paradox game without an event popping up at the absolute wrong time with no way to know if it will pause the game. It's just part of the RN... ahem fun of the game. :)
I don't understand the planet growth mechanics in Stellaris. So the game speed is never an issue for me. I always just get confused eventually and stop playing.
In my last game, the Ilhome cluster event with those fanatic purifiers happened, and I was playing some militarist, communist space cacti. I had refugees welcome as the fanatical purifiers captured a quarter of the galaxy. Then I declared war on them and quickly overwhelmed them, since they had overextended. During the 10 years of peace after the war, I was constantly busy ordering new districts and buildings build, so much that I regularly ran out of minerals, despite buying 1000 minerals each month.
So basically, late game there's too much paperwork for the turbo setting.