The colonizer still lives within that one...

It is, as long as you only target certain racial groups.

The real answer is that "white" isn't really a race. It's a socioeconomic in-group that exists to other those who aren't collectively agreed to be a part of it. Over time, a formerly racialized group may achieve "whiteness" via assimilation, like the Irish, Italian, and Jewish populations in America.