Most people cannot afford it and we'd have to bulldoze tons of space to accommodate the parking.
However for those who enjoy racing as a hobby, I 100% agree that it's best to go to a track instead of endangering those around you and causing noise pollution.