Why does stuff move in a parabola on earth but in an ellipse in space?

So of course both of these are slight simplifications, but what is the connection between the two? If the earth is basically a circle, is an ellipse just a parabola stretched around a circle? Is a parabola just an approximation of a tiny part of an ellipse? How high do you have to be before you change your calculations of a trajectory?

The Math ain't mathing.