Lots of people go on about how everyone could afford a home back in some golden age. Those homes were a lot smaller than the huge things that seem to be the standard today. Also living in prime areas like NYC has always been expensive. Slums were pretty common way back when in the cities.
Edit: I just saw an article saying the average size for new detached homes has gotten 10% smaller as land in developed areas has gotten scarcer. Prices aren't going down with this though. I wouldn't be surprised if we see a return to townhouses (similar to council houses in England). 3 deckers were built in the north for housing the influx of factory workers.
You're getting downvoted, but this is legitimately part of the issue. There is vastly less inventory of starter homes to buy, because home builders have been focused on building bigger and bigger (more lucrative) houses to sell at the upper end of the market.
Meanwhile apartment buildings (of the non luxury variety) are being built far less frequently.