Homes have increased their size by 300%+ since 1950 and the cost per square foot has remained relatively constant. We simply don't build enough smaller homes - they only exist on fringe and in urban centers where there the property costs increase the overall cost.
Edit: Also our overreliance on cars has only skyrocketed. More cars per household, more miles driven. When you are required to spend 20-30% of your income on a vehicle, homeownership gets harder.