As late as the mid-90s the vast majority of American cities had a median house price to income ratio of 3. California was 3.5-4 and New York was 5 iirc. A lot of factors converged to upend this, but within living memory housing markets were affordable. Whether everyone lived "where they wanted to" is unknowable.