Sure, there is a component of rent that comes from the structure. But that isn't the component that causes the price difference between an apartment in San Francisco or Manhattan and the same size apartment in West Texas or Detroit.
And what limits that supply? Some combination of a limited supply of land and limited planning permission, neither of which are created by the landlord and both of which could very reasonably be taxed.
