Well, except that the cost affects how much people are willing to pay, because if people are not willing to pay more then the cost then there won't be enough supply -> it will cost more -> people will either pay more or go without.
You seem to be ignoring my second paragraph. Supply only goes down because the builder has decided it’s not worth it to build any more to sell. That’s the market force that equalizes the price to remain above cost. At that point, as prices begin to rise, the buyer’s decision is once again only related to “how much are they willing to pay”.