It was an answer to the rhetorical comment that if housing stopped yielding a return on investment then there would be no housing.
I don't see the possible parallel with farmers, all there is in common is land potentially owned as property. Farmers bear fruits through labour. Real estate investors bear returns on capital.
That's interesting you mention some people going hungry, most people's inability to slap logs together to make shelter, and resorting to tax collection and distribution.
There have been many attempts to eliminate bearing returns on capital. They all resulted in a prostrate, non-functioning economy. There are two input requirements for a functioning business:
It's fine if people make a profit at providing essential goods as long as there's enough surplus to redistribute to those who markets can't reach.
No one complains about farmers profiting from farms. We care about people going hungry, so we buy some food from tax revenues and help feed them.