The middle class is leaving. Rich people in retirement can enjoy the better parts of Cali for all its worth. The poorest (homeless) can't really leave.
Most of the homeless come from out of state. The reason they can stay is that they didn't arrive with the ambition to rise at all because that's not a requirement for receiving free benefits or good weather.
