What difference does it make if we're comparing apples to apples? I don't think a company that offers free lunch deciding whether to go fully remote or not has any effect on another that doesn't offer free lunch. Privilege or not doesn't make my opinion any more or less invalid.
My point is that "decision fatigue" is not a valid point to begin with since most people don't deal with such issues therefore it's not really relevant in the grand schemes of remote-first discussions.
It does when talking about companies whose offices provide perks that potentially improve quality of life, productivity, happiness, etc. The decision between working at home and working in an empty warehouse is different than home and the brand new offices of companies like Stripe, Dropbox, and many others.