The idea that people aren't comfortable on a team that doesn't have other people matching their immutable sex/race characteristics, and that we should encourage this fragility is insane to me.

If this is the starting point, then wouldn't small, diverse teams be totally dysfunctional?

I have no sympathy for someone who can't work on a team of people of the opposite sex. In fact, in multiple jobs I've been the only man on an all female team. Not once did it occur to me that that could be a problem.