Andrew Gelman had a good post[0] about the blind orchestra auditions:
> I think they’re talking about the estimates of 0.011 +/- 0.013 and 0.006 +/- 0.013. To say that “the impact . . . is about 1 percentage point” . . . that’s not right. The point here is not to pick on the authors for doing what everybody used to do, 20 years ago, but just to emphasize that we can’t really trust these numbers.
Along the same lines, one recruiting startup did a voice modulation experiment[1] to test the effects of masking gender:
> After running the experiment, we ended up with some rather surprising results. Contrary to what we expected (and probably contrary to what you expected as well!), masking gender had no effect on interview performance with respect to any of the scoring criteria (would advance to next round, technical ability, problem solving ability). If anything, we started to notice some trends in the opposite direction of what we expected: for technical ability, it appeared that men who were modulated to sound like women did a bit better than unmodulated men and that women who were modulated to sound like men did a bit worse than unmodulated women. Though these trends weren’t statistically significant, I am mentioning them because they were unexpected and definitely something to watch for as we collect more data.
Australia did a study[2] where they hid gender on resumes for high level Australian Public Service positions:
> What we found is that de-identifying applications at the shortlisting stage of recruitment does not appear to assist in promoting diversity in hiring. In fact, in the trial we found that overall, APS officers generally discriminated in favour of female and minority candidates. This suggests that the APS has been successful to some degree in efforts to promote awareness and support for diversity among senior staff. It also means that introducing de-identification of applications in such a context may have the unintended consequence of decreasing the number of female and minority candidates shortlisted for senior APS positions, setting back efforts to promote more diversity at the senior management levels in the public service.
> I think they’re talking about the estimates of 0.011 +/- 0.013 and 0.006 +/- 0.013. To say that “the impact . . . is about 1 percentage point” . . . that’s not right. The point here is not to pick on the authors for doing what everybody used to do, 20 years ago, but just to emphasize that we can’t really trust these numbers.
Along the same lines, one recruiting startup did a voice modulation experiment[1] to test the effects of masking gender:
> After running the experiment, we ended up with some rather surprising results. Contrary to what we expected (and probably contrary to what you expected as well!), masking gender had no effect on interview performance with respect to any of the scoring criteria (would advance to next round, technical ability, problem solving ability). If anything, we started to notice some trends in the opposite direction of what we expected: for technical ability, it appeared that men who were modulated to sound like women did a bit better than unmodulated men and that women who were modulated to sound like men did a bit worse than unmodulated women. Though these trends weren’t statistically significant, I am mentioning them because they were unexpected and definitely something to watch for as we collect more data.
Australia did a study[2] where they hid gender on resumes for high level Australian Public Service positions:
> What we found is that de-identifying applications at the shortlisting stage of recruitment does not appear to assist in promoting diversity in hiring. In fact, in the trial we found that overall, APS officers generally discriminated in favour of female and minority candidates. This suggests that the APS has been successful to some degree in efforts to promote awareness and support for diversity among senior staff. It also means that introducing de-identification of applications in such a context may have the unintended consequence of decreasing the number of female and minority candidates shortlisted for senior APS positions, setting back efforts to promote more diversity at the senior management levels in the public service.
[0] https://statmodeling.stat.columbia.edu/2019/05/11/did-blind-...
[1] https://blog.interviewing.io/we-built-voice-modulation-to-ma...
[2] https://behaviouraleconomics.pmc.gov.au/sites/default/files/...