Our aim is to rival peer-reviewed journals in terms of the rigor and thoughtfulness in how we review research before publication.
But Gebru writes that HR and her management chain delivered her feedback in a surprise meeting where she was not allowed to read the actual feedback, understand the process which generated it, or engage in a dialogue about it:
Have you ever heard of someone getting “feedback” on a paper through a privileged and confidential document to HR?
A week before you go out on vacation, you see a meeting pop up at 4:30pm PST on your calendar (this popped up at around 2pm). No one would tell you what the meeting was about in advance. Then in that meeting your manager’s manager tells you “it has been decided” that you need to retract this paper by next week...
And you are told after a while, that your manager can read you a privileged and confidential document and you’re not supposed to even know who contributed to this document, who wrote this feedback, what process was followed or anything. You write a detailed document discussing whatever pieces of feedback you can find, asking for questions and clarifications, and it is completely ignored.
I've been through the peer review process at Physical Review Letters, SIGMOD, and VLDB. You get a document containing all the reviewer's comments, plus a metareviewer's take on the overall decision and what has to change. You can engage in dialogue with the metareviewer, including a detailed response letter justifying your choices, highlighting things the reviewers may have missed, and explaining where you plan to make changes. You get additional rounds of comments from the reviewers in light of that letter on later drafts.
I'm not a Googler, and I have no idea what the standard review process looks like there, but what Gebru describes does not sound at all like peer review. I also note that Dean does not contradict Gebru's account of the meeting or feedback process. If I had a paper rejected in this fashion, I would also demand to know what the hell was going on and who was responsible.
I think the person who submitted feedback in privileged and confidential way made a great choice. I expect more people taking this route in the future actually, people are scared in the current political athmosphere. Look at what Jeff Dean is getting, even though he didn’t do anything bad.
A manager in ethics shouldn’t ask Google to break the law by not providing confidientality that was requested.
I would certainly be fearful of providing an honest review of a paper championed by such a powerful figure at Google who could get me fired and unemployable with a tweet. Revealing the reviewers and throwing them under the bus would have been the end of any honest reviewing at Google. Dean made the right move here.
> A manager in ethics shouldn’t ask Google to break the law by not providing confidientality that was requested.
It is not implied anywhere that the reviewers requested confidentiality. It is certainly not implied that Google would be violating the law to rescind that confidentiality (and, to be clear: they almost certainly would not be).
I think it’s fairly strongly implied, particularly due to the fact that nobody would let her have possession of the written feedback. It sounds to me like there was concern that if she was provided with the written feedback she would be able to de-blind the reviewers via close analysis of the writing itself. Why else would management refuse to share the written feedback, considering that (a) they were willing to share the content, just not the specific writing itself
and (b) the feedback doesn’t seem to have been in and of itself particularly noteworthy?
My take is that the reviewers wanted to be able to express some mundane criticisms about the quality of the work without having to expose their identity to a person with a public reputation of unusual hostility.
The fact that she was able to hear the feedback but specifically not allowed to walk off with a written copy of it is what makes me think there was concern that she would try to discover the identities of the reviewers. That and the fact that she noted specifically in her writeup that she was told the feedback had come via HR.
Timnit was presented with a privileged and confidential document. That binds her from sharing the information in the document with others. It has nothing to do with the authors of the document or the confidentiality of their identities.
And importantly, in the context of a company like Google, an executive (like jeff Dean) likely has the authority to "unprivilge" said contents, whatever they are.
Given that Timnit ostensibly does not know who the people who gave the feedback, nor how they did so, and elsewhere notes that she was able to see the feedback, but was told she couldn't share it because it was privileged, can you explain how you reach this conclusion instead of that "to" was a typo in a tweet and instead should have been "via" or similar.
It does feel off, and it’s easy to read as Google trying to suppress her findings. But there’s a simple charitable reading as well. Gebru has recently and very publicly developed a reputation for behaving with hostility toward colleagues. Google is a company that prizes (or at least claims to prize) psychological safety of employees. I can see all of this being the chain reaction caused by a number of her coworkers expressing that they were unwilling to give an honest opinion about her work if she would be able to tie it back to them. If an ordinary employee caused their coworkers so much concern they would probably already have been dismissed, but Gebru is especially talented and was high-ranking.
I imagine some reviewers extracted agreement from management before giving negative feedback on this paper that the “anonymous” in “anonymous feedback” was a promise. This explains the unusualness of the situation, why the feedback flowed though a special HR channel, why specifically management was unwilling to let her have a written copy of the feedback (that could be closely analyzed to de-blind the reviewers), and why management accepted her resignation and the resulting fallout rather than agree to de-blind the reviewers.
Our aim is to rival peer-reviewed journals in terms of the rigor and thoughtfulness in how we review research before publication.
But Gebru writes that HR and her management chain delivered her feedback in a surprise meeting where she was not allowed to read the actual feedback, understand the process which generated it, or engage in a dialogue about it:
Have you ever heard of someone getting “feedback” on a paper through a privileged and confidential document to HR?
A week before you go out on vacation, you see a meeting pop up at 4:30pm PST on your calendar (this popped up at around 2pm). No one would tell you what the meeting was about in advance. Then in that meeting your manager’s manager tells you “it has been decided” that you need to retract this paper by next week...
And you are told after a while, that your manager can read you a privileged and confidential document and you’re not supposed to even know who contributed to this document, who wrote this feedback, what process was followed or anything. You write a detailed document discussing whatever pieces of feedback you can find, asking for questions and clarifications, and it is completely ignored.
(from https://www.platformer.news/p/the-withering-email-that-got-a...)
I've been through the peer review process at Physical Review Letters, SIGMOD, and VLDB. You get a document containing all the reviewer's comments, plus a metareviewer's take on the overall decision and what has to change. You can engage in dialogue with the metareviewer, including a detailed response letter justifying your choices, highlighting things the reviewers may have missed, and explaining where you plan to make changes. You get additional rounds of comments from the reviewers in light of that letter on later drafts.
I'm not a Googler, and I have no idea what the standard review process looks like there, but what Gebru describes does not sound at all like peer review. I also note that Dean does not contradict Gebru's account of the meeting or feedback process. If I had a paper rejected in this fashion, I would also demand to know what the hell was going on and who was responsible.
This feels off.