Live data from Hacker News

About Google's approach to research publication – Jeff Dean

docs.google.com

71–80 of 607 posts

Re: About Google's approach to research publication – Jeff Dean

#71

It's frankly unprofessional for Jeff Dean to post only his email in this document without providing more context. The news media has in cases provided balanced coverage that included the email from Timnit that prompted Google's action: https://www.platformer.news/p/the-withering-email-that-got-a... This post from Jeff Dean simply underscores that he has failed to balance the need for diplomacy with the research thesi…

[deleted]

Re: About Google's approach to research publication – Jeff Dean

#72
post #2

Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more. In all of my time at Google AI, I never heard of pubapproval being used for peer review or to criti…

And the examples of issues flagged in review that Jeff keeps highlighting—like Timnit’s alleged failure to mention recent work to reduce the environmental impact of large models—are themselves a bit worrisome. Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.

Google isnt publicly funded academic institution. Whatever they are doing, in particular publishing, is part of the business/PR. So if the management sees something not good for business it is a reasonable that they decided to not do it. If i were a shareholder i can see how i may have questioned why a person being paid $1M+/year (my understanding this is minimum what a manager in AI at Google would be making) for publicly disparaging Google.

Even more, it sounds like Google didn't ask originally for retraction, they just asked to take into account the newer research contradicting the paper - the thing that any researcher valuing integrity over agenda wouldn't refuse.

If somebody wants to do that research and publishing they just have to find another source of funding, i guess.

Anyway, the firing wasn't over the paper, the firing was over the unacceptably unprofessional reaction to it.

Re: About Google's approach to research publication – Jeff Dean

#73
post #21

This reads to me like Google felt that the paper painted some of their other technologies in a poor light, and wanted to insert language that made them look better. The way he describes their objections, they strike me as the sort of thing that is routinely addressed in the camera-ready version of papers by adding a few lines to the related work section. Not the sort of thing that a conference reviewer or an internal…

> they strike me as the sort of thing that is routinely addressed in the camera-ready version of papers by adding a few lines to the related work section

What I don't understand is why in the discussion nobody proposed amending the paper rather than withdrawing it. If Dean's issue was it didn't cite papers X,Y and Z, rather than demand a withdrawal, why didn't he just demand "I want you to amend the paper to add cites to X,Y,Z". And then, if Gebru and her coauthors were willing to add those cites, that would resolve it.

Indeed, from what I understand, "you should add a cite to X" is common peer review feedback, and a lot of papers get citations added due to requests from peer reviewers. So this isn't hugely different from that scenario.

It would seem that withdrawal over this issue would only make sense if Gebru and her coauthors refused to amend the paper to add the requested citations, but I haven't heard anything saying that she did refuse to do so. It isn't clear if the alternative solution of amending rather than withdrawing was ever brought up in the discussion by either party.

Not that I'm a researcher or anything, but if I was, and a superior told me "we need you to withdraw your paper because it doesn't cite X,Y,Z", my immediate response would be "How about I add the citations you are requesting and resubmit it with those additions?"

Re: About Google's approach to research publication – Jeff Dean

#74

TL;DR: Jeff and Co got sick of Timnit's woke bullshit, pushed back, she threw around some ultimatums, they called her bluff and pushed her out. Personally I'm glad that Google is finally cleaning house. More of this plz.

Wait, I thought the tech industry was literally North Korea, and as long as you sing the woke anthem every morning, you could never get fired?

Re: About Google's approach to research publication – Jeff Dean

#75
post #61

Earlier quoted context omitted.

> While I’m a spectator to this unfolding story and am reacting to a pretty cursory overview of what happened, this has the distinct feeling that Google thought they were moving in for a checkmate by, in their words, “accepting a resignation”, only to have it — very predictably? - blow up in their faces completely. I wouldn't say this has blown up in their face. But to be honest, I've not seen that much drama about i…

If somebody used HR as the mechanism to communicate feedback on the scientific rigor of research I was leading I would also find that concerning and objectionable. Human Resources is not peer review.

Honestly, if someone had to use HR to give feedback on anything, I would be more concerned as to why they couldn't say it directly to me.

HR wouldn't have made the final decision here, it seems other departments did. HR was just how someone felt it was safe to communicate that feedback, which for me is a major issue in a workplace.

Re: About Google's approach to research publication – Jeff Dean

#76

Earlier quoted context omitted.

And the examples of issues flagged in review that Jeff keeps highlighting—like Timnit’s alleged failure to mention recent work to reduce the environmental impact of large models—are themselves a bit worrisome. Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.

That was my impression as well. Having said that, if Jeff were to make public the paper, criticisms of the paper, and improvements made to address the problems described in the paper, that could go a long way towards clearing the air.

oh yes absolutely, i would love for timmit or jeff to release this paper as-is so we can see what happened. could be a good read.

Re: About Google's approach to research publication – Jeff Dean

#77
post #50

Timnit responded with an email requiring that a number of conditions be met in order for her to continue working at Google, including revealing the identities of every person who Megan and I had spoken to and consulted as part of the review of the paper and the exact feedback. Timnit wrote that if we didn’t meet these demands, she would leave Google and work on an end date. We accept and respect her decision to resig…

Not really. People bluff in negotiations all the time. A normal response would have been, “I can’t do that. Are you still committed to leaving?” Instead Google went all scorched earth and it’s still unclear why. It also indicates that they don’t particularly value her in progress work in the lab since normally you want the 2 weeks or whatever to squeeze some wrap up work out of folks.

People bluff but you generally have to be prepared to walk if your bluff is called. If I walk into a manager's office and demand 2x salary or I walk and the response is "we believe you are fairly compensated," you're now in a rather uncomfortable situation.

Which is of course not to say you can't have a more measured negotiation. But it can be hard to walk back from give me X or I do Y, especially if there isn't a lot of middle ground between giving you X or not giving you X.

Re: About Google's approach to research publication – Jeff Dean

#78
post #52
post #2

Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more. In all of my time at Google AI, I never heard of pubapproval being used for peer review or to criti…

This response really seems like gaslighting. He doesn't address her concerns and glosses over whether she was held to a different standard than others at GR.

He was also extremely vague, perhaps intentionally, about what the issue actually was. His sentence about when the paper was submitted and approved and all that is impossible to parse and make sense of who did what and when.

Re: About Google's approach to research publication – Jeff Dean

#79

Timnit responded with an email requiring that a number of conditions be met in order for her to continue working at Google, including revealing the identities of every person who Megan and I had spoken to and consulted as part of the review of the paper and the exact feedback. Timnit wrote that if we didn’t meet these demands, she would leave Google and work on an end date. We accept and respect her decision to resig…

According to her Twitter feed, Timnit was asking for the same transparency that normally goes with Google pub reviews -- where all the reviewers and their feedback are known to the reviewee. This red-stamp "review" did not follow the normal review process. Not a Googler, just going on what she says:

https://twitter.com/timnitGebru/status/1334881120920498180

Re: About Google's approach to research publication – Jeff Dean

#80
post #50

Timnit responded with an email requiring that a number of conditions be met in order for her to continue working at Google, including revealing the identities of every person who Megan and I had spoken to and consulted as part of the review of the paper and the exact feedback. Timnit wrote that if we didn’t meet these demands, she would leave Google and work on an end date. We accept and respect her decision to resig…

Not really. People bluff in negotiations all the time. A normal response would have been, “I can’t do that. Are you still committed to leaving?” Instead Google went all scorched earth and it’s still unclear why. It also indicates that they don’t particularly value her in progress work in the lab since normally you want the 2 weeks or whatever to squeeze some wrap up work out of folks.

It's fine to call bluffs. In fact, the definition of a bluff is something that, with perfect action on the other end, should be called.
Post reply on HN