Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

About Google's approach to research publication – Jeff Dean

docs.google.com

Post reply on HN
Print view View original

101–110 of 607 posts

Re: About Google's approach to research publication – Jeff Dean

#101

Post by sillysaurusx »

that_guy_iain wrote: post #47
bnj wrote: post #5

While I’m a spectator to this unfolding story and am reacting to a pretty cursory overview of what happened, this has the distinct feeling that Google thought they were moving in for a checkmate by, in their words, “accepting a resignation”, only to have it — very predictably? - blow up in their faces completely. Even if Google is acting presenting the chain of events faithfully, their final move to jump on an opport…

> While I’m a spectator to this unfolding story and am reacting to a pretty cursory overview of what happened, this has the distinct feeling that Google thought they were moving in for a checkmate by, in their words, “accepting a resignation”, only to have it — very predictably? - blow up in their faces completely. I wouldn't say this has blown up in their face. But to be honest, I've not seen that much drama about i…

I believe she basically sent an email to fellow employees basically telling them to stop working. That alone is a firable. That is completely nuts and unacceptable work place behaviour. The fact Google didn't fire her for that should speak volumes.

This is little more than gossip unless you can point to something specific that people can judge for themselves. Saying “she basically did X” is a clever way to influence how people feel about her.

I’ll admit, I had negative thoughts till this thread. Now I’m not so sure. She’s definitely passionate, and one might say aggressive, but more and more people are saying that it’s extremely unusual for Google to demand a rejection for academic reasons rather than business concerns. Whether she was a good employee is kind of beside the point now.

Re: About Google's approach to research publication – Jeff Dean

#102

Post by SpicyLemonZest »

reikonomusha wrote: post #94
asfadfdsa wrote: post #46

There are a lot of people commenting that she didn't actually resign. I agree, but it sounds like the conversation went like this: employee: I'm not happy about x, y and z. If you don't do those, I'm going to quit. manager: well we are not going to do those, so thank you for your time. We accept your resignation and would like it to start immediately (i.e. you're fired). If you are gonna tell your manager that you pl…

What if the language weren’t so direct? For example, > If I can’t know who is providing reviews to my work, it’s difficult for me to imagine continuing to work as a part of this group. Many people would still describe this (erroneously) as an ultimatum even though it could be bleakly summarized in the same vein as a true ultimatum.

We don't know the exact language, but by Timnit's account she made a numbered list of demands, which is pretty inherently direct.

Re: About Google's approach to research publication – Jeff Dean

#103

Post by vecter »

ketzo wrote: post #85
petulla wrote: post #52

Earlier quoted context omitted.

This response really seems like gaslighting. He doesn't address her concerns and glosses over whether she was held to a different standard than others at GR.

Right -- he once again talks about "accepting" her resignation, when it reeeeeally just looks like they fired her. At the very least, she certainly feels like she was fired; why is that not mentioned at all? Even just, I don't know, "sorry we were abrupt?"

Don't make ultimatums unless you're ready to accept the consequences of those ultimatums.

Re: About Google's approach to research publication – Jeff Dean

#104

Post by sudosysgen »

orsenthil wrote: post #88
acoard wrote: post #64

Earlier quoted context omitted.

Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. They couch these in terms of rigour, but the substance of the problem has to do with her criticisms against AI. Ultimately it makes the whole Ethical AI department look more like a rubber stamp for Google.

> Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. I didn't read that. I read the person _demanded_ who said a particular critical feedback, or questioned the approaches instead of addressing them. The person gave the ultimatum to resign if details were not shared.

Unless she lied in her first e-mail, which it doesn't seem like she did, the reason she made those demands is because they asked for a retraction of the paper without indicating why the paper should be retracted.

Asking for the identity of people that have the authority to ask for a withdrawal of your research without stating their issues with it seems understandable, if excessive.

But maybe I misunderstood something.

Re: About Google's approach to research publication – Jeff Dean

#105

Post by fatbird »

sidlls wrote: post #83
bnj wrote: post #41

Earlier quoted context omitted.

Isn’t it pretty normal to ask to “face your accuser” and engage with feedback specifically, instead of an anonymous and unassailable red stamp?

In the context of peer review it’s common, or at least not uncommon, for reviewers to be anonymous. Personally I think it should be completely blind both directions to promote maximum objectivity. Whether that applies at all in this case is a separate matter.

True, but it's also the norm that peer reviewer's feedback is provided to allow the author the address issues. In Timnit's account, she received neither identities nor the feedback itself.

Re: About Google's approach to research publication – Jeff Dean

#106

Post by albatruss »

trhway wrote: post #72
theobeers wrote: post #33

Earlier quoted context omitted.

And the examples of issues flagged in review that Jeff keeps highlighting—like Timnit’s alleged failure to mention recent work to reduce the environmental impact of large models—are themselves a bit worrisome. Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.

Google isnt publicly funded academic institution. Whatever they are doing, in particular publishing, is part of the business/PR. So if the management sees something not good for business it is a reasonable that they decided to not do it. If i were a shareholder i can see how i may have questioned why a person being paid $1M+/year (my understanding this is minimum what a manager in AI at Google would be making) for pu…

From a certain narrow, selfish perspective it's reasonable for Google to not want to have an AI ethics department placing a check on their leading edge research at all. Fortunately, we don't live in a world where corporations are the ones to determine right from wrong with total impunity.

Re: About Google's approach to research publication – Jeff Dean

#107

Post by eachro »

gcr wrote: post #2

Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more. In all of my time at Google AI, I never heard of pubapproval being used for peer review or to criti…

But in this scenario, shielding Google from liability is actually a primary concern given that Timnit is discussing ethics/bias. A paper on say novel transformer architecture, the lottery ticket hypothesis in a new setting, a new RL benchmark suite, etc is not going to expose Google to legal risk the way ethical AI research often can.

Re: About Google's approach to research publication – Jeff Dean

#108

Post by visarga »

theobeers wrote: post #33
gcr wrote: post #2

Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more. In all of my time at Google AI, I never heard of pubapproval being used for peer review or to criti…

And the examples of issues flagged in review that Jeff keeps highlighting—like Timnit’s alleged failure to mention recent work to reduce the environmental impact of large models—are themselves a bit worrisome. Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.

> Jeff gives the impression that they demanded retraction (!)

In paper reviews you can often see reviewers asking the authors to rewrite, clarify, add extra experiments, add missing citations. It's all normal.

Re: About Google's approach to research publication – Jeff Dean

#109

Post by UncleMeat »

orsenthil wrote: post #88
acoard wrote: post #64

Earlier quoted context omitted.

Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. They couch these in terms of rigour, but the substance of the problem has to do with her criticisms against AI. Ultimately it makes the whole Ethical AI department look more like a rubber stamp for Google.

> Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. I didn't read that. I read the person _demanded_ who said a particular critical feedback, or questioned the approaches instead of addressing them. The person gave the ultimatum to resign if details were not shared.

If your work is suddenly and unexpected roadblocked at the last second by internal review, the only way to make changes that prevent that from happening in the future is to clearly understand the situations and criticisms that led to the roadblock. This is why understanding who raised these concerns is important. Anonymous feedback blowing up a project at the final moments is sure to frustrate anybody. If what she has said is true then it was also very difficult for her to even have access to the substance of the critique in the first place, with the initial story from her management being that she would not be able to see the documents explaining why the paper was to be retracted.

The critique here appears to have been fairly minor, too. Failing to cite some recent research is rarely grounds for rejection.

Re: About Google's approach to research publication – Jeff Dean

#110

Post by lokar »

gcr wrote: post #2

Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more. In all of my time at Google AI, I never heard of pubapproval being used for peer review or to criti…

In technical infra I saw papers rejected that were deemed to be not interesting enough (not a big enough novel contribution).
Post reply on HN