About Google's approach to research publication – Jeff Dean • Hacker News Board

Re: About Google's approach to research publication – Jeff Dean

#451

Post by spoopyskelly » Sat, Dec 05, 2020, 1:13 AM UTC

runako wrote: post #144

that_guy_iain wrote: post #47

Earlier quoted context omitted.

> While I’m a spectator to this unfolding story and am reacting to a pretty cursory overview of what happened, this has the distinct feeling that Google thought they were moving in for a checkmate by, in their words, “accepting a resignation”, only to have it — very predictably? - blow up in their faces completely. I wouldn't say this has blown up in their face. But to be honest, I've not seen that much drama about i…

> not seen that much drama about it than one post This morning, there were front-page posts about this on the New York Times, Washington Post, Wired, Google, BBC, and Financial Times. It's being covered pretty widely.

Some bad PR is worth getting rid of toxic folks that stand in the way of progress.

Re: About Google's approach to research publication – Jeff Dean

#452

Post by jolux » Sat, Dec 05, 2020, 1:13 AM UTC

kps wrote: post #367

UncleMeat wrote: post #109

Earlier quoted context omitted.

If your work is suddenly and unexpected roadblocked at the last second by internal review, the only way to make changes that prevent that from happening in the future is to clearly understand the situations and criticisms that led to the roadblock. This is why understanding who raised these concerns is important. Anonymous feedback blowing up a project at the final moments is sure to frustrate anybody. If what she ha…

> at the last second It was only ‘at the last second’ because Gebru chose not to follow the normal procedure. If the paper genuinely can't be ready until one day before the external deadline, the right thing to do is engage with the reviewers in advance, explain the problem, and provide them with drafts and work in progress, so that they can complete their work a few hours after yours. What Gebru did is the equivalen…

According to Jeff Dean's account of events, yes. According to nearly everyone else's, this process was unusual in that it even involved reviewing for content and not just IP, and Gebru herself says that the website about the process says to submit at least 1 week before publication, not two.

Re: About Google's approach to research publication – Jeff Dean

#453

Post by minhazm » Sat, Dec 05, 2020, 1:15 AM UTC

swyx wrote: post #213

exolymph wrote: post #22

Earlier quoted context omitted.

People have argued both cases. The tl;dr is that she said "I will resign if XYZ" and Google was like "we accept your resignation effective immediately." Whether or not that counts as firing is a matter of opinion.

well; also whether google has to pay severance.

There’s no legal requirement to pay severance. That’s just a courtesy most companies do in exchange for a non-disparagement agreement or during layoffs to save face a little.

Re: About Google's approach to research publication – Jeff Dean

#454

Post by [deleted] » Sat, Dec 05, 2020, 1:16 AM UTC

dmix wrote: post #379

genericone wrote: post #153

Earlier quoted context omitted.

That's an angle I had not considered. She has a reputation for retaliation using -isms and those reviewers were able to successfully ask their employer for protections against a historically confrontational coworker. Google as a company takes the hit, as it is in their great interest to prevent their less-vocal employees from getting raked into a public fight.

Google still hired people like that. They only have themselves to blame at that point. What did they expect? This is hardly the first time it's happened at Google and won't be the last.

[deleted]

Re: About Google's approach to research publication – Jeff Dean

#455

Post by sjg007 » Sat, Dec 05, 2020, 1:16 AM UTC

contemporary343 wrote: post #286

gcr wrote: post #2

Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more. In all of my time at Google AI, I never heard of pubapproval being used for peer review or to criti…

Similar experience to another current Google Brain researcher: https://twitter.com/le_roux_nicolas/status/13346019609729064... Submitting conference papers last minute is... normal.

This tweet is on point.

Re: About Google's approach to research publication – Jeff Dean

#456

Post by nickff » Sat, Dec 05, 2020, 1:18 AM UTC

cbsks wrote: post #448

nickff wrote: post #315

Earlier quoted context omitted.

If someone wants to negotiate, they don't use an ultimatum.

I understand your sentiment, but in my experience ultimatums are a common negotiation tactic and are rarely "true" ultimatums. "X is the lowest price I can sell for, take it or leave it" "How about X-10?" "Done" "I want to read 5 books!" "You can either read one book before bedtime or go to bed now" "How about 2 books?" "Okay, but then straight to bed!"

Well there are a few factors which make this situation different. If the ultimatum had been made in person, I'd think there might be room for negotiation, depending on the relationship between manager and subordinate.

Putting the ultimatum in e-mail form really raises the stakes, because there may be other people CC-ed or BCC-ed, and any response could later be weaponized.

If the relationship was already troubled, anything like an ultimatum is an opportunity for the manager to be rid of all their troubles.

The level of the threat also comes into play, and more severe threats increase the risk/tension. If the guardian had threatened to disown the child rather than send them to bed, we would read the situation differently.

Re: About Google's approach to research publication – Jeff Dean

#457

Post by creato » Sat, Dec 05, 2020, 1:19 AM UTC

gcr wrote: post #2

Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more. In all of my time at Google AI, I never heard of pubapproval being used for peer review or to criti…

I don't think the approval process is being used to enforce rigor in general, the (claimed) problem is the paper lacks rigor specifically in regards to claims about google's behavior.

Publishing a paper with a lack of rigor about some obscure mathematical technique isn't a problem for google (beyond some possible but unlikely mild reputation damage). Publishing a paper with a lack of rigor that says google is doing unethical things, when those things are questionably accurate, that is something google would (and should) have a problem with.

Whether the paper actually lacks rigor in a relevant way is not something I can comment on.

Re: About Google's approach to research publication – Jeff Dean

#458

Post by spoopyskelly » Sat, Dec 05, 2020, 1:19 AM UTC

deeviant wrote: post #128

I feel that Jeff Dean is a much better engineer than he is a leader.

You're probably right, a good leader wouldn't have hired such an useless person in the first place.

Re: About Google's approach to research publication – Jeff Dean

#459

Post by headsupftw » Sat, Dec 05, 2020, 1:21 AM UTC

eachro wrote: post #107

gcr wrote: post #2

Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more. In all of my time at Google AI, I never heard of pubapproval being used for peer review or to criti…

But in this scenario, shielding Google from liability is actually a primary concern given that Timnit is discussing ethics/bias. A paper on say novel transformer architecture, the lottery ticket hypothesis in a new setting, a new RL benchmark suite, etc is not going to expose Google to legal risk the way ethical AI research often can.

Then what's the point of hiring her and people like her to work for Google in the first place? So that Google could claim that they have Ethical AI researchers and Google's AI research is indeed ethical?

Re: About Google's approach to research publication – Jeff Dean

#460

Post by nickff » Sat, Dec 05, 2020, 1:23 AM UTC

glangdale wrote: post #340

nickff wrote: post #224

Earlier quoted context omitted.

As a manager, if someone gives an ultimatum, you basically have to fire them. There's no real option; Ben Horowitz covered it somewhere, but the bottom line is that if you yield, you've given up all control.

This is rubbish. At Intel we called it "badge on the table". It's a statement of complete commitment (and I've been the beneficiary of someone going 'badge on the table' at least once, under circumstances I can't disclose). It doesn't imply that if a manager or VP or CEO concedes the point at issue once, then the person can now go around "putting their badge on the table" and getting their own way over and over again…

Interesting, do people who put their 'badge on the table' actually quit when they lose, or is it just a saying that's used to show emphasis?