Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

About Google's approach to research publication – Jeff Dean

docs.google.com

Post reply on HN
Print view View original

131–140 of 607 posts

Re: About Google's approach to research publication – Jeff Dean

#131

Post by nebulous1 »

asfadfdsa wrote: post #46

There are a lot of people commenting that she didn't actually resign. I agree, but it sounds like the conversation went like this: employee: I'm not happy about x, y and z. If you don't do those, I'm going to quit. manager: well we are not going to do those, so thank you for your time. We accept your resignation and would like it to start immediately (i.e. you're fired). If you are gonna tell your manager that you pl…

> manager: well we are not going to do those, so thank you for your time. We accept your resignation and would like it to start immediately (i.e. you're fired).

The thing is, there's a big difference between resigning and being fired for cause, even if both end with you not working at the company anymore.

Re: About Google's approach to research publication – Jeff Dean

#133

Post by lallysingh »

gcr wrote: post #2

Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more. In all of my time at Google AI, I never heard of pubapproval being used for peer review or to criti…

Also, when has feedback on a paper in this process been relayed via HR?

You only bring in HR protections to protect the company from a legal standpoint.

Re: About Google's approach to research publication – Jeff Dean

#134

Post by calf »

gcr wrote: post #2

Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more. In all of my time at Google AI, I never heard of pubapproval being used for peer review or to criti…

The key sentence is the bullshit about a paper not offering mitigation as not helping. Since when is a scientific paper required to do that.

Re: About Google's approach to research publication – Jeff Dean

#135

Post by s1artibartfast »

vecter wrote: post #80
wilde wrote: post #50

Earlier quoted context omitted.

Not really. People bluff in negotiations all the time. A normal response would have been, “I can’t do that. Are you still committed to leaving?” Instead Google went all scorched earth and it’s still unclear why. It also indicates that they don’t particularly value her in progress work in the lab since normally you want the 2 weeks or whatever to squeeze some wrap up work out of folks.

It's fine to call bluffs. In fact, the definition of a bluff is something that, with perfect action on the other end, should be called.

That's correct, but calling a bluff can mean different things. In this case, it could be "No, and we don't think you will actually leave" or 'No, goodbye".

My guess is they chose the latter because they don't like employees that run such hard negotiation tactics, and she was becoming too internally disruptive. Either way, any employee should know they are at risk after playing those cards, if not in the short term, then in the long term.

Re: About Google's approach to research publication – Jeff Dean

#136

Post by btown »

acoard wrote: post #64
theobeers wrote: post #33

Earlier quoted context omitted.

And the examples of issues flagged in review that Jeff keeps highlighting—like Timnit’s alleged failure to mention recent work to reduce the environmental impact of large models—are themselves a bit worrisome. Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.

Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. They couch these in terms of rigour, but the substance of the problem has to do with her criticisms against AI. Ultimately it makes the whole Ethical AI department look more like a rubber stamp for Google.

Let's be even more clear - they pushed out someone in their Ethical AI department because she wanted to have human conversations to determine the basis for being asked to soften critiques.

It's one thing for reviewers, even anonymous reviewers, to reject a paper on its merits; it's another, in Timnit's own words [0], to be told "'it has been decided'" through "a privileged and confidential document to HR" despite clearing the subject matter beforehand. In light of a more general frustration, it's very reasonable for Timnit to escalate the situation by putting her own career on the table, simply to request that people engage with the paper rather than flat-out rejecting it.

And if Jeff wants to respond by immediately cutting ties, and by putting out a document that doesn't even address the situation at hand (edit: much less the underlying issues of unequal treatment for women that Timnit describes)... that's a reflection of his ethics and the ethics of the company that stands behind him.

[0] For those who haven't read Timnit's memo that Jeff references in the OP, it's worth reading: https://www.platformer.news/p/the-withering-email-that-got-a...

EDIT 2: follow https://twitter.com/timnitGebru to see more of her side of the story. She retweeted https://www.wired.com/story/prominent-ai-ethics-researcher-s... as a good explanation of the situation for laypeople.

Re: About Google's approach to research publication – Jeff Dean

#137

Post by jonathankoren »

orsenthil wrote: post #88
acoard wrote: post #64

Earlier quoted context omitted.

Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. They couch these in terms of rigour, but the substance of the problem has to do with her criticisms against AI. Ultimately it makes the whole Ethical AI department look more like a rubber stamp for Google.

> Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. I didn't read that. I read the person _demanded_ who said a particular critical feedback, or questioned the approaches instead of addressing them. The person gave the ultimatum to resign if details were not shared.

Given that internal prepublication review at every company I've ever been with is merely there to avoid IP leakage, I find it very hard to believe that the feedback is is given in good faith. It's like the oil industry claiming that a climate change paper isn't talking enough about the economic benefits of growing citrus in Alaska. Quite frankly, there's simply no reason to address them, because the problems with BERT, exist with every LLM.

Google stepped in and changed the procedure for this paper, because they wanted to spike it because they were embarrassed by it.

Re: About Google's approach to research publication – Jeff Dean

#138

Post by ghaff »

nebulous1 wrote: post #120
ghaff wrote: post #77

Earlier quoted context omitted.

People bluff but you generally have to be prepared to walk if your bluff is called. If I walk into a manager's office and demand 2x salary or I walk and the response is "we believe you are fairly compensated," you're now in a rather uncomfortable situation. Which is of course not to say you can't have a more measured negotiation. But it can be hard to walk back from give me X or I do Y, especially if there isn't a lo…

> People bluff but you generally have to be prepared to walk if your bluff is called. Then you aren't bluffing.

Fair enough. And maybe the lesson is don't bluff in situations where you aren't prepared to deal with the consequences if the other party calls your bluff.

Re: About Google's approach to research publication – Jeff Dean

#139

Post by mdanger007 »

spinningslate wrote: post #93

I don't know Jeff Dean. I have read some of his work, watched some of his presentations. He seems a credible bloke. This, though, looks and feels like thinly-veiled retroactive and pretty unconvincing PR. It's short on detail and appears somewhat at odds with several points from Timnit Gebru's resignation note [0]: - Dean says the paper was reviewed by a "cross-functional team". Gebru says she received the feedback t…

Your honor, I didn't beat my wife. But if I did, she deserved it.

Re: About Google's approach to research publication – Jeff Dean

#140

Post by chejazi »

that_guy_iain wrote: post #67
theobeers wrote: post #33

Earlier quoted context omitted.

And the examples of issues flagged in review that Jeff keeps highlighting—like Timnit’s alleged failure to mention recent work to reduce the environmental impact of large models—are themselves a bit worrisome. Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.

> Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks. I get the impression that she wrote a hit piece on Google and published with Google's name. For me, it's correct they demand a retraction. It's simply unprofessional to critque your company for something while not mentioning the work they'…

> I get the impression that she wrote a hit piece

Hopefully the paper gets leaked so we can judge for ourselves.

Post reply on HN