AI researcher Timnit Gebru resigns from Google • Hacker News Board

Re: AI researcher Timnit Gebru resigns from Google

#821

Post by cmiles74 » Fri, Dec 04, 2020, 2:48 PM UTC

sjs382 wrote: post #397

It's hard to make too much sense about what happened. Here's what I made of it, if anyone can point out any important mistakes/inaccuracies/details: 1. She and her co-authors submitted a paper for review 1 day before its (external, for publication?) due date. It's supposed to be submitted to Google 2 weeks prior, so they can review. 2. The authors submitted it externally before the review was returned. 3. Google deci…

I'm curious about this "submitted paper for review 1 day before publication" assertion. From my reading, it sounded like work on this paper had been going on for a while. The email quoted on "Platformer" makes it sound like communication with HR (which seems weird, honestly, for a research paper) had been going on for at least two months.[0]

> Imagine this: You’ve sent a paper for feedback to 30+ researchers, you’re awaiting feedback from PR & Policy who you gave a heads up before you even wrote the work saying “we’re thinking of doing this”, working on a revision plan figuring out how to address different feedback from people, haven’t heard from PR & Policy besides them asking you for updates (in 2 months).

The NY Times reported that four of the researchers working on the paper also worked at Google.[1]

> In an interview with The Times, Dr. Gebru said her exasperation stemmed from the company’s treatment of a research paper she had written with six other researchers, four of them at Google.

It seems to me Google management should have been aware of this paper prior to the day before publication.

----

[0]: https://www.platformer.news/p/the-withering-email-that-got-a...

[1]: https://www.nytimes.com/2020/12/03/technology/google-researc...

Re: AI researcher Timnit Gebru resigns from Google

#822

Post by cmiles74 » Fri, Dec 04, 2020, 2:51 PM UTC

jsmith45 wrote: post #486

sjs382 wrote: post #397

It's hard to make too much sense about what happened. Here's what I made of it, if anyone can point out any important mistakes/inaccuracies/details: 1. She and her co-authors submitted a paper for review 1 day before its (external, for publication?) due date. It's supposed to be submitted to Google 2 weeks prior, so they can review. 2. The authors submitted it externally before the review was returned. 3. Google deci…

It sounds to me like Google has a policy that papers must be internally reviewed before they can be "published". She understood publish in the academic sense, while Google views sending the paper out for conference review as publishing. The paper ends up failing internal review, so per policy it must be promptly retracted. This is confusing to the academic who expects to get access to the raw review responses so that…

I think it's worth highlighting that Google's internal review of papers is probably significantly different than what we'd expect in an academic setting. I'm not big corporate entity expert, but I wouldn't be shocked to find out Google's internal review heavily weighs impacts on Google from both a public relations and earnings aspect.

Re: AI researcher Timnit Gebru resigns from Google

#823

Post by jsmith45 » Fri, Dec 04, 2020, 2:54 PM UTC

rishav_sharan wrote: post #733

jsmith45 wrote: post #486

Earlier quoted context omitted.

It sounds to me like Google has a policy that papers must be internally reviewed before they can be "published". She understood publish in the academic sense, while Google views sending the paper out for conference review as publishing. The paper ends up failing internal review, so per policy it must be promptly retracted. This is confusing to the academic who expects to get access to the raw review responses so that…

I don't think it's miscommunication in anyway. Timnit had submitted dozens of papers before and unless the process requirements are completely new, she would be well aware of them. This seems more of a case of selective enforcement, where she had been allowed to go around the process in the past (or felt that she has enough leverage to be able to do so now) and then not being allowed to do so this time. This feels li…

It could still be miscommunication. It is quite plausibly that she did not have any previous papers fail internal review, and almost certain that she did not have any fail internal review while already submitted to a conference (and thus "published" in Google's POV).

Not understanding Google's policy of retract it now, possibly re-"publish" (resubmit) later after addressing Google's concerns, could very well escalate things, especially since it sounds like this all occurred right as a bunch of people were taking vacation, so ability to get answers to questions/concerns was likely greatly hampered.

Re: AI researcher Timnit Gebru resigns from Google

#824

Post by cmiles74 » Fri, Dec 04, 2020, 2:55 PM UTC

_Wintermute wrote: post #796

ehsankia wrote: post #743

Earlier quoted context omitted.

> the academic who expects to get access to the raw review responses so that the paper can be fixed. Asking for the review notes is one thing, it seems like the main demand Jeff refused was giving the names of every reviewer. Why would the names of the reviewers be relevant to the academic, unless they specifically wanted to use the reviewers identity in order to undermine the notes?

Blind peer review should be pretty obvious to anyone from an academic background anyway. Asking for reviewers names from a publisher would be an outrageous request.

Was this a Google internal blind peer review? It sounds more like management pushing back for management specific reasons. I'm not sure it's the same thing, or that management should expect to be able to roadblock publication at the last minute without working with the authors directly.

Re: AI researcher Timnit Gebru resigns from Google

#825

Post by jackallis » Fri, Dec 04, 2020, 2:59 PM UTC

can't help thinking; there is her side of the story, google side of the story and than there is truth. need story from inside.

Re: AI researcher Timnit Gebru resigns from Google

#826

Post by academia_hack » Fri, Dec 04, 2020, 3:07 PM UTC

mehrdada wrote: post #750

jsmith45 wrote: post #486

Earlier quoted context omitted.

It sounds to me like Google has a policy that papers must be internally reviewed before they can be "published". She understood publish in the academic sense, while Google views sending the paper out for conference review as publishing. The paper ends up failing internal review, so per policy it must be promptly retracted. This is confusing to the academic who expects to get access to the raw review responses so that…

Obviously it's hard to pass a judgement without knowing the paper contents. Reading the emails carefully, it seems Jeff acknowledges that the initial review culminated in an "approval" despite some other "reviewers" still in pending state. That is a curious detail. Presumably someone with authority must have flipped the Approval bit within that 1 day. Why that individual did not object? The two-week excuse sounds lik…

The back channel theory sounds like a good point to me. Anonymous peer review is largely a charade, I'd be more surprised if Google didn't have fingers on the scales and flies on the wall at top ML venues than if they did. It's very possible that the leak/breach of anonymity is what kicked off her demands to get the names of people who filed complaints that led to the paper's retraction.

Re: AI researcher Timnit Gebru resigns from Google

#827

Post by mattm » Fri, Dec 04, 2020, 3:09 PM UTC

dekhn wrote: post #489

effingwewt wrote: post #214

Earlier quoted context omitted.

I think this isn't nearly so bad as forcing an early termination of two weeks notice. I had a job pretty recently, was senior tech and was eager to please, which ended up meaning having to take the hard/after hours calls no one else wanted to handle. It also meant I eventually became the only on-call tech for over 4 months straight. This in turn meant I knew the most about all of our accounts, knew the most about the…

I really think you missed an opportunity to come back as a paid consultant.

I just want to say I love this comment. It would have taken a negative situation and turned it into a possible win-win scenario.

Re: AI researcher Timnit Gebru resigns from Google

#828

Post by nmfisher » Fri, Dec 04, 2020, 3:10 PM UTC

hackerfromthefu wrote: post #818

rbanffy wrote: post #583

Earlier quoted context omitted.

She was never uncivil and she pointed out a series of issues, from what she believed was imprecise in his argument, to the societal issues that caused her to be viciously attacked by other people for engaging in the discussion. Other people questioned whether data only can be accountable for the biases and were not subjected to the kind of vitriol she was. The fact I am a white cishet male shields me from a lot of mi…

Sorry that's ridiculous - as a observer with no vested interest it was * very * clear she was very uncivil in her tone, repeatedly.

Not just that, she never provided any links to tutorials/papers/etc that she had given on the topic, and when I looked into the one workshop (from memory) that someone else mentioned, it had literally nothing to do with the issue of dataset bias.

That episode gave me the impression she was more interested in drum-beating and axe-grinding than engaging constructively. I'm not surprised she seemed to be doing the same inside Google.

Re: AI researcher Timnit Gebru resigns from Google

#829

Post by xiphias2 » Fri, Dec 04, 2020, 3:10 PM UTC

,,Google Head of AI Jeff Dean, in an internal email, told staff that an internal team determined that Ms. Gebru’s most recent AI research was insufficiently rigorous. ''

When Jeff Dean wrote me feedback for a small thing that I made, I was extremely happy to even get an answer from him. I think Timnit doesn't have any idea of the things Jeff and Sanjay wrote together, their code base was always clean, beautiful and well documented and tested, with lots of benchmark tests. She should have gone with the comments and work much more on the paper.

Re: AI researcher Timnit Gebru resigns from Google

#830

Post by FeepingCreature » Fri, Dec 04, 2020, 3:10 PM UTC

IfOnlyYouKnew wrote: post #766

rishav_sharan wrote: post #733

Earlier quoted context omitted.

I don't think it's miscommunication in anyway. Timnit had submitted dozens of papers before and unless the process requirements are completely new, she would be well aware of them. This seems more of a case of selective enforcement, where she had been allowed to go around the process in the past (or felt that she has enough leverage to be able to do so now) and then not being allowed to do so this time. This feels li…

> classic privileged tantrum just... wow. Can anyone point to a man ever being labelled in such a way on HN? Edit: comparing the number of comment hits for " his tantrum" vs " her tantrum" to the relative numbers for " his " and " her " shows that "tantrum" is, indeed, used 2.5 times as often to describe women as it is for men.

It's not automatically a given that women don't do things that get labelled as tantrums at a higher rate. To demonstrate this is sexist, you would have to show that people judge the same behavior as tantrum or not tantrum at a different rate depending on the specified gender. Which may well be the case, in which case please link the study.

To turn it around: an interesting self-experiment is to try flipping the pronouns in the story and seeing if your internal reaction to it changes.