About Google's approach to research publication – Jeff Dean • Hacker News Board

Re: About Google's approach to research publication – Jeff Dean

#521

Post by kaitai » Sat, Dec 05, 2020, 3:27 AM UTC

kortilla wrote: post #392

stjohnswarts wrote: post #358

Earlier quoted context omitted.

Maybe in certain situations. But as an engineer on more than a couple of occasions I have pushed back on safety concerns and I was adamant that certain things be fixed for the company's reputation and for safety reasons. I did go over my manager's head because he wouldn't listen. Should I have been fired? Ultimately on one occasion I went up 4 layers of org chart to a VP who finally had the sense to listen because my…

If you didn’t threaten to quit then it’s not an ultimatum at all.

That's not true at all. You can say you'll (go to the media/refuse to sign off on the regulatory paperwork/refuse to change the code that way and they don't have anyone else who can do it/refuse to change the password) or any of a number of other things.

Re: About Google's approach to research publication – Jeff Dean

#522

Post by kyrieeschaton » Sat, Dec 05, 2020, 3:28 AM UTC

Ar-Curunir wrote: post #489

dodobirdlord wrote: post #466

Earlier quoted context omitted.

It sounds like there wasn’t a great working relationship. It seems management was concerned (reasonably, based on her track record) about the prospect of her responding with hostility directed at the coworkers who expressed concerns about the quality of the work if she managed to discover their identities. Refusing to let a person have a written copy of anonymous feedback is a rational thing to do if you’re concerned…

"based on her track record"? The fuck does that mean? If you wanna say that people shouldn't call out their employers, then say that. No need for character assassination.

Anyone can look up her tweets and her interactions with, eg, Yann Lecun, and make a pretty good inference.

Re: About Google's approach to research publication – Jeff Dean

#523

Post by dvtrn » Sat, Dec 05, 2020, 3:28 AM UTC

free_rms wrote: post #520

dvtrn wrote: post #515

Earlier quoted context omitted.

What stands in your way other than being faced with possible objections to, and responses in kind to whatever your critiques may be? Objections and responses that-I would boldly say-are not stopping you from making said critiques, or rather, they hold no enforceable power that precludes you or anyone from making rebuttals of your own. They are just that, objections and responses. Which you are free to entertain or no…

Alright, here's my substantive criticism of how CRT affects various groups in practice (not doxxing my membership in any of these groups): Elite coastal white: Absolutely not threatened. Beneficiary of the system and knows how to navigate all of the social codes. Less elite or poor white: Takes the bullet that was aimed at the elite white. Asian: Scores way too high on tests for their % of the population and this is…

We just went through a mini-version of it: the go-to move is that any criticism is immediately labelled as closet white supremacy.

Is that what you truly believe I did above? That I am labeling you, and think you to be a white supremacist?

If so then allow me to be clear for a moment: I have literally no way of knowing if you're a white supremacist. I have no way of knowing if you're not actually an armada of ants collectively working to actuate the keys of a mechanical keyboard or a Boltzmann brain sending these messages through some strange and baffling form of quantum entanglement. What I am trying to expose is the very real reality that these are uncomfortable conversations, that's just intrinsic to this topic and the climate we are in.

This is fine. It is fine to admit being uncomfortable trying to process where we are, how we got here, and how we got out of it.

But one has to start by looking that beast in the face first in order to reckon with it. For some, that uncomfort gets unwittingly channeled into anger and frustration and they might not even know why or even realize it, but that can be focused, and turned into knowledge and wisdom on the issues. One's just gotta start, like I said: see it for what it is, and working from there.

If you took that to be me associating you with white supremacy, I'll try to find other ways of seeking out clarity from people next time.

Re: About Google's approach to research publication – Jeff Dean

#524

Post by rirarobo » Sat, Dec 05, 2020, 3:29 AM UTC

jkbhzsdfkhjnzs wrote: post #237

> "So if you would like to change things, I suggest focusing on leadership accountability and thinking through what types of pressures can also be applied from the outside. For instance, I believe that the Congressional Black Caucus is the entity that started forcing tech companies to report their diversity numbers. Writing more documents and saying things over and over again will tire you out but no one will listen.…

I believe this excerpt is actually what led to her firing on such short notice. Actually, I find it surprising that this is going largely unnoticed.

Jeff's emails, which were surely also crafted with sign off from PR and legal, attempt to frame this "resignation" as an issue of the paper's content, as well as the inability to meet Timnit's demands. However, this framing does not seem to align with the experiences of other Google researchers, and does not explain why she was terminated in this particular manner. If that were really the case, why didn't they just wait for Timnit to return from vacation, try to resolve the issue, and if unsuccessful, accept her resignation? Why not try to deescalate and handle the situation more tactfully?

Timnit's account of the email she received also seems to confirm this:

"we believe the end of your employment should happen faster than your email reflects because certain aspects of the email you sent last night to non-management employees in the brain group reflect behavior that is inconsistent with the expectations of a Google manager. As a result, we are accepting your resignation immediately ..." [1]

So, what aspects of that email triggered such a prompt termination? Well, Jeff's first email says that Timnit was telling employees to "stop work on critical DEI programs", and I've similarly seen some comments focus on the part of her email that reads, "What I want to say is stop writing your documents because it doesn't make a difference," but Timnit is clearly not saying to stop DEI work, she's saying to "focus on leadership accountability" and to apply external pressure.

That seems to me to be the real issue, and the sort of thing that would trigger pressure from legal. The fact that the tone of the email is frustrated, and it airs some dirty laundry, would not necessarily result in firing, if leadership really wanted to keep her on, they could work something out. According to Timnit, her own manager was not informed, she's well liked by her academic community and her Google reports (who are expressing their dismay on Twitter), and Jeff has regularly praised Timnit in the past. So, the mention of congressional pressure in her email really seems to me to be the real catalyst here.

[1] https://www.twitter.com/timnitGebru/status/13343647354463313...

Re: About Google's approach to research publication – Jeff Dean

#525

Post by jonahbenton » Sat, Dec 05, 2020, 3:30 AM UTC

Imnimo wrote: post #21

This reads to me like Google felt that the paper painted some of their other technologies in a poor light, and wanted to insert language that made them look better. The way he describes their objections, they strike me as the sort of thing that is routinely addressed in the camera-ready version of papers by adding a few lines to the related work section. Not the sort of thing that a conference reviewer or an internal…

Actually, the opposite. Speaking personally, I have been a fan boy of Dean's work on a number of fronts for a decade. I had awareness of her work but more superficially, and saw it as largely pissing into the wind. Reading hers more closely now, her papers and her communication, she has my complete respect. I better understand the way in which she was scaling this incredibly difficult mountain. Deeply, deeply impressive.

And Jeff has lost mine. Even if his comments are not just bad faith, his blindness borders on incompetence for the role he has. Yet another clever, blind white man.

I hope a position in the Biden administration can be found for her, and her vision can have a larger societal impact.

Re: About Google's approach to research publication – Jeff Dean

#526

Post by kaitai » Sat, Dec 05, 2020, 3:32 AM UTC

xapata wrote: post #370

abeppu wrote: post #359

Earlier quoted context omitted.

> It's simply unprofessional to critque your company for something while not mentioning the work they're doing to combat that. It would seem deeply problematic for an AI ethics researcher to have the expectation that when they critique their own employer, they should mention all their work to ameliorate bias or ethics problems, but to not have a similar expectation when they're critiquing other companies. Is the poin…

That's not the assertion here. A good researcher should be aware of the current state of the field, and thus mention current efforts to solve a problem when discussing that problem. Regardless of who's doing the work.

That's what the peer-review process is for, though -- for the peers to suggest that their papers get cited a few more times ;)

Seriously. Choosing who to cite is a discussion and a battle. It's not done thoughtlessly.

Re: About Google's approach to research publication – Jeff Dean

#527

Post by ThomPete » Sat, Dec 05, 2020, 3:34 AM UTC

pseudalopex wrote: post #326

lokar wrote: post #244

Earlier quoted context omitted.

> I think that there isn't internal academic review of the sort implied by Jeff at Google. Why do you think that?

"My submissions were always checked for disclosure of sensitive material, never for the quality of the literature review."[1] "In all of my time at Google AI, I never heard of pubapproval being used for peer review or to critique the scientific rigor of the work. It was never used as a journal, it was an afterthought that folks on my team would usually clear only hours before important deadlines. We like to leave pee…

other people who work at google says it was not unusual at all.

Re: About Google's approach to research publication – Jeff Dean

#528

Post by kaitai » Sat, Dec 05, 2020, 3:38 AM UTC

acituan wrote: post #428

freeone3000 wrote: post #398

Earlier quoted context omitted.

It's not "runaway activism" if you hire ethics researchers who find ethics problems. Reasons for not citing research, especially recent research, range from lack of relevance (since even though environmental improvements could have been done, they were not done! So the actual impact wasnt lessened by them at all!); To simply not having known about it. The correct reviewer response to this would be an "accept with cor…

> It's not "runaway activism" if you hire ethics researchers who find ethics problems. The fundamental difference between activism and research is that activism sets the agenda ahead of time while research explores the knowledge space and reports findings. One helps us incrementally make better sense of the world, the other wants us to narrow on particular facts while omitting other relevant facts in the name of adva…

"The omitted research is clearly relevant" -- how do you know? Why do you get to decide?

I wrote a paper recently in which I omitted most tangentially-related papers in mathematical physics, as they would not be mathematically accessible to the audience in question and also do not address the questions posed in the paper. A mathematical physicist wrote to me and was grumpy about it. Fine, I added his name one more time to make him happy. That's the reality of research papers.

It's clear that Google didn't want to be affiliated with this paper. And it's clear that it's time for Gebru to find a place with intellectual freedom, so her work can be judged on its merits.

Re: About Google's approach to research publication – Jeff Dean

#529

Post by glangdale » Sat, Dec 05, 2020, 3:46 AM UTC

nickff wrote: post #460

glangdale wrote: post #340

Earlier quoted context omitted.

This is rubbish. At Intel we called it "badge on the table". It's a statement of complete commitment (and I've been the beneficiary of someone going 'badge on the table' at least once, under circumstances I can't disclose). It doesn't imply that if a manager or VP or CEO concedes the point at issue once, then the person can now go around "putting their badge on the table" and getting their own way over and over again…

Interesting, do people who put their 'badge on the table' actually quit when they lose, or is it just a saying that's used to show emphasis?

I only know of 2 cases where I know for sure that it happened, although I've heard rumors about more. It's not going to be one of those things where you run around yelling about it, especially if you succeeded (after all, flexing on your management that you did it is likely to make your management unhappy).

One case someone succeeded. The other case some other person resigned.

Re: About Google's approach to research publication – Jeff Dean

#530

Post by glangdale » Sat, Dec 05, 2020, 3:47 AM UTC

glangdale wrote: post #529

nickff wrote: post #460

Earlier quoted context omitted.

Interesting, do people who put their 'badge on the table' actually quit when they lose, or is it just a saying that's used to show emphasis?

I only know of 2 cases where I know for sure that it happened, although I've heard rumors about more. It's not going to be one of those things where you run around yelling about it, especially if you succeeded (after all, flexing on your management that you did it is likely to make your management unhappy). One case someone succeeded. The other case some other person resigned.

Sorry, that was clumsily put - what I meant was in one case person A got what they wanted, and in another case not involving anyone from the first case, person B didn't get what they wanted and resigned.