About Google's approach to research publication – Jeff Dean • Hacker News Board

Re: About Google's approach to research publication – Jeff Dean

#481

Post by GVIrish » Sat, Dec 05, 2020, 2:16 AM UTC

nickff wrote: post #224

UncleMeat wrote: post #109

Earlier quoted context omitted.

If your work is suddenly and unexpected roadblocked at the last second by internal review, the only way to make changes that prevent that from happening in the future is to clearly understand the situations and criticisms that led to the roadblock. This is why understanding who raised these concerns is important. Anonymous feedback blowing up a project at the final moments is sure to frustrate anybody. If what she ha…

As a manager, if someone gives an ultimatum, you basically have to fire them. There's no real option; Ben Horowitz covered it somewhere, but the bottom line is that if you yield, you've given up all control.

I once worked on a federal IT contract where the project manager for the team was from another company. He was a dishonest, backstabbing snake, and it reached a point where I was quite fed up.

I told my company that I was fed up and the only way I would continue in that situation is if I was given a sizable raise because I wasn't paid enough to put up with him. They gave me the raise rather than having me walk. I worked there for several more years after that.

I never issued an ultimatum before or since. Maybe there are people issuing threats all the time, bug it seems to me that people usually do that if they're frustrated but they want to stay at the company. For IT folks with desirable skills it's far easier to just get another job.

Re: About Google's approach to research publication – Jeff Dean

#482

Post by joshuamorton » Sat, Dec 05, 2020, 2:18 AM UTC

dodobirdlord wrote: post #469

joshuamorton wrote: post #397

Earlier quoted context omitted.

> A manager in ethics shouldn’t ask Google to break the law by not providing confidientality that was requested. It is not implied anywhere that the reviewers requested confidentiality. It is certainly not implied that Google would be violating the law to rescind that confidentiality (and, to be clear: they almost certainly would not be).

I think it’s fairly strongly implied, particularly due to the fact that nobody would let her have possession of the written feedback. It sounds to me like there was concern that if she was provided with the written feedback she would be able to de-blind the reviewers via close analysis of the writing itself. Why else would management refuse to share the written feedback, considering that (a) they were willing to shar…

She was ultimately able to hear the exact feedback, just not keep it. So this doesn't seem compelling in light of that.

Re: About Google's approach to research publication – Jeff Dean

#483

Post by GVIrish » Sat, Dec 05, 2020, 2:22 AM UTC

nickff wrote: post #335

cmsj wrote: post #321

Earlier quoted context omitted.

I understand some of the reasons why some managers think that way, but you can't have such simplistic rules. An ultimatum like this is an opportunity for a responsible manager to talk and rethink, but it seems like Google jumped at the opportunity to double-down on their mistake and then send out cowardly emails claiming the employee had actually resigned. If I were to apply a simplistic rule here, I would actually i…

The problem with accepting anti-social behavior is that you encourage it. This individual or another will use the same strategy against you in the future, and it will create antagonism and toxicity. No, an ultimatum is a choice between two options; she offered Google a choice, and they selected. Expecting them to try to carve out a 'third way' is just unrealistic. I agree that you can frame this many ways; she could…

Is it anti-social behavior if a company tells you, 'do X or else'? Even recently plenty of companies have told employees that they can move and work remotely but they had better report it so their salary can be adjusted. The penalty for not reporting being firing.

Ultimatums shouldn't be a frequent occurrence but they are a part of business relationships. It seems a bit unfair for an employer to treat an employee ultimatum as a fireable offense when company policies are sometimes the equivalent.

Employees sometimes decide that an employer ultimatum is offensive and quit sometimes too. But I don't think it is nor should be a set-in-stone rule that an employee that issues an ultimatum should be terminated.

Re: About Google's approach to research publication – Jeff Dean

#484

Post by s4n1ty » Sat, Dec 05, 2020, 2:26 AM UTC

dvtrn wrote: post #439

free_rms wrote: post #416

Earlier quoted context omitted.

Critical race/gender theory is all about power dynamics permeating every interaction. You might think that's exhausting, I certainly do, but that's what we're dealing with here.

You might think that's exhausting, I certainly do Why? Is there something about CRT that threatens your means and way of living, or is it forcing a type of introspection about what minorities have and continue to go through in various forms and machinations you'd rather not entertain?

If you're unfamiliar with criticisms of critical race theory then you might find this useful: https://newdiscourses.com/2020/06/reasons-critical-race-theo...

Re: About Google's approach to research publication – Jeff Dean

#485

Post by darknet_legion » Sat, Dec 05, 2020, 2:34 AM UTC

dodobirdlord wrote: post #466

tfehring wrote: post #305

Earlier quoted context omitted.

Also from Gebru's memo ([0] in the parent comment): > And you are told after a while, that your manager can read you a privileged and confidential document Emphasis mine. Showing your employee that you don't even trust her with a written copy of the rejection of her paper is not a great way to engender a good working relationship. Note that this pretty clearly seems to have happened before Gebru sent the email that D…

It sounds like there wasn’t a great working relationship. It seems management was concerned (reasonably, based on her track record) about the prospect of her responding with hostility directed at the coworkers who expressed concerns about the quality of the work if she managed to discover their identities. Refusing to let a person have a written copy of anonymous feedback is a rational thing to do if you’re concerned…

What I could read was and from the responses of her own teammates was the fact that the paper passed the internal review and that she already gave heads-up to the PR department about her work and they gave a heads up to her and suddenly a meeting pops-up and a manger's manager says to her that you need to either retract the paper or make certain changes. She was fine with the internal committee being anonymous but at this stage anyone would have demanded the same, i.e. who is the authority that thinks this paper sets a lower precedence for what google stands for i.e. some sort of human engagement and what does the authority do they take her at her word twist it and fire her by "accepting her resignation" what does this sound to you, for me a kind of high and might attitude by the authorities i.e. how come a black woman that too from the ethics department question our conception of the matter, let us how her what we can do Fire Her!!!.

This might sound a bit exaggerated but all of this is just putting google in bad light and top of that over 500 googlers have written a letter demanding an explanation for the same, those guys know about there internal workings more than you and me, so it is surprising how many review processes does google have, its just like double pressure first get the internal clearance and then work with the original reviewers of the conference. And now Jeff comes up with this explanation:https://docs.google.com/document/d/1f2kYWDXwhzYnq8ebVtuk9CqQ....

And for not once he mentions that the paper did already pass the internal standard review process.

Re: About Google's approach to research publication – Jeff Dean

#486

Post by LocalTrust » Sat, Dec 05, 2020, 2:36 AM UTC

I am reluctant to accuse Jeff Dean of bad faith, but this argument doesn't scan. I've been on the inside of an AI team during a crisis and then seen how senior management spun details to obfuscate critical details to avoid responsibility. Dean is slandering Gebru in a manner that will make it easier to dismiss her work and the work of other AI ethicists (especially women and bipoc) in the future. He and Google are actually themselves guilty of a lacking rigor (i.e. ethical rigor). Worst of all, I expect that racists inside Google and the wider industry will utilize this argumentative structure in the service of neutralizing ethicists and bipoc in the future. This is truly despicable. I used to be a great admirer of Jeff Dean.

Re: About Google's approach to research publication – Jeff Dean

#487

Post by [deleted] » Sat, Dec 05, 2020, 2:36 AM UTC

gcr wrote: post #2

Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more. In all of my time at Google AI, I never heard of pubapproval being used for peer review or to criti…

[deleted]

Re: About Google's approach to research publication – Jeff Dean

#488

Post by GVIrish » Sat, Dec 05, 2020, 2:38 AM UTC

tylerl wrote: post #477

theobeers wrote: post #33

Earlier quoted context omitted.

And the examples of issues flagged in review that Jeff keeps highlighting—like Timnit’s alleged failure to mention recent work to reduce the environmental impact of large models—are themselves a bit worrisome. Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.

The communication doesn't give that impression; instead it says that the paper makes claims that ignore significant and credible challenges to those claims. Dean said that these factors would need to be addressed, not agreed with. Publishing a transparently one-sided paper in Google's name would be a problem, not because of the side it picks, but because it suggests the researchers are too ideologically motivated to…

If that is indeed why the demand for retraction, why didn't they state that up front in the meeting where they told Timnit she needed to retract the paper or remove her name? Instead they initially refused to tell her the reasons for the demand for retraction.

They didn't give her a chance to address those factors at first.

Later they had a manager read the confidential feedback on the paper in question, but still didn't leg her read it herself.

If that feedback was only saying that the paper lacked relevant new context and advancements, why were they being so cagey about it? Something doesn't smell right about that.

Re: About Google's approach to research publication – Jeff Dean

#489

Post by Ar-Curunir » Sat, Dec 05, 2020, 2:45 AM UTC

dodobirdlord wrote: post #466

tfehring wrote: post #305

Earlier quoted context omitted.

Also from Gebru's memo ([0] in the parent comment): > And you are told after a while, that your manager can read you a privileged and confidential document Emphasis mine. Showing your employee that you don't even trust her with a written copy of the rejection of her paper is not a great way to engender a good working relationship. Note that this pretty clearly seems to have happened before Gebru sent the email that D…

It sounds like there wasn’t a great working relationship. It seems management was concerned (reasonably, based on her track record) about the prospect of her responding with hostility directed at the coworkers who expressed concerns about the quality of the work if she managed to discover their identities. Refusing to let a person have a written copy of anonymous feedback is a rational thing to do if you’re concerned…

"based on her track record"? The fuck does that mean? If you wanna say that people shouldn't call out their employers, then say that. No need for character assassination.

Re: About Google's approach to research publication – Jeff Dean

#490

Post by Ar-Curunir » Sat, Dec 05, 2020, 2:48 AM UTC

jariel wrote: post #472

jolux wrote: post #452

Earlier quoted context omitted.

According to Jeff Dean's account of events, yes. According to nearly everyone else's, this process was unusual in that it even involved reviewing for content and not just IP, and Gebru herself says that the website about the process says to submit at least 1 week before publication, not two.

"According to nearly everyone else's, this process was unusual in that it even involved reviewing for content and not just IP" This is ethics for Google's AI+Search which is currently undergoing global scrutiny, particularly by Congress and specific politicians who are considering anti-trust measures against Google - and who believe that 'their political party is being treated unfairly'. It's existential concern for…

> when really their starting point is 'silence' and they really don't have to do anything at all.

They can't have their cake and eat it too; if they want hire people to do AI ethics research, and then censor them for doing their job, then they should get called out for ethics-washing, which is exactly what's happening.

I don't know why so many people love to defend power, especially when that power is not benevolent.