Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

About Google's approach to research publication – Jeff Dean

docs.google.com

Post reply on HN
Print view View original

281–290 of 607 posts

Re: About Google's approach to research publication – Jeff Dean

#281

Post by freeone3000 »

that_guy_iain wrote: post #67
theobeers wrote: post #33

Earlier quoted context omitted.

And the examples of issues flagged in review that Jeff keeps highlighting—like Timnit’s alleged failure to mention recent work to reduce the environmental impact of large models—are themselves a bit worrisome. Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.

> Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks. I get the impression that she wrote a hit piece on Google and published with Google's name. For me, it's correct they demand a retraction. It's simply unprofessional to critque your company for something while not mentioning the work they'…

Company with unethical practices hires ethics researcher

Ethics researcher publishes piece critical of company's activities

Company is shocked as to how this could happen.

Re: About Google's approach to research publication – Jeff Dean

#282

Post by tempest_ »

mtgx wrote: post #158
acoard wrote: post #64

Earlier quoted context omitted.

Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. They couch these in terms of rigour, but the substance of the problem has to do with her criticisms against AI. Ultimately it makes the whole Ethical AI department look more like a rubber stamp for Google.

I remember when Google made a whole big deal about their "AI Ethics Board" (or something along those lines), and then not even a year later they reshuffled it because those people were too critical of the company's practices. And then when there was backlash they "promised to do better" and Sundar Pichai came out with some "principles" that the company would follow for AI. Another 1-2 years later and here we are agai…

At then end of the day the incentives for large companies are always monetary.

It might be that they follow ethics because the appearance to do so has a monetary public relations value. It always comes down to that, and for publicly traded companies that set up things like an "AI Ethics Board" it is always for show since the incentives don't allow for anything else.

At the end of the day someones compensation depends on these things and you can't be hurting the bottom line.

Re: About Google's approach to research publication – Jeff Dean

#283

Post by kazamaloo »

gcr wrote: post #8
drfuchs wrote: post #6

My researcher acquaintances at industry labs at IBM, Microsoft, HP, Xerox, ATT, Bell, DEC, Compaq, etc. never have had to have their papers reviewed internally before submitting them to conferences or journals. What's up with Google?

Until now, nobody on my previous team at Google had to have their papers reviewed internally for scientific rigour either. (We have a publication approval process, but its purpose is to prevent IP leaks and maybe shield liability, not uphold scientific rigour standards)

Technically you can submit it without pubapprove unless somebody rats you out and you might face some repercussions. Otherwise every paper is supposed to go through AI Ethics committee review along with other types of reviews.

Re: About Google's approach to research publication – Jeff Dean

#284

Post by pseudalopex »

tfehring wrote: post #264
pseudalopex wrote: post #89

Earlier quoted context omitted.

Gebru complained about not being told what the criticisms were. That implies she was willing to consider feedback.

More specifically, she stated in her original email [0]: > Have you ever heard of someone getting “feedback” on a paper through a privileged and confidential document to HR? Does that sound like a standard procedure to you or does it just happen to people like me who are constantly dehumanized? So she did get feedback of some kind, though my reading is that it was probably vague legalese that she reasonably deemed in…

She implied she had to push just to get that too.

Re: About Google's approach to research publication – Jeff Dean

#285

Post by AnimalMuppet »

chris11 wrote: post #240
AnimalMuppet wrote: post #113

Earlier quoted context omitted.

It is. It's not a request you're ever going to get when you're dealing with the HR department, though. Which puts you in a no-win situation when you suspect that HR is doing shady stuff against you...

I agree with that. But it would be strange for HR to be handling peer review for scientific papers.

Which, if I understand correctly, is in fact what happened. Or did I get lost in this saga somewhere?

Re: About Google's approach to research publication – Jeff Dean

#286

Post by contemporary343 »

gcr wrote: post #2

Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more. In all of my time at Google AI, I never heard of pubapproval being used for peer review or to criti…

Similar experience to another current Google Brain researcher: https://twitter.com/le_roux_nicolas/status/13346019609729064...

Submitting conference papers last minute is... normal.

Re: About Google's approach to research publication – Jeff Dean

#287

Post by meetups323 »

aylmao wrote: post #165
Dig1t wrote: post #86

Earlier quoted context omitted.

"Do X or I'll quit" is a childish way to negotiate, its akin to a child throwing a tantrum if they don't get their way. It's extremely unprofessional and not something a team player would do. I think its clear why they took her up on her offer.

> "Do X or I'll quit" is a childish way to negotiate I disagree with this. It's really the only way to negotiate, pretty common. The wording is important here. If you word it as "Do X or I'll quit" it sounds childish, if you word it as "I believe the company should X, otherwise I can't see myself working here comfortably and will have to consider my resignation" it sounds professional, but the idea is the exact same.

Better to word it as "I believe the company should X instead of the current W, and to that end I will work with Y and Z to come up with a plan for moving to W' and what that should look like, once that is complete I will have better clarity over how to get to (W+X)/2, or if perhaps some V would actually be better for all parties.

Re: About Google's approach to research publication – Jeff Dean

#288

Post by zaksoup »

that_guy_iain wrote: post #75
zaksoup wrote: post #61

Earlier quoted context omitted.

If somebody used HR as the mechanism to communicate feedback on the scientific rigor of research I was leading I would also find that concerning and objectionable. Human Resources is not peer review.

Honestly, if someone had to use HR to give feedback on anything, I would be more concerned as to why they couldn't say it directly to me. HR wouldn't have made the final decision here, it seems other departments did. HR was just how someone felt it was safe to communicate that feedback, which for me is a major issue in a workplace.

I think there's a place for HR to help somebody give feedback about social or work-style issues. If somebody has a track record of getting defensive with face-to-face feedback it might be useful to have a manager present as a mediator and go through a formal process.

That's just not the same as giving scientific feedback in the review process for research and shouldn't be in the way there.

Re: About Google's approach to research publication – Jeff Dean

#289

Post by kazamaloo »

usmannk wrote: post #150
sillysaurusx wrote: post #127

Earlier quoted context omitted.

Dean says the paper was submitted for review on the day it was due to be published; Gebru says they had notified "PR & Policy 2 months before". Wait, really? That’s an important detail if true. The one-day timeline was a central part of the narrative surrounding this story. Notifying them two months ahead of time makes this a completely different situation. I’m a bit skeptical. Could this claim be verified somehow? S…

From what I read it sounds like Timnit cleared the general idea of the paper 2 months before. The paper itself seems to have been submitted for approval 1 day prior to the deadline. Another HN commenter says that submitting papers for approval "hours" before the deadline is common at Google.

It is common to submit for approvals hours before the deadline. However, if pubapprove process finds something that needs to be redacted, you have to withdraw the paper. That's basically the risk in it.

Re: About Google's approach to research publication – Jeff Dean

#290

Post by chaps »

nickff wrote: post #224
UncleMeat wrote: post #109

Earlier quoted context omitted.

If your work is suddenly and unexpected roadblocked at the last second by internal review, the only way to make changes that prevent that from happening in the future is to clearly understand the situations and criticisms that led to the roadblock. This is why understanding who raised these concerns is important. Anonymous feedback blowing up a project at the final moments is sure to frustrate anybody. If what she ha…

As a manager, if someone gives an ultimatum, you basically have to fire them. There's no real option; Ben Horowitz covered it somewhere, but the bottom line is that if you yield, you've given up all control.

This sounds incredibly inhumane.
Post reply on HN