Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

About Google's approach to research publication – Jeff Dean

docs.google.com

Post reply on HN
Print view View original

331–340 of 607 posts

Re: About Google's approach to research publication – Jeff Dean

#331

Post by abeyer »

Footkerchief wrote: post #300
angry_octet wrote: post #292

Earlier quoted context omitted.

That isn't the impression I had from reading the principal's statements, to wit: > Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted. It seems like someone short-circuited the review process and submitted without review to meet a deadl…

How do you interpret "approved for submission" as "submitted without review"?

Perhaps by not eliding the first part of "instead of awaiting reviewer feedback, it was approved for submission."

Re: About Google's approach to research publication – Jeff Dean

#332

Post by calf »

BeetleB wrote: post #226
calf wrote: post #134

Earlier quoted context omitted.

The key sentence is the bullshit about a paper not offering mitigation as not helping. Since when is a scientific paper required to do that.

When I was in academia, it was not unusual for the referees to reject a paper for this reason. Of course, you are informed of that and always have the option to rewrite the paper to include that information. In some cases, though, it's not simply a matter of listing it as other work in the Intro - you may need to incorporate it into your models, etc.

That's no justification for the rationale that critique or negative results in general are not paper worth or do not advance science in less short-sighted ways.

Re: About Google's approach to research publication – Jeff Dean

#333

Post by oivey »

visarga wrote: post #269
oivey wrote: post #68

I would be absolutely furious if a manager blocked my work from being published. Even a manager who has worked as a researcher, in my experience, significantly lacks the expertise to be making such judgments. A research manager’s job is to be familiar with work across an entire portfolio, so they will be necessarily less knowledgeable than individual senior researchers. Presumably Google generally also feels this way…

Then you probably have to get hired at a corporation that never puts its own well being above the rest.

No, in my experience, you are wrong. Becoming an industrial researcher trades some independence for money, steady project funding, and the opportunity to more directly impact the world. In return your employer gets first or sole access to your research in order to make their products better and more profitable. Clueless suits forcing you to degrade the integrity of your work by demanding you soften your message or by puffing it up with PR need not be part of the equation. This case is over a research paper, not a press release. Smart companies know that employing researchers that are honest people rather than sycophants is good for their bottom line.

Probably, even Google knows this, and this is just a mistake. If I was in the position of this researcher, I’d threaten to quit, too. Google’s actions suggest it wants to appear to care about ethics in AI rather than actually caring. That’s not a environment conducive to her to have any impact, so why stay? Clearly she’s not going to have a problem finding a job elsewhere.

Re: About Google's approach to research publication – Jeff Dean

#334

Post by cmsj »

that_guy_iain wrote: post #67
theobeers wrote: post #33

Earlier quoted context omitted.

And the examples of issues flagged in review that Jeff keeps highlighting—like Timnit’s alleged failure to mention recent work to reduce the environmental impact of large models—are themselves a bit worrisome. Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.

> Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks. I get the impression that she wrote a hit piece on Google and published with Google's name. For me, it's correct they demand a retraction. It's simply unprofessional to critque your company for something while not mentioning the work they'…

Self-critique is more precious than gold, particularly when you have the scale of influence that Google does.

Re: About Google's approach to research publication – Jeff Dean

#335

Post by nickff »

cmsj wrote: post #321
nickff wrote: post #224

Earlier quoted context omitted.

As a manager, if someone gives an ultimatum, you basically have to fire them. There's no real option; Ben Horowitz covered it somewhere, but the bottom line is that if you yield, you've given up all control.

I understand some of the reasons why some managers think that way, but you can't have such simplistic rules. An ultimatum like this is an opportunity for a responsible manager to talk and rethink, but it seems like Google jumped at the opportunity to double-down on their mistake and then send out cowardly emails claiming the employee had actually resigned. If I were to apply a simplistic rule here, I would actually i…

The problem with accepting anti-social behavior is that you encourage it. This individual or another will use the same strategy against you in the future, and it will create antagonism and toxicity.

No, an ultimatum is a choice between two options; she offered Google a choice, and they selected. Expecting them to try to carve out a 'third way' is just unrealistic.

I agree that you can frame this many ways; she could have portrayed this as her resigning in protest, instead of blaming Google for being vindictive.

Re: About Google's approach to research publication – Jeff Dean

#336

Post by BeetleB »

calf wrote: post #332
BeetleB wrote: post #226

Earlier quoted context omitted.

When I was in academia, it was not unusual for the referees to reject a paper for this reason. Of course, you are informed of that and always have the option to rewrite the paper to include that information. In some cases, though, it's not simply a matter of listing it as other work in the Intro - you may need to incorporate it into your models, etc.

That's no justification for the rationale that critique or negative results in general are not paper worth or do not advance science in less short-sighted ways.

I wasn't discussing the appropriateness - I was pointing out that this behavior is normal in science. Your original comment was:

> Since when is a scientific paper required to do that.

Unfortunately, for a long time (since well before my time, at least).

Re: About Google's approach to research publication – Jeff Dean

#337

Post by cmsj »

aylmao wrote: post #239
trhway wrote: post #72

Earlier quoted context omitted.

Google isnt publicly funded academic institution. Whatever they are doing, in particular publishing, is part of the business/PR. So if the management sees something not good for business it is a reasonable that they decided to not do it. If i were a shareholder i can see how i may have questioned why a person being paid $1M+/year (my understanding this is minimum what a manager in AI at Google would be making) for pu…

> If i were a shareholder i can see how i may have questioned why a person being paid $1M+/year (my understanding this is minimum what a manager in AI at Google would be making) for publicly disparaging Google. Salary aside (because I do doubt she earned $1M+/year, my guess is probably more on the ballpark of $300k~$500k and either way not really denting Google's finances), you are not wrong, but also it's worth unde…

Yes, this is absolutely far worse than just letting the paper be published. AI ethics papers are not exactly the kind of material that gets a lot of conversation at the best of times, outside that world, but Google firing a black woman for speaking up is the kind of thing that definitely does get talked about (as we can see here).

But that aside, Google should want this kind of paper published. They absolutely should want to know and discuss every possible weakness in the ethics of their approach to AI - Google has a scale of influence so large that how they act in areas like AI, trickles down to many other organisations. To me, that gives them a responsibility to make it as ethical as is reasonably possible, and that will only happen if experts are allowed to speak freely.

One can make short-term arguments about how that hurts them, but the long-term damage of getting massive AI systems wrong, will be far, far worse.

Re: About Google's approach to research publication – Jeff Dean

#338

Post by [deleted] »

driverdan wrote: post #252
asfadfdsa wrote: post #46

There are a lot of people commenting that she didn't actually resign. I agree, but it sounds like the conversation went like this: employee: I'm not happy about x, y and z. If you don't do those, I'm going to quit. manager: well we are not going to do those, so thank you for your time. We accept your resignation and would like it to start immediately (i.e. you're fired). If you are gonna tell your manager that you pl…

As a manager I would only do this if I really wanted to fire the person already. For employees I care about I would give them an out. Part of being a good manager is understanding your employees and helping them succeed. If someone makes a statement like this in the heat of frustration it doesn't necessarily mean they will actually quit. If they're a valuable member of the team you should present them with an opportu…

[deleted]

Re: About Google's approach to research publication – Jeff Dean

#339

Post by dlp211 »

sillysaurusx wrote: post #170
usmannk wrote: post #150

Earlier quoted context omitted.

From what I read it sounds like Timnit cleared the general idea of the paper 2 months before. The paper itself seems to have been submitted for approval 1 day prior to the deadline. Another HN commenter says that submitting papers for approval "hours" before the deadline is common at Google.

Thank you. I’m not sure what to think. It seemed absurd to submit a paper for internal academic review one day prior to a major deadline. Yet today we’re hearing that it’s common (I can believe that; standard big company stuff) and that Google hasn’t been doing academic reviews at all until very recently / possibly this incident. So it feels like, suddenly, the cornerstones of the arguments against her are vaporizing…

> This could go badly for El Goog

What does this even mean? The power and resource disparity between Google and an individual researcher are so vast that this in no way can go bad for Google.

The idea that a few members on HN are disillusioned with Google is just another Tuesday for them. They literally do not care...no business of this size and magnitude do. The general public will never hear about this and if they do, they won't understand it, and if they do then they aren't the general public.

Re: About Google's approach to research publication – Jeff Dean

#340

Post by glangdale »

nickff wrote: post #224
UncleMeat wrote: post #109

Earlier quoted context omitted.

If your work is suddenly and unexpected roadblocked at the last second by internal review, the only way to make changes that prevent that from happening in the future is to clearly understand the situations and criticisms that led to the roadblock. This is why understanding who raised these concerns is important. Anonymous feedback blowing up a project at the final moments is sure to frustrate anybody. If what she ha…

As a manager, if someone gives an ultimatum, you basically have to fire them. There's no real option; Ben Horowitz covered it somewhere, but the bottom line is that if you yield, you've given up all control.

This is rubbish. At Intel we called it "badge on the table". It's a statement of complete commitment (and I've been the beneficiary of someone going 'badge on the table' at least once, under circumstances I can't disclose).

It doesn't imply that if a manager or VP or CEO concedes the point at issue once, then the person can now go around "putting their badge on the table" and getting their own way over and over again on other issues; probably making a habit of issuing ultimatums (ultimata?) will get you fired PDQ.

Post reply on HN