About Google's approach to research publication – Jeff Dean • Hacker News Board

Re: About Google's approach to research publication – Jeff Dean

#491

Post by dodobirdlord » Sat, Dec 05, 2020, 2:48 AM UTC

joshuamorton wrote: post #482

dodobirdlord wrote: post #469

Earlier quoted context omitted.

I think it’s fairly strongly implied, particularly due to the fact that nobody would let her have possession of the written feedback. It sounds to me like there was concern that if she was provided with the written feedback she would be able to de-blind the reviewers via close analysis of the writing itself. Why else would management refuse to share the written feedback, considering that (a) they were willing to shar…

She was ultimately able to hear the exact feedback, just not keep it. So this doesn't seem compelling in light of that.

The fact that she was able to hear the feedback but specifically not allowed to walk off with a written copy of it is what makes me think there was concern that she would try to discover the identities of the reviewers. That and the fact that she noted specifically in her writeup that she was told the feedback had come via HR.

Re: About Google's approach to research publication – Jeff Dean

#492

Post by KittenInABox » Sat, Dec 05, 2020, 2:49 AM UTC

headsupftw wrote: post #459

eachro wrote: post #107

Earlier quoted context omitted.

But in this scenario, shielding Google from liability is actually a primary concern given that Timnit is discussing ethics/bias. A paper on say novel transformer architecture, the lottery ticket hypothesis in a new setting, a new RL benchmark suite, etc is not going to expose Google to legal risk the way ethical AI research often can.

Then what's the point of hiring her and people like her to work for Google in the first place? So that Google could claim that they have Ethical AI researchers and Google's AI research is indeed ethical?

Yes.

Re: About Google's approach to research publication – Jeff Dean

#493

Post by htrp » Sat, Dec 05, 2020, 2:49 AM UTC

omgwtfbyobbq wrote: post #59

theobeers wrote: post #33

Earlier quoted context omitted.

And the examples of issues flagged in review that Jeff keeps highlighting—like Timnit’s alleged failure to mention recent work to reduce the environmental impact of large models—are themselves a bit worrisome. Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.

That was my impression as well. Having said that, if Jeff were to make public the paper, criticisms of the paper, and improvements made to address the problems described in the paper, that could go a long way towards clearing the air.

Abstract from a reviewer (source reddit /r/ml)

Abstract

The past three years of work in natural language processing have been characterized by the development and deployment of ever larger language models, especially for English. GPT-2, GPT-3, BERT and its variants have pushed the boundaries of the possible both through architectural innovations and through sheer size. Using these pre- trained models and the methodology of fine-tuning them for specific tasks, researchers have extended the state of the art on a wide array of tasks as measured by leaderboards on specific benchmarks for English. In this paper, we take a step back and ask: How big is too big? What are the possible risks associated with this technology and what paths are available for mitigating those risks? We end with recommendations including weighing the environmental and financial costs first, investing resources into curating and carefully documenting datasets rather than ingesting everything on the web, carrying out pre-development exercises evaluating how the planned approach fits into research and development goals and supports stakeholder values, and encouraging research directions beyond ever larger language models.

Anyone have the actual paper?

Re: About Google's approach to research publication – Jeff Dean

#494

Post by fatbird » Sat, Dec 05, 2020, 2:50 AM UTC

sidlls wrote: post #478

fatbird wrote: post #362

Earlier quoted context omitted.

A week before you go out on vacation, you see a meeting pop up at 4:30pm PST on your calendar (this popped up at around 2pm). No one would tell you what the meeting was about in advance. Then in that meeting your manager’s manager tells you “it has been decided” that you need to retract this paper by next week... And you are told after a while, that your manager can read you a privileged and confidential document and…

Sure that’s a fair point but asking for the reviewers to be identified is way out of bounds. What possible legitimate reason could she have for that request?

I think it was less "who said this?" and more "who was the reviewer, how were they chosen, what guidelines did they follow?"

Peer review in Academia is anonymous but has relative guarantees that the reviewer will have sufficient knowledge to judge your paper, won't be an academic rival out to skewer your work, and is generally capable of being a disinterested judge. As many others have pointed out here, Google doesn't do peer review in the academic sense, they do pre-publication review, almost always on short notice, for the purposes of protecting IP, mainly.

So imagine you're called in and told to retract your paper on substantive grounds provided by an anonymous reviewer whose feedback is sufficiently bad for you that you don't get a chance to amend your paper, it's just retracted, full stop. You can't have their written feedback, but after you kick up a fuss, you'll be granted a recitation of the feedback that won't change anything about the fate of your paper.

All this in the context of review that's normally a rubber stamp when trade secrets aren't involved.

Demanding to know who torpedoed your paper in an extraordinary way doesn't seem so unreasonable.

Re: About Google's approach to research publication – Jeff Dean

#495

Post by sitkack » Sat, Dec 05, 2020, 2:50 AM UTC

tempest_ wrote: post #282

mtgx wrote: post #158

Earlier quoted context omitted.

I remember when Google made a whole big deal about their "AI Ethics Board" (or something along those lines), and then not even a year later they reshuffled it because those people were too critical of the company's practices. And then when there was backlash they "promised to do better" and Sundar Pichai came out with some "principles" that the company would follow for AI. Another 1-2 years later and here we are agai…

At then end of the day the incentives for large companies are always monetary. It might be that they follow ethics because the appearance to do so has a monetary public relations value. It always comes down to that, and for publicly traded companies that set up things like an "AI Ethics Board" it is always for show since the incentives don't allow for anything else. At the end of the day someones compensation depends…

This has nothing to do with being a publicly traded company.

The founders have a controlling stake.

Re: About Google's approach to research publication – Jeff Dean

#496

Post by diffserv » Sat, Dec 05, 2020, 2:50 AM UTC

freeone3000 wrote: post #296

cavisne wrote: post #116

Earlier quoted context omitted.

We know large language models are super important to google, and there are lots of competitors. If they approved the paper the message would be "google thinks language models are a waste of resources and racist". There would be no academic debate on this topic as its been framed as woke and published by a militant activist, so any disagreement would be racist (see prior interactions between this researcher and other…

Academic debate is, in fact, done through conferences and journals. You saying there can be no debate is a strawman position with no basis in reality. The idea that standard rigor cannot be applied to ethics research is absurd, and seems to insinuate that the entire field is absent discipline. The proper response to her position would be to publish a response or critique. Attacking her entire field does nothing to fu…

The statement that "Academic debate is, in fact, done through conferences and journals" is not strictly true. Specially given that a lot of reviews in more popular conference are very hit and miss. You can submit the same paper to the same conference multiple times and get wildly different opinions on the same paper.

The variation in reviewers' response is often due their lack of knowledge and unfamiliarity with the problem. Take a look at the recent reviews on some of the more popular conferences on OpenReview.net. Most of the reviews don't have any substance and are often vague/generic.

I'd take the reviews from peers that I trust and are aware of my work more seriously than reviewers of conferences.

Re: About Google's approach to research publication – Jeff Dean

#497

Post by Ar-Curunir » Sat, Dec 05, 2020, 2:51 AM UTC

roenxi wrote: post #354

acoard wrote: post #64

Earlier quoted context omitted.

Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. They couch these in terms of rigour, but the substance of the problem has to do with her criticisms against AI. Ultimately it makes the whole Ethical AI department look more like a rubber stamp for Google.

Big oil companies don't pump out damning studies on oil use, big tech companies won't pump out research damning the use of their tech. If people have an expectation of Google to turn out academically pure research then I certainly respect the position and encourage it in reality. But thinking like that means life is going to contain a bunch of surprises that really shouldn't be surprising. Google is simply not going…

Yeah, but then they should get called out for wanting the good PR of having an AI ethics team, but without the headaches of having their poor ethical standards exposed.

You can't have it both ways.

Re: About Google's approach to research publication – Jeff Dean

#498

Post by sitkack » Sat, Dec 05, 2020, 2:52 AM UTC

acituan wrote: post #428

freeone3000 wrote: post #398

Earlier quoted context omitted.

It's not "runaway activism" if you hire ethics researchers who find ethics problems. Reasons for not citing research, especially recent research, range from lack of relevance (since even though environmental improvements could have been done, they were not done! So the actual impact wasnt lessened by them at all!); To simply not having known about it. The correct reviewer response to this would be an "accept with cor…

> It's not "runaway activism" if you hire ethics researchers who find ethics problems. The fundamental difference between activism and research is that activism sets the agenda ahead of time while research explores the knowledge space and reports findings. One helps us incrementally make better sense of the world, the other wants us to narrow on particular facts while omitting other relevant facts in the name of adva…

You don't have enough context or evidence to assert the things you asserting.

Re: About Google's approach to research publication – Jeff Dean

#499

Post by jolux » Sat, Dec 05, 2020, 2:53 AM UTC

jariel wrote: post #472

jolux wrote: post #452

Earlier quoted context omitted.

According to Jeff Dean's account of events, yes. According to nearly everyone else's, this process was unusual in that it even involved reviewing for content and not just IP, and Gebru herself says that the website about the process says to submit at least 1 week before publication, not two.

"According to nearly everyone else's, this process was unusual in that it even involved reviewing for content and not just IP" This is ethics for Google's AI+Search which is currently undergoing global scrutiny, particularly by Congress and specific politicians who are considering anti-trust measures against Google - and who believe that 'their political party is being treated unfairly'. It's existential concern for…

> This is not a public university, you don't get perfectly tenured academic freedom, if Google wants to put a reasonable subnote in there - and take 2 weeks to do it, it's perfectly fine.

They may be allowed to, but they're fools if they think world-class academics are going to work for them under draconian publishing standards that are not even consistently specified. I'm sure Gebru could get a tenured position at a university of her choice. They're throwing away a lot by choosing to die on this hill.

Re: About Google's approach to research publication – Jeff Dean

#500

Post by Ar-Curunir » Sat, Dec 05, 2020, 2:53 AM UTC

tmathmeyer wrote: post #429

chejazi wrote: post #140

Earlier quoted context omitted.

> I get the impression that she wrote a hit piece Hopefully the paper gets leaked so we can judge for ourselves.

I've read it internally. It's rather bland and doesn't actually indite google. I'm inclined to believe that it was a paper-submission-process issue.

That disagrees with what Dean himself is saying...