Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We read the paper that forced Timnit Gebru out of Google

technologyreview.com

Post reply on HN
Print view View original

111–120 of 683 posts

Re: We read the paper that forced Timnit Gebru out of Google

#111

Post by adrian_mrd »

dang wrote: post #18

The threads so far on this ongoing story: https://news.ycombinator.com/item?id=25307167 https://news.ycombinator.com/item?id=25292386 https://news.ycombinator.com/item?id=25285502 https://news.ycombinator.com/item?id=25289445

Hi dang,

I’ve recently noticed that this commonly occurs ie multiple relevant threads that make up the top 50 of HN over say a 72 hour period, all related or rifts on a similar discussion.

Wondering if there is a way to group them as part of the same topic/submission? (thus saving you the manual work of posts like this). I appreciate this would (i) require a code change and (ii) would shift the HN model from individual threads based on 1 URL submission, but just thought I’d suggest it to the HN brains trust.

In other news: Keep up the good work that you do on HN to give this online community ‘structure’.

Re: We read the paper that forced Timnit Gebru out of Google

#113

Post by DenverCode »

lopmotr wrote: post #53
quantumwoke wrote: post #40

Dr Gebru's research AFAICT shines a valuable light on some interesting ethical problems. Yet the response so far from Google and even HN has been one of censorship and suppression of Dr Gebru's free speech. How can we reconcile free speech and Google's right of review to her research?

Not sure the details of her employment arrangement, but usually work you do for your company belongs to the company and you don't have any right to freely publish it. Seems she was able to publish it anyway, so that's pretty good, isn't it? She's got more than the usual helping of free speech. Where's the censorship or suppression?

It was published? Where?

Re: We read the paper that forced Timnit Gebru out of Google

#114

Post by mhh__ »

chaganated wrote: post #21

> There have been a few high-profile cases, such as the college student who churned out AI-generated self-help and productivity advice on a blog, which went viral. This is how much of the internet has felt for a long time. After this nugget, I now wonder if Vox is just a model trained on Piketty and Tumblr. edit: Also not sold on the CO2 argument. Too many variables! Nerds will calculate and re-calculate such things,…

Vox are economically usually a lot more neoliberal than Piketty.

Re: We read the paper that forced Timnit Gebru out of Google

#115

Post by marmaduke »

sthatipamala wrote: post #10

Are these numbers the energy to train a model? The whole point of these new NLP models is transfer learning, meaning you train the big model once and fine-tune it for each use case with a lot less training data. 5 cars worth of carbon emissions is not a lot given that it is a fixed cost. Very few are retraining BERT from scratch. EDIT: The other two points are also disingenuous. * "[AI models] will also fail to captu…

> reducing parameter sizes of big networks is the next big race

I think this race has been ongoing for a while now

Re: We read the paper that forced Timnit Gebru out of Google

#116

Post by _cs2017_ »

Despite what both sides claim, IMHO papers like this are primarily PR, ideology, and politics, rather than science and technology.

This is akin to a speech writer working for a politician, writing a piece that disagrees with the party platform, and refusing to fix it when asked.

Re: We read the paper that forced Timnit Gebru out of Google

#117

Post by xenihn »

aaomidi wrote: post #35
machinelabo wrote: post #34

Earlier quoted context omitted.

I think it is causing a reverse effect. It's going to create more racist behavior from employers - "If we cannot fire a PoC or some minority, let's not even bother hiring them in the first place." This is not good.

That is also, in fact, illegal. And it's not very hard to realize someone is doing that.

Sure, but how hard is it to prove it in a court of law?

Re: We read the paper that forced Timnit Gebru out of Google

#118

Post by rococode »

sthatipamala wrote: post #10

Are these numbers the energy to train a model? The whole point of these new NLP models is transfer learning, meaning you train the big model once and fine-tune it for each use case with a lot less training data. 5 cars worth of carbon emissions is not a lot given that it is a fixed cost. Very few are retraining BERT from scratch. EDIT: The other two points are also disingenuous. * "[AI models] will also fail to captu…

Carbon emissions arguments tend to ignore the value of what's being done as well. BERT and other transformers were meaningful experiments that were valuable in furthering a major research direction and enabling more effective consumer and business applications. In that sense, it's like any other company doing R&D - of course energy will be used and of course there will be some inefficiencies.

I think it's quite misleading to compare the energy usage of an industry-wide research effort to individual consumption. The graphs look bad - "wow, 626,000 lbs! that's 284 metric tons of CO2! a plane flight is way less!" - but there's a fundamental difference between "progress on a problem being worked on by thousands of highly-paid researchers" and "I bought a car".

Meanwhile, the worst power plants are generating on the order of 10+ million tons of CO2 every year. There are at least a dozen of these in the US alone. Car factories are emitting hundreds of thousands of tons of CO2 (Tesla is somewhere around 150,000 tons a year, apparently, and it's designed to be efficient). Perhaps activism around CO2 emissions in ML training might be better focused on improving the efficiency of those things instead, seeing as a 1% improvement would outweigh the entirety of the NLP model training industry. It's certainly good to keep in mind the energy costs of training in case things balloon out of control, but right now the costs relative to the results seem small and not worth highlighting as some forgotten sin.

Re: We read the paper that forced Timnit Gebru out of Google

#119

Post by visarga »

_cs2017_ wrote: post #116

Despite what both sides claim, IMHO papers like this are primarily PR, ideology, and politics, rather than science and technology. This is akin to a speech writer working for a politician, writing a piece that disagrees with the party platform, and refusing to fix it when asked.

yes, basically it's activism disguised as research

Re: We read the paper that forced Timnit Gebru out of Google

#120

Post by visarga »

sanxiyn wrote: post #54
hezag wrote: post #49

I did not read the paper (just like most people here), but by the title — “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” — it does not look like the CO2 emissions thing is the main topic of this research. BTW, "Stochastic Parrots" is a very descriptive name for the problem > Moreover, because the training datasets are so large, it’s hard to audit them to check for these embedded biases. “A me…

> Since these models are being applied in a lot of fields that directly affects the life of millions of people In particular, it is being applied right now to rank Google search results, and probably responsible for lots and lots of Google's profit. You should be skeptical of Google's appraisal of the paper that is material to Google's profit.

I bet you don't need to evaluate the NLP model when you have direct access to the end result, Google Search.
Post reply on HN