We read the paper that forced Timnit Gebru out of Google • Hacker News Board

Re: We read the paper that forced Timnit Gebru out of Google

#381

Post by thu2111 » Sat, Dec 05, 2020, 1:47 PM UTC

thinkingemote wrote: post #365

Anyone else see the narrative on Twitter to be so much different than Reddit and hacker news? On reddit and hacker news I have never seen any unconditional support for Timmit. Even amongst those anti Google and broadly in support of her, there's no whole agreement with all of what Timmit says; the assertion of her story, the reasons why and the conclusions. that a. She was fired b. That she was fired because of sexis…

Remember that outside the valley AI bubble Gebru is primarily famous for having publicly whipped up a mob against Yann LeCun and chasing him off Twitter by constantly claiming he was a racist sexist white man with no clue. Given that, it's maybe not a huge surprise that the tweets you're finding are more pro-Timnit than actually exists.

Another obvious problem: she keeps being described as a highly respected AI researcher, a leader of the field etc. Er, no. I read a lot of AI papers. I don't respect Gebru or the sub-field of AI ethnics much at all. What's happening here is simple: criticising a black woman in tech is impossible to do under your own name in corporate America, as radical leftists will immediately set out as a group to destroy you. The only acceptable position is to praise her and claim she's a genius. On HN and Reddit you can see what people really think and it's very far from that.

Re: We read the paper that forced Timnit Gebru out of Google

#382

Post by linspace » Sat, Dec 05, 2020, 1:51 PM UTC

thinkingemote wrote: post #365

Anyone else see the narrative on Twitter to be so much different than Reddit and hacker news? On reddit and hacker news I have never seen any unconditional support for Timmit. Even amongst those anti Google and broadly in support of her, there's no whole agreement with all of what Timmit says; the assertion of her story, the reasons why and the conclusions. that a. She was fired b. That she was fired because of sexis…

Maybe deeper knowledge of the issues at hand than in Twitter. Or maybe different interests. Why would they agree? There is a reason why I don't use Twitter and I read HN.

Re: We read the paper that forced Timnit Gebru out of Google

#383

Post by CydeWeys » Sat, Dec 05, 2020, 1:54 PM UTC

sthatipamala wrote: post #10

Are these numbers the energy to train a model? The whole point of these new NLP models is transfer learning, meaning you train the big model once and fine-tune it for each use case with a lot less training data. 5 cars worth of carbon emissions is not a lot given that it is a fixed cost. Very few are retraining BERT from scratch. EDIT: The other two points are also disingenuous. * "[AI models] will also fail to captu…

> 5 cars worth of carbon emissions is not a lot given that it is a fixed cost. Very few are retraining BERT from scratch.

It's not a fixed cost though, it's just how much was spent on this year's iteration of the model. The overall point being made is that model training costs are growing unbounded. Next year it could be 30 cars' worth or whatever for BERT-2, then 600 cars' worth for BERT-3 the year after. That's what it's warning against. At some point it isn't worth it.

Re: We read the paper that forced Timnit Gebru out of Google

#384

Post by thu2111 » Sat, Dec 05, 2020, 1:55 PM UTC

choeger wrote: post #201

While I will not dive into the extremes here, this is yet another battle in the fight over language, right? The most important point of criticism seems to be that the language models are "unethical", i.e., not subject to control by a particular political standard. Am I the only one that finds this line of argumentation highly troubling? The idea that once you do something with language there should be someone proacti…

You're not the only one.

What we're seeing here is a simple problem spilling out into the public: a lot of AI research is being done at a small number of companies that are in turn dominated by a tiny minority of people with politically extreme views. Those people consider control of language and expression very important to achieve their goals of a racialist and gender-biased society. But they're building AIs which require training on huge language corpuses, and thus inherently learn the way people writing language actually think.

Meanwhile the techniques to do "mind control" of AI aren't that good, at least not yet, so this leads to a lot of tensions when AI learns something true but unacceptable to the woke mindset.

Re: We read the paper that forced Timnit Gebru out of Google

#385

Post by golergka » Sat, Dec 05, 2020, 1:56 PM UTC

adrian_mrd wrote: post #111

dang wrote: post #18

The threads so far on this ongoing story: https://news.ycombinator.com/item?id=25307167 https://news.ycombinator.com/item?id=25292386 https://news.ycombinator.com/item?id=25285502 https://news.ycombinator.com/item?id=25289445

Hi dang, I’ve recently noticed that this commonly occurs ie multiple relevant threads that make up the top 50 of HN over say a 72 hour period, all related or rifts on a similar discussion. Wondering if there is a way to group them as part of the same topic/submission? (thus saving you the manual work of posts like this). I appreciate this would (i) require a code change and (ii) would shift the HN model from individu…

Lobsters have pretty good implementation of this, see this merged advent of code thread, for example: https://lobste.rs/s/3uxtgb/advent_code_2020_promotion_thread

Re: We read the paper that forced Timnit Gebru out of Google

#386

Post by CydeWeys » Sat, Dec 05, 2020, 1:57 PM UTC

adrian_mrd wrote: post #126

adrian_mrd wrote: post #111

Earlier quoted context omitted.

Hi dang, I’ve recently noticed that this commonly occurs ie multiple relevant threads that make up the top 50 of HN over say a 72 hour period, all related or rifts on a similar discussion. Wondering if there is a way to group them as part of the same topic/submission? (thus saving you the manual work of posts like this). I appreciate this would (i) require a code change and (ii) would shift the HN model from individu…

One potential implementation: at the top of each thread, after the metadata for the thread itself and before the comment box, are a list of ‘Related discussions’ with a bulletpoint list of related HN threads.

Unfortunately, PG is of the opinion that he "finished" HN over a decade ago, which is why we don't get any iteration on the feature set here, no matter how useful it might be. I agree that this would be a good addition but I don't see it happening with this mindset.

https://news.ycombinator.com/item?id=18183822

Re: We read the paper that forced Timnit Gebru out of Google

#387

Post by bisectable » Sat, Dec 05, 2020, 1:57 PM UTC

sillysaurusx wrote: post #362

qPM9l3XJrF wrote: post #351

Earlier quoted context omitted.

>Timnit responded with an email requiring that a number of conditions be met in order for her to continue working at Google, including revealing the identities of every person who Megan and I had spoken to and consulted as part of the review of the paper and the exact feedback. Timnit wrote that if we didn’t meet these demands, she would leave Google and work on an end date. We accept and respect her decision to resi…

The weird part is, the paper didn't seem to be saying that. It was mostly about energy usage, and bias in language models. The reviewer described it as "anodyne," which seems apt: https://old.reddit.com/r/MachineLearning/comments/k69eq0/n_t... I'd totally agree with you. That would indeed be ridiculous. But... It's strange... each time a new argument pops up, I dig into the new detail, and surprise: it seems to have…

The entire course of action you suggest could have happened were not for Timnit going public by her own volition, accusing her employer and coworkers of unethical behavior in the process, and encouraging sympathetic colleagues to apply external political and judicial pressure on Google. What for, so she can publish a review paper disregarding relevant internal feedback?

She made a lot of fuss and burnt a lot of bridges. Nobody forced her to do so.

Re: We read the paper that forced Timnit Gebru out of Google

#388

Post by confuseshrink » Sat, Dec 05, 2020, 1:58 PM UTC

sthatipamala wrote: post #10

Are these numbers the energy to train a model? The whole point of these new NLP models is transfer learning, meaning you train the big model once and fine-tune it for each use case with a lot less training data. 5 cars worth of carbon emissions is not a lot given that it is a fixed cost. Very few are retraining BERT from scratch. EDIT: The other two points are also disingenuous. * "[AI models] will also fail to captu…

The Strubell paper which is the origin of this "5 cars" number isn't even in the right ballpark for this stuff. What they did was take desktop GPU power consumption running the model in fp32, extrapolate to a 240x GPU (P100) setup that would run for a year straight at 100% power consumption.

Yes if you do run 240x p100s at literally 100% 24/7 for a year you get the power consumption of 5 cars. This run never happened though, this all ran on TPUs at lower precision, lower power consumption and much lower time to converge.

If anything this tells you that electronics are ridiculously green even when operating at 100%. I've never profiled world-wide carbon production but something tells me if you wanted to carbon optimise you'd be better served trying to take cars off the road and planes out of the sky.

Re: We read the paper that forced Timnit Gebru out of Google

#389

Post by sokoloff » Sat, Dec 05, 2020, 2:01 PM UTC

> An AI model trained on vast swaths of the internet won’t be attuned to the nuances of this vocabulary and won’t produce or interpret language in line with these new cultural norms.

As those become pervasive cultural norms, why would the model not adapt to include them?

Re: We read the paper that forced Timnit Gebru out of Google

#390

Post by yowlingcat » Sat, Dec 05, 2020, 2:03 PM UTC

dragonwriter wrote: post #72

HarryHirsch wrote: post #17

Earlier quoted context omitted.

1438 pounds of CO2 is 250 liters of gasoline, or thereabouts, that's pumping gas five times. How long does a tank of gas last the average Google employee? Less than a week, considering what East Bay traffic looks like. People get paid to publish that stuff?

The issue isn't how much BERT uses, the issue is the trend over time in how much models use, with BERT being a recent data point. The whole point of having AI ethicists is to identify current indicators of potential future ethical problems so that they can be considered in guiding the direction of development, so that you minimize acute ethical crisis.

> The whole point of having AI ethicists is to identify current indicators of potential future ethical problems so that they can be considered in guiding the direction of development, so that you minimize acute ethical crisis.

An alternative motivation could be to launder accountability so that when the acute ethical crisis /does/ come, you can throw your hands in the air and say "See? Look at how much resources we poured into this and it still happened! At least we tried!"