Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On the dangers of stochastic parrots: Can language models be too big? (2021)

dl.acm.org

Post reply on HN
Print view View original

21–30 of 111 posts

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#21

Post by visarga »

srvmshr wrote: post #14

I am of the general understanding that this paper became less about the LLMs & more of a insinuating hit piece against Alphabet. At least, some of the controversial nuggets got Gebru (and later M Mitchell) fired. From a technical standpoint, there is little new stuff that I found this paper offered in understanding why LLMs can have unpredictable nature, or what degree of data will get exposed by clever hacks (or if…

It was activism masquerading as science. Many researches noted that positives and negatives were not presented in a balanced way. New approaches and efforts were not credited.

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#22

Post by oh_sigh »

xiaolingxiao wrote: post #3

I believe this is the papers that got timnit and mmitchel fired from google, followed by a protracted media/legal campaign against google and vice versa.

A small correction: this paper didn't get her fired, her reaction to feedback on this paper got her fired.

Note to all: if you give an employer an ultimatum "do X or I resign", don't be surprised if they accept your resignation.

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#23

Post by visarga »

monkaiju wrote: post #7

The problems with LLM are numerous but whats really wild to me is that even as they get better at fairly trivial tasks the advertising gets more and more out of hand. These machine dont think, and they dont understand, but people like the CEO of OpenAI allude to them doing just that, obviously so the hype can make them money.

> These machine dont think, and they dont understand

But they do solve many tasks correctly, even problems with multiple steps and new tasks for which they got no specific training. They can combine skills in new ways on demand. Call it what you want.

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#24

Post by weeksie »

srvmshr wrote: post #18
weeksie wrote: post #9

This was mostly political guff about environmentalism and bias, but one thing I didn't know was that apparently larger models make it easier to extract training data. > Finally, we note that there are risks associated with the fact that LMs with extremely large numbers of parameters model their training data very closely and can be prompted to output specific information from that training data. For example, [28] dem…

That is sort of understood facts with even models like Copilot & ChatGPT. With the amount of information we are generally churning, all PII may not get scrubbbed. And these LLMs often could be running on unsanitized data - like a cache of Web on Archive.org, Getty images & the likes. I feel this is a unavoidable consequence of using LLM. We cannot ensure all data is free from any markers. I am not a expert on databas…

Copilot has a ton of well publicised examples of verbatim code being used, but I didn't realize that it was as trivial as all that to go plumbing for it directly.

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#25

Post by visarga »

weeksie wrote: post #9

This was mostly political guff about environmentalism and bias, but one thing I didn't know was that apparently larger models make it easier to extract training data. > Finally, we note that there are risks associated with the fact that LMs with extremely large numbers of parameters model their training data very closely and can be prompted to output specific information from that training data. For example, [28] dem…

Is this a good or bad thing? We hear "hallucination" this and that. You can't rely on the LLM. It is not like a search engine. But then you hear on the other side "it memorises PII".

Being able to memorise information is demanded when we want the top 5 countries by population in Europe or the height of Everest. But then we don't want it in other contexts.

Looks more like a dataset pre-processing issue.

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#26

Post by xwn »

platypii wrote: post #6

This paper is embarrassingly bad. It's really just an opinion piece where the authors rant about why they don't like large language models. There is no falsifiable hypothesis to be found in it. I think this paper will age very poorly, as LLMs continue to improve and our ability to guide them (such as with RLHF) improves.

I don't know, without enumerating risks to check, there's little basis for doing due diligence and quelling investors. This massively-cited paper gave a good point of departure for establishing rigorous use of LLMs in the real world. Without that, they're just an unestablished tech with unknown downsides - that's harder to get into true mass acceptance outside the SFBA/tech bubble.

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#27

Post by visarga »

freyr wrote: post #20
xiaolingxiao wrote: post #3

I believe this is the papers that got timnit and mmitchel fired from google, followed by a protracted media/legal campaign against google and vice versa.

I suspect it was Timnit’s behavior after the paper didn’t pass internal review that actually got her fired (issuing an ultimatum and threatening to resign unless the company met her demands; telling her coworkers to stop writing documents because their work didn’t matter; insinuations of racist/misogynistic treatment from leadership when she didn’t get her way).

I think it was a well calculated career move, she wanted fame, she got what she wanted. Now she's leading a new research institute

> We are an interdisciplinary and globally distributed AI research institute rooted in the belief that AI is not inevitable, its harms are preventable, and when its production and deployment include diverse perspectives and deliberate processes it can be beneficial. Our research reflects our lived experiences and centers our communities.

https://www.dair-institute.org/about

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#28

Post by srvmshr »

visarga wrote: post #21
srvmshr wrote: post #14

I am of the general understanding that this paper became less about the LLMs & more of a insinuating hit piece against Alphabet. At least, some of the controversial nuggets got Gebru (and later M Mitchell) fired. From a technical standpoint, there is little new stuff that I found this paper offered in understanding why LLMs can have unpredictable nature, or what degree of data will get exposed by clever hacks (or if…

It was activism masquerading as science. Many researches noted that positives and negatives were not presented in a balanced way. New approaches and efforts were not credited.

I haven't kept track but the activism of the trio could be severe sometimes.

(Anecdotally, I have faced a bite-sized brunt: When discussion surrounding this paper was going on in Twitter, I had mentioned in my timeline (in a neutral tone) that "dust needed to settle to understand what was going wrong". This was unfortunately picked up & RTed by Gebru & the mob responded by name-calling, threatening DMs accusing me of racism/misogyny etc, and one instance of a call to my employer asking to terminate me - all for that one single tweet. I don't want confrontations - not my forte to deal.)

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#29

Post by [deleted] »

[deleted] wrote: post #15
srvmshr wrote: post #14

I am of the general understanding that this paper became less about the LLMs & more of a insinuating hit piece against Alphabet. At least, some of the controversial nuggets got Gebru (and later M Mitchell) fired. From a technical standpoint, there is little new stuff that I found this paper offered in understanding why LLMs can have unpredictable nature, or what degree of data will get exposed by clever hacks (or if…

[deleted]

[deleted]

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#30

Post by weeksie »

visarga wrote: post #25
weeksie wrote: post #9

This was mostly political guff about environmentalism and bias, but one thing I didn't know was that apparently larger models make it easier to extract training data. > Finally, we note that there are risks associated with the fact that LMs with extremely large numbers of parameters model their training data very closely and can be prompted to output specific information from that training data. For example, [28] dem…

Is this a good or bad thing? We hear "hallucination" this and that. You can't rely on the LLM. It is not like a search engine. But then you hear on the other side "it memorises PII". Being able to memorise information is demanded when we want the top 5 countries by population in Europe or the height of Everest. But then we don't want it in other contexts. Looks more like a dataset pre-processing issue.

I think I agree with this take.

Is it conceivable that a model could leak PII that is present but extremely hard to detect in the data set? For example, spread out in very different documents in the corpus that aren't obviously related, but that the model would synthesize relatively easily?

Post reply on HN