Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Emily Bender meant by "stochastic parrots"

spectrum.ieee.org

Post reply on HN
Print view View original

171–180 of 280 posts

Re: What Emily Bender meant by "stochastic parrots"

#171

Post by daishi55 »

dwa3592 wrote: post #37

I paid a bit of attention to this paper and the phrase 'stochastic parrots' when it came out and i thought this was worth saying and doing at that time. their suggestions about financial and environmental costs are worth studying, their concern about carefully evaluating datasets to feed to the model rather than feeding the entire internet is fully justified. so - to everyone saying this was a bad paper; if you have…

My criticism centers on the part of the paper they chose for their title, the “stochastic parrot” metaphor. And my criticism is that if you observe Claude code with opus 4.8 working through an entirely novel problem that nobody has ever worked on before and which certainly wasn’t in its training data, the choice to even metaphorically call them stochastic parrots turned out to be egregiously wrong.

And secondarily, and maybe only partially the authors’ fault, is the enormous tidal wave of morons that this paper minted who plague us with their misunderstandings to this day.

Re: What Emily Bender meant by "stochastic parrots"

#173

Post by PaulDavisThe1st »

naasking wrote: post #156
PaulDavisThe1st wrote: post #136

Earlier quoted context omitted.

I prefer to work the other way around. That is, accept that a lot of human speech (and text) is generated via similar mechanisms to the ones that drive LLMs, but note that there is another kind of behavior - reasoning - which seems to be distinct.

I think you need understanding to reason, but you don't need reasoning to understand. A child understands how to catch a ball without reasoning about forces, air resistance, gravity, etc. I think LLMs understand without reasoning. They've built a large associative network of concepts (a kind of understanding), but we don't yet have a good handle on the process of reasoning using that network.

I don't think it is useful to say that a child "understands how to catch a ball", even though it is something many of us do say quite often.

The child knows how to catch the ball, without understanding. Later, the child learns both reason and physics, and can reason about ball catching in a different way.

I don't think that it is useful to say that LLMs understand anything they say, or that we say to them.

Re: What Emily Bender meant by "stochastic parrots"

#174

Post by throwaway7356 »

Diogenesian wrote: post #123
josh-sematic wrote: post #108

Earlier quoted context omitted.

> Transformer ANNs are dramatically dumber than cockroaches Source?

My source is "none of us have ever seen a robot that can navigate unfamiliar 3D spaces as well as a cockroach." If transformers were capable of the job we would have seen a smart robot by now. But all of our robots are truly mindless compared to the simplest insects. I will change my mind if someone demonstrates such a robot. Absent this demonstration, cockroach-level AI is still an unsolved problem. Given how ignora…

Then one can claim that humans are less intelligent than a goldfish as "none of us have ever seen a human swim as well as a goldfish".

Re: What Emily Bender meant by "stochastic parrots"

#175

Post by thx67 »

simonw wrote: post #135
NeutralCrane wrote: post #105

Earlier quoted context omitted.

What exactly are the facts about AI water usage? I have trouble separating hysteria from reality but most of what I see still claims water usage is enormous

This is a useful piece on that: https://andymasley.substack.com/p/the-ai-water-issue-is-fake

How do you know this doesn't suffer from Gell-Mann Amnesia? The first version had so many glaring errors that have been "corrected" (removed), and I don't have the energy to comb through this one.

I am highly skeptical of layperson debunking like this.

Re: What Emily Bender meant by "stochastic parrots"

#176

Post by thx67 »

guelo wrote: post #170
Aurornis wrote: post #21

> in part because Google fired two of the authors, Timnit Gebru I remember being angry about this situation when I first saw it on social media, until I read the details: This person submitted a list of demands to her employer and said that if they weren’t met, she quit. Google wasn’t going to meet her demands so they considered it acceptance of her resignation. There has been a movement trying to debate whether it w…

Ugh. Ok if you're going to push one-sided propaganda I'll push the other side. Google forced its researchers to retract an already submitted paper because it undermined its strategic and commercial story around large language models. The "we just accepted her resignation" is just a lie. Google made harsh demands with opaque reviewers that made vague objections, and then Jeff Dean moved very quickly to get rid of Gebr…

Dean pulled a sweet Dungeon Master move in "accepting her resignation." She should have made them fire her, esp for ostensibly doing the job she was hired to do.

Re: What Emily Bender meant by "stochastic parrots"

#177

Post by thomastjeffery »

simianwords wrote: post #167
thomastjeffery wrote: post #164

Earlier quoted context omitted.

But the stochastic parrot (LLM) is the world model, isn't it? What's the difference?

Yeah… LLMs clearly already have a world model

I think it's a good distinction to make between having and being, which seems to be what the whole "stochastic parrots" bit was intended to make all along.

It doesn't make sense to say a model is in possession of its self. That's exactly the sort of poetic anthropomorphization that Bender was criticising here, and a good reason to not refer to an LLM as "an AI".

Re: What Emily Bender meant by "stochastic parrots"

#178

Post by andrewla »

delis-thumbs-7e wrote: post #133
andrewla wrote: post #95

Earlier quoted context omitted.

I dunno, man, I looked at that text and I see one word after another. Obviously language and the connection to human thought is more subtle than this; I think we all have a rich inner life. Just from an external perspective we can't observe it; all we can see is the token/phoneme stream. I'm just saying that it's a mistake to try to criticize LLMs on this basis because it's hard to see how the same criticism would no…

If you want to see words form a shape I could point you towards concrete poetry, but I guess there is no point. Joyce wrote Finnegan’s Wake for 17 years and although superficially it seems complete gibberish, trodding through it you find meaning to words that are in no dictionary, sentence structures alien to English, etc. but still you are able to understand it, and perhaps some way the mind that produced it. So I d…

Oh, now I see where we have an actual difference of opinion. I don't think you can deny that even Finnegan's wake proceeds one token at a time; your interpretation of it may require more context or out-of-order interpretation, but that's just as true when observing text in German or Japanese, which have word ordering constraints that are alien to English speakers. How it was written is irrelevant; all we can observe is how it was presented. Of course we can observe each other's inner life, but we do so one token at a time, even if the process of producing each token is done (internally or actively) via a backtracking or zeitgeist approach.

You seem to believe, on a more fundamental level, that LLMs are simply not capable of producing text that has deeper connections to itself or represents abstract thoughts. In my opinion, 99% of text written by humans does not show this, just as 99% of text produced by LLMs does not show this, but both have the capability, and I don't believe that LLMs are constrained in such a way that they can never do this.

Re: What Emily Bender meant by "stochastic parrots"

#179

Post by marshray »

Aurornis wrote: post #91
utopiah wrote: post #52

Earlier quoted context omitted.

True but also... she wasn't a software engineer putting code in production nor a researcher working no the fundamentals of machine learning negotiating a raise. She was part of the "Ethical Artificial Intelligence Team" of what was then, and still is now, one of the corporations World wide spending the largest amount of resources precisely on using AI commercially.

I'm saying the paper itself wasn't a bombshell or even that noteworthy. The reason it got PR and continues to come up was because the authors manufactured this self-inflicted drama around it, not because it was leaking secret revelations that harmed the company.

Never underestimate the power of a catchy title that resonates with the intuitions and preconceptions of people who will never read the paper.

Cf. https://machinelearning.apple.com/research/illusion-of-think...

Re: What Emily Bender meant by "stochastic parrots"

#180

Post by mondrian »

nixon_why69 wrote: post #69
Aurornis wrote: post #21

> in part because Google fired two of the authors, Timnit Gebru I remember being angry about this situation when I first saw it on social media, until I read the details: This person submitted a list of demands to her employer and said that if they weren’t met, she quit. Google wasn’t going to meet her demands so they considered it acceptance of her resignation. There has been a movement trying to debate whether it w…

Her demands included wanting to know the identities of anyone who wanted to comment on her paper, after she had a history of going after people publicly. That's enough right there, nobody should tolerate toxic behavior regardless of whether you agree with the politics. Meanwhile, the paper has 2 points of criticism towards AI. 1 is a bunch of carbon consumption complaints assuming NVIDIA cards with coal-fired power,…

One of the key points of that paper is that the body of written works is biased and those biases will be amplified by compressing that body into LLMs, with outlier data, with low coincidence rate, being suppressed as unreliable - data that is structurally dissimilar to the bulk is noise. Similarly to how PageRank suppressed nodes with low number of edges, probably contributing significantly to the homogenous corporate mall-internet we enjoy today.
Post reply on HN