Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Emily Bender meant by "stochastic parrots"

spectrum.ieee.org

Post reply on HN
Print view View original

151–160 of 280 posts

Re: What Emily Bender meant by "stochastic parrots"

#151

Post by burnte »

rob74 wrote: post #146
delis-thumbs-7e wrote: post #87

Earlier quoted context omitted.

Well in a sense that monkeys and Great Condor inhabit the exact same environment.

That's a bit exaggerated - monkeys can't fly, but both dolphins and octopuses can swim. I'm aware octopuses prefer to stay at the bottom, while dolphins have to come to the surface regularly to breathe, but for me it's still the same environment...

Tidal basins and ocean bottoms are very different from the open ocean surface. Much like tree tops and 3 miles in the air are different. Both are up in the air, but still quite different, just like "ocean" but still different.

Re: What Emily Bender meant by "stochastic parrots"

#152

Post by burnte »

koverstreet wrote: post #98
TaupeRanger wrote: post #79

Earlier quoted context omitted.

Whether Bender intended it or not, the term has an inherently pejorative sense. "Parroting" is not really indicative of what modern LLMs do. However, when most people bring it up as a criticism of "AI in general" in 2026, they're using it as a pointer to all of the social/environmental criticisms, rather than the technological capabilities.

Yeah, there's some beautiful math underlying what LLMs are doing, and it's the same math our neocortex runs on.

> and it's the same math our neocortex runs on.

The math is based upon theories of how the brain work, but even if those theories are right, this math is a great simplification and subset of what organic brains do.

Re: What Emily Bender meant by "stochastic parrots"

#153

Post by breppp »

arthurjj wrote: post #142
breppp wrote: post #131

Earlier quoted context omitted.

[flagged]

I missed that thread when it came out, it's really a wild read. The difference between how people describe her vs how she's normally portrayed in the media is really startling

People generally get fed the stories they already believe to be true

So that's why "Brave minority woman unfairly fired by evil AI corporation" sells better than "Self-entitled minority woman is terrible bully to colleagues"

Re: What Emily Bender meant by "stochastic parrots"

#154

Post by ainch »

koverstreet wrote: post #118
ainch wrote: post #102

Earlier quoted context omitted.

Could you provide more detail? My understanding is that the neocortex is predominantly focused on forwards simulation, which seems distinct to how transformers operate.

That's fundamental to how anything that compresses/understands the world has to work, in the Kolmogeravian sense. That's why people denigrate LLMs as being just "next token predictors" - they're not wrong, but they're missing the point. Because to do that kind of prediction out in the world you have to build up an accurate model of reality - a model that includes yourself! Which is why we and LLMs are self aware. For…

I'm afraid the precise connection you're making isn't totally obvious to me.

As far as prediction - I mean sure the cortex and LLMs do prediction, but then so can RNNs or diffusion models or any other generative model. Really any ML architecture is learning to compress its environment in pursuit of modelling. More broadly, the predictive brain model would suggest that all of the brain, not just the neocortex, is dedicated to prediction. What would you say makes LLMs similar to the neocortex, rather than the basal ganglia or Broca's area?

Similarly, if you agree with the Manifold Hypothesis, then all machine learning models operate on manifolds. I agree it's an exciting thought, but then I don't know what would distinguish an LLM from a VAE or SVM in terms of operating over a low-dimensional manifold embedded in high dimensional spaces - maybe just scale?

Re: What Emily Bender meant by "stochastic parrots"

#155

Post by viccis »

ayhanfuat wrote: post #33

Here is what Jeff Dean said about the firing at the time: https://docs.google.com/document/d/1f2kYWDXwhzYnq8ebVtuk9CqQ...

She was probably repeating behaviors she learned in academia. These kinds of extremely toxic "don't just apologize for disagreeing with me but also give me the name of every person involved for collective punishment" is a classic move for academic tyrants throwing weight around. The understanding there is that they will then move to cut off every named person from power or access to academic resources. Google did the right thing by protecting their people.

I have watched it happen multiple times that someone from academia joins a research group at a large corporation and finds out to their chagrin that they can't just overtly bully colleagues as easily without tenure.

edit: And it looks like Mitchell immediately left an angry comment about being called a "linguist" despite the article never doing so. Starting to notice a pattern of confrontation, ego, and escalation among some of these people.

Re: What Emily Bender meant by "stochastic parrots"

#156

Post by naasking »

PaulDavisThe1st wrote: post #136
naasking wrote: post #113

Earlier quoted context omitted.

I think it's a mistake to disentangle their abilities from understanding. Just swallow the pill that they have some form of understanding, even if it slightly differs from ours. I really don't see the problem.

I prefer to work the other way around. That is, accept that a lot of human speech (and text) is generated via similar mechanisms to the ones that drive LLMs, but note that there is another kind of behavior - reasoning - which seems to be distinct.

I think you need understanding to reason, but you don't need reasoning to understand. A child understands how to catch a ball without reasoning about forces, air resistance, gravity, etc.

I think LLMs understand without reasoning. They've built a large associative network of concepts (a kind of understanding), but we don't yet have a good handle on the process of reasoning using that network.

Re: What Emily Bender meant by "stochastic parrots"

#157

Post by yubblegum »

rdedev wrote: post #134
agentultra wrote: post #129

Earlier quoted context omitted.

Breaking it down this way is a great way to minimize the numbers so that it appears reasonable. See? Middle-Eastern investors are growing alfalfa in the western desert using legal allotments of water! That is so much worse than what we’re doing! Go after them! They can both be using an egregious amount of water for silly purposes. The other part of the water debate is also the pollution different systems create. Many…

Putting things into perspective is not minimising the problem. We literally have to do this to prioritise where our efforts can be useful. Your argument makes sense if ai datacenters were using something close to like alfafa farming but the difference between them is soo massive it does not make sense. Reducing pollution is a much better problem to fight for

I am guessing (not asserting) that there is a sort of cap on water used for agriculture. It's possible we've already reached it. (?)

So, on the matter of scale: there likely isn't a cap on water use of these datacenters. Both the heat emission and usage levels for these systems will likely go up unless there is a fundamental technical breakthrough.

On the matter of utility: As a sibling of GP mentioned, the utility of food is clear.

On the matter of polution: I am not remotely read on waste water and contamination due to industrial agriculture. Is this also something where the judgmental scale is tipped in favor of food production vs cooling systems?

Re: What Emily Bender meant by "stochastic parrots"

#159

Post by hintymad »

Aurornis wrote: post #21

> in part because Google fired two of the authors, Timnit Gebru I remember being angry about this situation when I first saw it on social media, until I read the details: This person submitted a list of demands to her employer and said that if they weren’t met, she quit. Google wasn’t going to meet her demands so they considered it acceptance of her resignation. There has been a movement trying to debate whether it w…

I'm surprised that people still take Gebru seriously. She is a disgrace to the community because she always, I mean literally always, attacks her critics by motives. You think bias is a data problem? You're a bigot (See her dispute with LeCun). You disagree with my assessment on an ML model? You are white male oppressor (her attacking a Google's SVP). Oh, did I mention that she even said that some loss functions are more racists than others on X?

Gebru is not a researcher. She is a modern-age Trofim Lysenko, who politicizes everything and wields political correctness as a weapon to purge any dissent.

Re: What Emily Bender meant by "stochastic parrots"

#160

Post by benlivengood »

What I look forward to after research like https://arxiv.org/abs/2603.02491, which demonstrate the necessity of world-modeling capability to achieve satisfactory performance on certain goals, is a refractor the SoTA test suites to demonstrate how much world-modeling is necessary in various task distributions.

There have been a few years now of arguments about the level to which transformers do or do not have a world model (v.s. being purely stochastic parrots like early pre-trained LLMs) and now we have some tools to actually make quantifiable determinations.

Post reply on HN