Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Emily Bender meant by "stochastic parrots"

spectrum.ieee.org

Post reply on HN
Print view View original

231–240 of 280 posts

Re: What Emily Bender meant by "stochastic parrots"

#231

Post by ToValueFunfetti »

fluoridation wrote: post #214
ToValueFunfetti wrote: post #210

Earlier quoted context omitted.

Fair point, and on its own it would be surprising to learn what "five" means from that sentence. But you can extrapolate- across a billion sentences, there will be "the next sentence has five words"s and "this sentence are grammared wrong" and so on. It would not be at all impossible to ground a world model on pure text for that reason. And 'not impossible' is sufficient to invalidate the paper's argument.

Let me provide a less superficial response, then. >If multimodal models were still stochastic parrots by the original argument, humans would have to be as well; we don't have any way to ground anything beneath sense data Animals don't passively learn from their perceptions, don't have a separation between training and inference, and don't have a prompt-response execution model. Besides its fundamental biology, the gr…

A bird doesn't learn gravity or aerodynamics, it has no 'sense of physics'. It has sensory neural activity that a scientist can show is tied to these things, but you could, at least in theory, falsify the entire experience of the bird. There is nothing in a bird's brain that directly percieves reality. Colors and sounds and textures and so on are all false primitives that don't exist in nature without us, if that's clarifying. And because evolution acts through organisms, its only access to base reality is through their senses. After thousands of years of effort, we have some pretty good models of what reality actually is, but they're still not 'grounded' in the sense you describe.

Re: What Emily Bender meant by "stochastic parrots"

#232

Post by fluoridation »

ToValueFunfetti wrote: post #231
fluoridation wrote: post #214

Earlier quoted context omitted.

Let me provide a less superficial response, then. >If multimodal models were still stochastic parrots by the original argument, humans would have to be as well; we don't have any way to ground anything beneath sense data Animals don't passively learn from their perceptions, don't have a separation between training and inference, and don't have a prompt-response execution model. Besides its fundamental biology, the gr…

A bird doesn't learn gravity or aerodynamics, it has no 'sense of physics'. It has sensory neural activity that a scientist can show is tied to these things, but you could, at least in theory, falsify the entire experience of the bird. There is nothing in a bird's brain that directly percieves reality. Colors and sounds and textures and so on are all false primitives that don't exist in nature without us, if that's c…

>A bird doesn't learn gravity or aerodynamics, it has no 'sense of physics'.

That's not what I said. What I said was that it's physics that provides the ground truth.

>you could, at least in theory, falsify the entire experience of the bird

It wouldn't be a bird anymore, but a dysfunctional cyborg with false perceptions.

>There is nothing in a bird's brain that directly percieves reality.

Yes, of course there is. Animal sensory organs do not produce false information, nor do they provide the brain an interpretation of what they perceive. The brain may fail to distinguish hallucinations from reality, but those are processes internal to the brain. What it gets from the body is raw physical measurements, and what it sends out is raw motor commands.

A multimodal model doesn't have the same direct access to reality, it just has collections of words and images. It has no capacity to determine the reality of a photograph of a sunset or a CGI render of a dragon. The word "real" is itself meaningless to it; they're both real in that they appear in its training corpus. It lacks the capacity to investigate these stimuli in any way, and can just learn to associate different stimuli in arbitrary ways that appear to make sense to us, but nothing else.

Re: What Emily Bender meant by "stochastic parrots"

#233

Post by zzrrt »

yellowapple wrote: post #225
scarmig wrote: post #117

Earlier quoted context omitted.

The hysteria around water usage rests on people not knowing the scale of industrial civilization. First thing to do is compare any estimate of data center water usage with the water usage of almond farming. Or, if you want to focus on individual consumer choices, the water footprint of eating a hamburger.

> Or, if you want to focus on individual consumer choices, the water footprint of eating a hamburger. To drive this point home: if every American ate exactly one less hamburger per year, it would entirely offset the annual water consumption of all US datacenters (including, therefore, the water footprint of AI).

> the annual water consumption of all US datacenters

How many hamburgers will it be if all the proposed/pending datacenters are completed and running?

Re: What Emily Bender meant by "stochastic parrots"

#234

Post by mike_hearn »

nyeah wrote: post #197
mike_hearn wrote: post #188

Earlier quoted context omitted.

The authors were wrong about their core thesis and are now lying about it. That's the only criticism needed. They said, quote: > LMs are not performing natural language understanding (NLU), and only have success in tasks that can be approached by manipulating linguistic form ... which is presented as unarguable fact, yet is untrue. It was obviously wrong at the time it was written and it's been proven wrong in many w…

No, they were correct. In fact an LLM stitches together stuff it observed in its training data. That scales up way better than a lot of us expected, but it's still correct. If you train it on lots of working code, then it's useful for coding. If you trained it primarily on non-working code it would produce nonsense.

It's not correct. Please read some more research papers, this isn't what people working in AI believe at all. You can prove with experiments that different human languages get translated to the same abstract conceptual space in the middle layers, for example. It's why interpretability is so difficult.

The claim is odd in another way: you can train a person on non-working code and they'll produce nonsense. That doesn't mean people are just stitching together words they've previously seen.

Re: What Emily Bender meant by "stochastic parrots"

#235

Post by mike_hearn »

dwa3592 wrote: post #199
mike_hearn wrote: post #188

Earlier quoted context omitted.

The authors were wrong about their core thesis and are now lying about it. That's the only criticism needed. They said, quote: > LMs are not performing natural language understanding (NLU), and only have success in tasks that can be approached by manipulating linguistic form ... which is presented as unarguable fact, yet is untrue. It was obviously wrong at the time it was written and it's been proven wrong in many w…

I read your comment multiple times and sorry to say none of your criticisms make any logical sense in any shape or form whatsoever. >> > LMs are not performing natural language understanding (NLU), and only have success in tasks that can be approached by manipulating linguistic form did you read the paper carefully??? this line is directly cited from another paper (pay attention - it's from july 2020) . the full line…

> this line is directly cited from another paper (pay attention - it's from july 2020)

It's her own paper, she's citing herself. And the full sentence is "As we discuss in §5, LMs are not performing..." so she's actually just teeing up the same claim she's made previously for further discussion. Why are you trying to claim I'm misrepresenting her words?

> did you read the article after that line or was there a shortage of attention span??

I did. It's more of the same, so there's nothing to say about it.

Re: What Emily Bender meant by "stochastic parrots"

#236

Post by fxwin »

xgulfie wrote: post #58
fxwin wrote: post #35

Earlier quoted context omitted.

I think "(intelligent) language understander" is an apt term. It contains within it the fact that these models are mainly trained on text, and "understand" it beyond a simple token-by-token level (i.e. their latent space maps to more and more complex concepts). It also separates them from "world understanders" since any understanding they might have about the world comes from text (or images if we include multimodal…

Understand is a pretty imprecise term. What does it mean for a computer to understand? Does an H264 decoder understand Eraserhead.mkv?

Sure, but that's the nature of language (which is also why i put "understand" in quotation marks. I usually follow it up with "whatever that means" lol) . I think in this case, it carries with it implicit properties (abstract semantic representation) that i think models possess, which a decoder doesn't.

Re: What Emily Bender meant by "stochastic parrots"

#237

Post by kilpikaarna »

a34729t wrote: post #217

If your field (NLP/computational lingustics) has evaporated the next best thing to do is hit the pundit circuit.

No you see, she says right there in here "Frequently Unasked Questions" (lol) that she is in fact not at all bitter about her field being effectively solved by someone else, thank you very much! Her preferred approach is in fact alive and well, more important than ever and still very very interesting. One wonders if her PhD students feel the same.

The authors of the paper make some good points (in it and elsewhere), but I've seen precisely nothing that suggests they aren't awful people who resort to playing the victim card, inciting cancel mobs and in general behaving like the worst stereotypes of "leftist academia" in response to any criticism.

Re: What Emily Bender meant by "stochastic parrots"

#238

Post by [deleted] »

mike_hearn wrote: post #234
nyeah wrote: post #197

Earlier quoted context omitted.

No, they were correct. In fact an LLM stitches together stuff it observed in its training data. That scales up way better than a lot of us expected, but it's still correct. If you train it on lots of working code, then it's useful for coding. If you trained it primarily on non-working code it would produce nonsense.

It's not correct. Please read some more research papers, this isn't what people working in AI believe at all. You can prove with experiments that different human languages get translated to the same abstract conceptual space in the middle layers, for example. It's why interpretability is so difficult. The claim is odd in another way: you can train a person on non-working code and they'll produce nonsense. That doesn'…

[deleted]

Re: What Emily Bender meant by "stochastic parrots"

#239

Post by heresie-dabord »

yellowapple wrote: post #225
scarmig wrote: post #117

Earlier quoted context omitted.

The hysteria around water usage rests on people not knowing the scale of industrial civilization. First thing to do is compare any estimate of data center water usage with the water usage of almond farming. Or, if you want to focus on individual consumer choices, the water footprint of eating a hamburger.

> Or, if you want to focus on individual consumer choices, the water footprint of eating a hamburger. To drive this point home: if every American ate exactly one less hamburger per year, it would entirely offset the annual water consumption of all US datacenters (including, therefore, the water footprint of AI).

> if every American ate exactly one less hamburger per year

I fear that you have unwittingly led us further into the inevitable Environmental Hamburger-Cost Continuum.

Wherein we will use EHCC units to inform the hamburger voters about a) the nobility of even the smallest hamburgerian sacrifice, and b) the gentle, loving hand of industrialisation.

Re: What Emily Bender meant by "stochastic parrots"

#240

Post by josh-sematic »

Diogenesian wrote: post #123
josh-sematic wrote: post #108

Earlier quoted context omitted.

> Transformer ANNs are dramatically dumber than cockroaches Source?

My source is "none of us have ever seen a robot that can navigate unfamiliar 3D spaces as well as a cockroach." If transformers were capable of the job we would have seen a smart robot by now. But all of our robots are truly mindless compared to the simplest insects. I will change my mind if someone demonstrates such a robot. Absent this demonstration, cockroach-level AI is still an unsolved problem. Given how ignora…

If you buy the argument “If A can do X but B can’t, then B is less intelligent than A”, then you have just as strong of an argument that LLMs are more intelligent than cockroaches. When’s the last time you saw a cockroach solve an Erdös problem?
Post reply on HN