Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Emily Bender meant by "stochastic parrots"

spectrum.ieee.org

Post reply on HN
Print view View original

221–230 of 280 posts

Re: What Emily Bender meant by "stochastic parrots"

#221

Post by marshray »

noduerme wrote: post #202
marshray wrote: post #165

Earlier quoted context omitted.

What is " real understanding", and what question can we ask ChatGPT to determine whether it has it?

Ask it how to prevent Spotify from automatically playing every time you get in your car. The answer will involve a bunch of Android settings that don't actually exist, cobbled together from a bunch of bad advice in online forums. Explain to it how it's wrong. Then clear your cache and ask it the same question again from scratch, and get the same garbage. Repeat until it's clear that it doesn't understand anything.

OK, so I asked ChatGPT "How do I prevent Spotify from automatically playing every time I get in my car?"

The answer looks entirely reasonable. I can't check the Android settings, but I was able to confirm that all of the suggested iPhone and Spotify and settings exist. Most of them I had already configured that way, some I didn't know about before.

Re: What Emily Bender meant by "stochastic parrots"

#222

Post by skeledrew »

dekhn wrote: post #116
skeledrew wrote: post #77

Doesn't really matter that much what they're called as long as they're useful, and LLMs (particularly when harnessed) are already ridiculously useful. But it also begs the question: are stochastic parrots useful?

Yes, they are. Likely due to a deep relationship between math and physics, statistical modelling of complex natural phenomena has repeatedly been shown to be the most effective approach. This is true of LLMs, but also of many stochastic (and other) systems.

Useful for regurgitative pattern generation I can get. The core definition though is it doesn't understand what it's generating. Meanwhile I'm seeing LLMs constantly perform tasks which I'd say requires understanding, like reading the manual for a tool it's never encountered before and then going on to effectively use that tool. That's 2 different kinds of useful.

Re: What Emily Bender meant by "stochastic parrots"

#223

Post by ebiederm »

Aurornis wrote: post #21

> in part because Google fired two of the authors, Timnit Gebru I remember being angry about this situation when I first saw it on social media, until I read the details: This person submitted a list of demands to her employer and said that if they weren’t met, she quit. Google wasn’t going to meet her demands so they considered it acceptance of her resignation. There has been a movement trying to debate whether it w…

I think a better summary is that google gave someone the responsibility to ensure AI was developed and used ethically, and didn't give them the power to execute that responsibility.

Apparently google did not even give them the freedom to come to their own conclusions.

I just skimmed through the paper again and what it says seems to hold up well.

The paper is heritical in that it suggests slowing down, using carefully chosen less biased data, and understanding how it all works.

Basically challenging the AI bitter lesson.

I can see how that would meet with a lot of resistance from people who have gone all in on the bitter lesson.

Re: What Emily Bender meant by "stochastic parrots"

#224

Post by dekhn »

skeledrew wrote: post #222
dekhn wrote: post #116

Earlier quoted context omitted.

Yes, they are. Likely due to a deep relationship between math and physics, statistical modelling of complex natural phenomena has repeatedly been shown to be the most effective approach. This is true of LLMs, but also of many stochastic (and other) systems.

Useful for regurgitative pattern generation I can get. The core definition though is it doesn't understand what it's generating. Meanwhile I'm seeing LLMs constantly perform tasks which I'd say requires understanding, like reading the manual for a tool it's never encountered before and then going on to effectively use that tool. That's 2 different kinds of useful.

I'm of the general belief that something can be unaware and not have any understanding (in the subjective experience of consciousness), while also appearing to do so, and be useful.

That's what Bender doesn't get. Some folks tried techniques different from her favorite techniques and made unbelievably fast progress across a wide range of previously insoluble problems (regardless of whether they satisfy properties Bender believes are required for intelligent systems).

I think it's safe to say that none of the main LLMs have some sort of self-awareness as we think of it in humans, but I also expect that more sophisticated systems in the future could. If I had to guess, they would have significantly more activity going on in the network- not just individual end-to-end top-down forward graph, along with cycles instead of trees, and the neurons themselves would be sigifnificantly more capable (effectively little state machines that run functions on input that passes through). I guess also you'd want to have some sort of rules-based (but statistically trained) execution component managing everything.

Re: What Emily Bender meant by "stochastic parrots"

#225

Post by yellowapple »

scarmig wrote: post #117
NeutralCrane wrote: post #105

Earlier quoted context omitted.

What exactly are the facts about AI water usage? I have trouble separating hysteria from reality but most of what I see still claims water usage is enormous

The hysteria around water usage rests on people not knowing the scale of industrial civilization. First thing to do is compare any estimate of data center water usage with the water usage of almond farming. Or, if you want to focus on individual consumer choices, the water footprint of eating a hamburger.

> Or, if you want to focus on individual consumer choices, the water footprint of eating a hamburger.

To drive this point home: if every American ate exactly one less hamburger per year, it would entirely offset the annual water consumption of all US datacenters (including, therefore, the water footprint of AI).

Re: What Emily Bender meant by "stochastic parrots"

#226

Post by yellowapple »

fluoridation wrote: post #204
jimmygrapes wrote: post #192

Earlier quoted context omitted.

I prefer to use the hospital analogy. Locally, the water concerns are a big deal but a lot of people in my community are riled up about the potential need for diesel backup generators because of the noise pollution. They are not wrong and it's good to consider, but they are at this point grasping for reasons that would not (and have not) been concerns for other large footprint projects like hospitals with similar inf…

And you don't understand why what's tolerated of a hospital may not be tolerated of other kinds of buildings?

Here in Reno the anti-datacenter crowd has been extending that tolerance to the local casinos and golf courses (of both of which there are many more than we arguably need). If the choice is between running servers to keep the Internet going v. letting poor schmucks get swindled out of their money on the slots, I'm picking the servers over the slots any day.

Re: What Emily Bender meant by "stochastic parrots"

#227

Post by yellowapple »

thx67 wrote: post #196
simonw wrote: post #185

Earlier quoted context omitted.

Andy has a good track record for writing about this. He shares plenty of credible citations - more so than most other people commenting in this space. He also caught a major error in one of the most widely read books that helped kick off the whole data center water debate: https://blog.andymasley.com/p/empire-of-ai-is-wildly-mislead...

The thing that is off putting about how he uses rhetoric is that it feels like and-you deflection (tu quoque). > Claim that a data center is using 1000x as much water as a city of 88,000 people, where it’s actually using about 0.22x as much water as the city, and only 3% of the municipal water system the city relies on. She’s off by a factor of 4500. This is the single largest error in any popular book that I’ve foun…

> But a data center also using 1/5th of the water consumption of an 88k person city should still be what are debating.

Why, when that's such a minuscule amount of water in the grand scheme of things? Why focus our energy there? Why not spend that energy on the myriad many-orders-of-magnitude-worse offenders?

Re: What Emily Bender meant by "stochastic parrots"

#228

Post by scarmig »

agentultra wrote: post #129
scarmig wrote: post #117

Earlier quoted context omitted.

The hysteria around water usage rests on people not knowing the scale of industrial civilization. First thing to do is compare any estimate of data center water usage with the water usage of almond farming. Or, if you want to focus on individual consumer choices, the water footprint of eating a hamburger.

Breaking it down this way is a great way to minimize the numbers so that it appears reasonable. See? Middle-Eastern investors are growing alfalfa in the western desert using legal allotments of water! That is so much worse than what we’re doing! Go after them! They can both be using an egregious amount of water for silly purposes. The other part of the water debate is also the pollution different systems create. Many…

Everything we do has side effects. Some have more than others; in the case of water, even heavy users of AI have a relatively small AI water footprint compared other things they do.

There are ideological perspectives--another comment in this chain declares that anything that an LLM does has zero value, purely by virtue of being from an LLM. There's not much to discuss there: maybe I find alfalfa useless and LLMs provide a lot of value, maybe you do the opposite. Short of violence, either immediate or delegated, the best way to resolve this conflict is to incorporate costs for resources like water appropriately (accounting for externalities), and let market participants bid to determine who gets access to it.

Datacenter owners are highly likely to find this a reasonable process, because they believe that what they are investing in and operating will provide a great deal of value on the market. And other industries (like your alfalfa) are far more likely to throw a fit at this process.

Or if you think alfalfa farms have fundamental, deep-seated water rights that don't extend to data centers, then data center operators can just purchase water from alfalfa farmers, and everyone involved will eagerly make that trade.

Re: What Emily Bender meant by "stochastic parrots"

#229

Post by ToValueFunfetti »

dwa3592 wrote: post #220
ToValueFunfetti wrote: post #208

Earlier quoted context omitted.

I don't think this tone is at all justified. If you think otherwise, I do ask that you point out where I went too far in a comment that I feared was overburdened by caveats and admissions of my own human flaws. "This sentence has five words" is going to appear far more often than "This sentence has four words". This is the entire premise of LLMs working at all, stochastic parrots or otherwise.

you are right, i was more curt than i should have been. apologies. but you helped prove my point: >>"This sentence has five words" is going to appear far more often than "This sentence has four words". it's not about this at all. your point is about data quality. you need to take a step back. the point is that if you trained a language model just on this data set which has sentences akin to "this sentence has two wor…

If someone substituted all of your sensory inputs for something else for your entire life, how would you notice? If you wore contacts from birth that made the sky red and earbuds that censored when people said it was blue, on what basis would you realize that was wrong? I don't see what this says about the architecture of your brain, and I don't think it's the point being made in the paper. That the training data must statistically connect to reality in order for the model to model reality doesn't seem that important.

Re: What Emily Bender meant by "stochastic parrots"

#230

Post by rfv6723 »

Diogenesian wrote: post #43
jongjong wrote: post #6

The term is not very useful since most humans are stochastic parrots... At least most of the time. Not suggesting that I don't say stuff on autopilot sometimes but for many people, it's their only mode of operation. They never actually think about anything from first principles. Their whole approach to language is just chaining catchphrases together. It's how a toddler thinks; it seems like many people never moved pa…

Humans are not stochastic parrots. You are 100% wrong about toddlers. This was clearly explained by St. Augustine 1500 years ago: Did I not, then, as I grew out of infancy, come next to boyhood, or rather did it not come to me and succeed my infancy? My infancy did not go away (for where would it go?). It was simply no longer present; and I was no longer an infant who could not speak, but now a chattering boy. I reme…

Toddlers do not actually start with a highly advanced "superchimpanzee" mind. Instead, adults project their own mature logic and thinking onto a child’s simple, playful actions.

> "The basic topic of my address today concerns how much of cognition is in the head of the infant and how much in the mind of the theoretician. My general stance is that we are being treated to an interpretive flavor of infant behavior that is much too rich."

—— Who put the cog in infant cognition? Is rich interpretation too costly? https://home.fau.edu/lewkowic/web/Haith_Critique%20of%20Cogn...

Post reply on HN