Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Emily Bender meant by "stochastic parrots"

spectrum.ieee.org

Post reply on HN
Print view View original

51–60 of 280 posts

Re: What Emily Bender meant by "stochastic parrots"

#51

Post by tsimionescu »

lennoff wrote: post #26
otabdeveloper4 wrote: post #12

Earlier quoted context omitted.

LLMs do not match patterns. They predict one statistically most likely token (only one!) given a history of some N previously known tokens.

afaik before the final sampling, every "next" token has a probability, so theoretically it could select the 10 most likely tokens (based on some kind of sampling algorithm), but you'd end up with exponentially many output-sequences, so nobody does that.

I think the point the poster above was making is that it doesn't predict a phrase or anything like that - just the single next token. So all 10 or 1000 or whatever number of tokens you want are each individually candidates for the single next token, not a sequence of 10 or 100 next tokens. If you wanted to create multiple possible seuqneces, you'd then feed each of the 10 tokens to the network in the initial state, and extract the next token (or 10 next tokens) from that one, than revert back and feed another single one of the 10 tokens, etc.

Re: What Emily Bender meant by "stochastic parrots"

#52

Post by utopiah »

Aurornis wrote: post #21

> in part because Google fired two of the authors, Timnit Gebru I remember being angry about this situation when I first saw it on social media, until I read the details: This person submitted a list of demands to her employer and said that if they weren’t met, she quit. Google wasn’t going to meet her demands so they considered it acceptance of her resignation. There has been a movement trying to debate whether it w…

True but also... she wasn't a software engineer putting code in production nor a researcher working no the fundamentals of machine learning negotiating a raise.

She was part of the "Ethical Artificial Intelligence Team" of what was then, and still is now, one of the corporations World wide spending the largest amount of resources precisely on using AI commercially.

Re: What Emily Bender meant by "stochastic parrots"

#53

Post by waffletower »

> when OpenAI imposed ChatGPT on the world...

OpenAI offered ChatGPT to the world. A large, monied cross-section of the world had yet to throw its capital behind the Large Language Model technology that made the ChatBot possible. While it is fair to see AI development now as a global imposition, OpenAI did not have the agency as a 2022 startup to impose on the scale we see now.

Re: What Emily Bender meant by "stochastic parrots"

#54

Post by gpderetta »

NyxWulf wrote: post #47

After having used LLMs for some time now, I don't agree with the concept they are just token generators, unless you think that's all humans are too. The way we test in most schools is just picking the right token. We also give them unique problems that they never saw in their training, which is the nature of programming. I realize they are probabilistic token generator models, but I find it harder and harder to accep…

They are just token generators. It is just that 'just' does a lot of lifting!

Re: What Emily Bender meant by "stochastic parrots"

#55

Post by romaniv »

scarmig wrote: post #17
noduerme wrote: post #2

Five years on, which term do we see as less accurate to describe LLMs? Artificial Intelligence or Stochastic Parrot? I guess it's still an open debate.

Which frame inspires a more productive research program? Which has better predicted the trajectory of capabilities over the past five years?

>Which frame inspires a more productive research program?

This question depends on how you define research productivity. There is close to two hundred AI papers published every weekday. Most of them are about GenAI. Most don't seem to be all thay good. The progress in actual model improvement had mostly stalled. If you interact with the latest "raw" models they display all of the issues we've seen in GPT-3.5, just at a smaller rate. The "amazing gamechanger breakthroughs" I read about on social media every week do not seem to lead anywhere. It's all kind of boring, really.

The new "hotness" in AI is clearly building more and more elaborate harnesses. This is not at all the direction AI boosters have predicted couple years ago.

Personally, I think the "stochastic parrot" mental model is far more useful for science, because it primes people for proper testing, skepticism and researching alternatives. If you want useful AI, you want people working on it being skeptical, not credulous.

Re: What Emily Bender meant by "stochastic parrots"

#56

Post by Sharlin »

delis-thumbs-7e wrote: post #25

> With the octopus thought experiment, I initially had told the story in terms of a dolphin, because dolphins clearly are intelligent animals. My co-author on that paper, Alexander Koller, said it should be an octopus, because first of all, the environment that octopuses live in is much more distinct from where people live. It makes the metaphor more vivid, that the octopus is just feeling these pulses in the cable a…

It's such a tragedy that they're also extremely solitary animals and die shortly after reproducing the first (and only) time.

Almost all other particularly intelligent animals seem to be gregarious, and it's easy to conclude that a social lifestyle tends to select for more intelligence, a sophisticated theory of mind, and so on (I like to think that that's exactly what was responsible for a runaway intelligence explosion in humans). But in the case of cephalopods, there's something else that has been applying selection pressure towards exceptional intelligence.

Re: What Emily Bender meant by "stochastic parrots"

#57

Post by dekhn »

scarmig wrote: post #17
noduerme wrote: post #2

Five years on, which term do we see as less accurate to describe LLMs? Artificial Intelligence or Stochastic Parrot? I guess it's still an open debate.

Which frame inspires a more productive research program? Which has better predicted the trajectory of capabilities over the past five years?

Statistical models have repeatedly shown themselves to be the most productive research method for working with complex human-based systems (and in the larger study of natural phenomena). It remains unclear whether there is any short term path for symbolic methods to catch up and exceed the capabilities of current/near-future statistical systems.

To me the real question begins only once we have a clear example of a non-trivial scientific discovery that is implicit (IE, not an obvious outcome of reading the literature and talking to the experts) and experimentally verifiable. Once that happens- especially if it is a reproducible process (IE, more discoveries) and it's significant (IE, impacts human life and mind in some profound way)- then the onus very much lies on Bender and her coauthors to explain whether we need more than a sufficiently advanced stochastic parrot.

Re: What Emily Bender meant by "stochastic parrots"

#58

Post by xgulfie »

fxwin wrote: post #35
noduerme wrote: post #2

Five years on, which term do we see as less accurate to describe LLMs? Artificial Intelligence or Stochastic Parrot? I guess it's still an open debate.

I think "(intelligent) language understander" is an apt term. It contains within it the fact that these models are mainly trained on text, and "understand" it beyond a simple token-by-token level (i.e. their latent space maps to more and more complex concepts). It also separates them from "world understanders" since any understanding they might have about the world comes from text (or images if we include multimodal…

Understand is a pretty imprecise term. What does it mean for a computer to understand? Does an H264 decoder understand Eraserhead.mkv?

Re: What Emily Bender meant by "stochastic parrots"

#59

Post by andrewla »

paulcole wrote: post #38
noduerme wrote: post #2

Five years on, which term do we see as less accurate to describe LLMs? Artificial Intelligence or Stochastic Parrot? I guess it's still an open debate.

> Stochastic Parrot Nearly all (99%+) people who use this phrase are anti-AI and just looking to show off how much they dislike AI and how clever they can be in insulting it. So it's a great phrase because in just about every case I can ignore what someone says afterwards. Similar to "glorified autocomplete."

At least "glorified autocomplete" is technically accurate, even if vastly underestimating the capability of LLMs. It's just trying to make something very impressive sound trivial.

From an external standpoint, talking to another human, it's like the other human says one word and then says the next word. That's just how language works. Humans look like "glorified autocomplete" from this perspective.

I mean, looking at the time evolution of the state of the universe, one could say that all of physics and creation is "glorified autocomplete" to posit a next state of the universe given current and past state.

Re: What Emily Bender meant by "stochastic parrots"

#60

Post by jeremyjh »

bunderbunder wrote: post #10
noduerme wrote: post #2

Five years on, which term do we see as less accurate to describe LLMs? Artificial Intelligence or Stochastic Parrot? I guess it's still an open debate.

Though, I would point out that where people fall on that seems to correlate very highly with their ability to explain how an attention head works.

Which direction is the correlation?

I don’t think this phrase means what people assume when it’s applied to post trained instruct models - which did not exist when the paper was written.

After RL it is not predicting based on samples of the original corpus - but is also chasing a reward function that does require other features.

There has been a lot of subsequent research that really calls many of the statements in this article into question.

Post reply on HN