Live data from Hacker News

The LLM warnings Google fired Timnit Gebru over have all come true

tumblr.com

1–10 of 124 posts

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#5
The warnings:

  > The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language.

  > The second warning was about bias amplification. The paper documented in detail that internet-scale training data contains systematic overrepresentation of dominant viewpoints and underrepresentation of marginalized ones. The models would not just absorb this bias. They would amplify it...

  > The third warning was about environmental cost.

  > The fourth warning was about documentation. The paper argued that the training datasets being assembled were too large for anyone to actually audit.

  > The fifth warning was the one Google cared about most. Bender and Gebru argued that the deployment of these systems would centralize linguistic and cultural power in the hands of the small number of companies that could afford to train them.

Personally I'm not convinced on the first two. The third is obviously a concern. The fourth seems logical, but I'm sure what the impact is, if any. The fifth is a problem, I suppose, but one that already exists in so many other capacities.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#8

The warnings: > The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language. > The second warning was about bias amplification. The paper documented in detail that internet-scale training data contains systematic overrepresentation of dominant viewpoints…

More than not being entirely sure what the impact is, I don't see any suggestion at what to do about it?

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#9
This paper has not held up, like, at all. The first half of it recites Woke 1.0 principles, like a concern that LMs will thwart efforts to "decolonialize education by shifting to oral histories" in order to avoid the biases of "text". The second half of it makes predictions from axioms about LMs not truly understanding text that nobody would take seriously today.

There's philosophical grappling to be done, as with the Ted Chiang post on the front page right now, about what it is LLMs are actually doing (I'm mostly with Chiang on those core philosophical issues). But Gebru went way past that, attacking their underlying utility. The coherency of GPT 5.5 responses are not simply tricks of the mind, and frontier models (leaving aside Grok, if you want to call it a frontier model) have not in fact been engines for bias.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#10

The warnings: > The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language. > The second warning was about bias amplification. The paper documented in detail that internet-scale training data contains systematic overrepresentation of dominant viewpoints…

[flagged]
Post reply on HN