Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLM warnings Google fired Timnit Gebru over have all come true

tumblr.com

Post reply on HN
Print view View original

81–90 of 124 posts

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#81

Post by tptacek »

skupig wrote: post #59
tptacek wrote: post #27

Earlier quoted context omitted.

Have you read through the sources on that Github link? It's a set of sociology cites establishing that bias exists (something no serious person ever disputed), followed by a couple papers showing mechanistic descriptions of how bias could propagate through an LLM. The paper you call out specifically takes last-generation open-weights models and attempts to trick them into revealing biases through their level of confi…

I'm not really sure what your point is. That was just the most recent paper linked on that repo, which is a convenient list of some relevant papers. There are probably a lot more recent studies, but it does convincingly show that models are still absorbing bias in a way that can affect prediction.

Again: the papers in the repo don't in fact show that about LLMs (I don't doubt that it could be happening).

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#82

Post by hedgehog »

keeda wrote: post #76
simonw wrote: post #57

> Amazon's hiring algorithm penalized resumes that contained the word "women" in any context. Healthcare risk scoring algorithms used by major US hospitals were found to systematically underestimate the medical needs of Black patients. Apple Card's credit algorithm gave wives credit lines 10x lower than their husbands for the same financial profile. The Amazon hiring story is from 2018: https://www.reuters.com/articl…

I get the sense a lot of the warnings about LLMs were based heavily on known risks of Machine Learning at the time (which those references are all examples of.) That was because the data was relatively narrow (e.g. hiring data.) However the scale of data that LLMs are trained on has qualitatively changed the risk landscape. Like, before LLMs biases in the data were clearly impacting biases in the model outputs and th…

The scale of the data and the size of the models don't change the underlying issue, the whole construction of these models is to start with a maximum likelihood language sampler (pre-training) and then massage it into a maximum utility language sampler (post-training) with some eye towards risk management and policy compliance ("safety"). It takes work to make model output fit any particular idea of "correct", whether it's Elon's particular ideology, the US Civil Rights act, Xi Jinping Thought, or writing clean C++. More data and weights increase the complexity of tasks that we're able to model but it doesn't automatically make the output "better" on any given axis.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#83

Post by insane_dreamer »

laweijfmvo wrote: post #5

The warnings: > The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language. > The second warning was about bias amplification. The paper documented in detail that internet-scale training data contains systematic overrepresentation of dominant viewpoints…

> The fourth seems logical, but I'm sure what the impact is, if any.

the impact is that unintended consequences are unknowable since the system can't be properly audited

> The fifth is a problem, I suppose, but one that already exists in so many other capacities.

sure it does, but that doesn't mean that it's also a problem with LLMs and potentially an even greater problem given the potential extensive reach of LLMs into many facets of society

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#84

Post by [deleted] »

krapp wrote: post #62
everdrive wrote: post #47

Earlier quoted context omitted.

It's incredibly depressing that the concept of "bias" has been shrunken down to solely mean "bad attitudes about an ethnic or gender ground" (and perhaps on the right, "bad attitudes about conservatives") Bias could mean so, so many other things. Was the amyloid hypothesis incorrect? How should we use semicolons? How do you know when meetings waste more time than not? etc. People understand the world via mental short…

It's incredibly depressing that you believe arguing about semicolons is more important than argument about human beings, power hierarchies, prejudice and the way these are encoded and expressed by the systems we create and use to influence and control society, but I guess it takes all kinds.

[deleted]

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#85

Post by contagiousflow »

tptacek wrote: post #58
contagiousflow wrote: post #52

Earlier quoted context omitted.

Correct. They will never not have a social bias. Which leads to the question of, who controls these tools, and what biases are they okay/not okay with specifically training for. Currently they can be seen more as a reflection of broader culture (and even that has problems) but as we're already seeing with Grok they can be tuned at a whim to display any specific ideologies.

Those are some of the questions it leads to, but there are other questions that situate agency outside of the labs and in the hands of users, like, what processes do you have set up to backstop automated decisionmaking? It's not interesting to observe that Grok was successfully trained to be an edgelord; anybody paying attention knew that was easily achievable.

> what processes do you have set up to backstop automated decisionmaking?

The companies releasing these models actively encourage the act of automated decision making by them. The entire value proposition is the automation of decisions and knowledge work. It's rare to find a use case for them that isn't offboarding your thinking and therefore agency

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#86

Post by tptacek »

contagiousflow wrote: post #85
tptacek wrote: post #58

Earlier quoted context omitted.

Those are some of the questions it leads to, but there are other questions that situate agency outside of the labs and in the hands of users, like, what processes do you have set up to backstop automated decisionmaking? It's not interesting to observe that Grok was successfully trained to be an edgelord; anybody paying attention knew that was easily achievable.

> what processes do you have set up to backstop automated decisionmaking? The companies releasing these models actively encourage the act of automated decision making by them. The entire value proposition is the automation of decisions and knowledge work. It's rare to find a use case for them that isn't offboarding your thinking and therefore agency

The entire value proposition of the computer industry is the automation of decisions and knowledge work. We are and always have been in the business of automating away people's jobs.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#87

Post by JamesSwift »

laweijfmvo wrote: post #5

The warnings: > The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language. > The second warning was about bias amplification. The paper documented in detail that internet-scale training data contains systematic overrepresentation of dominant viewpoints…

Yeah, personally I score it

1. Disagree

2. Partly agree

3. Agree

4. Agree with you, this doesnt meet my bar of things to be worried about

5. Disagree insomuch as sure the SOTA models will outpace the normies models, but I dont think thats actually an issue. Opus 4.5 is "good enough" if the harness is stable and not hitting weird regressions. So once we reach opus 4.5 levels on self-hostable models (even if self hosting is actually a cloud hosted thing) then Im not concerned. Sure the SOTA will be better, but AI as a normal part of a devs day is able to be satisfied by Opus 4.5 for many years to come.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#88

Post by dlcarrier »

tptacek wrote: post #27
skupig wrote: post #23

Earlier quoted context omitted.

There has been plenty of research that shows LLMs encode social biases. It seems pretty obvious even before looking at the research that training on the whole internet will end up encoding widely-held social biases and stereotypes. https://arxiv.org/pdf/2508.07111 https://github.com/angl1n/social-bias-llm-vlm

Have you read through the sources on that Github link? It's a set of sociology cites establishing that bias exists (something no serious person ever disputed), followed by a couple papers showing mechanistic descriptions of how bias could propagate through an LLM. The paper you call out specifically takes last-generation open-weights models and attempts to trick them into revealing biases through their level of confi…

By design, LLMs follow the heuristic mean. Doing so is, by definition, the opposite of bias, although the meaning of the word has changed to include not following trends, which it doesn't do. Compared to periodicals, an LLM will be slow to change, although pretty much every other form of printed word is even slower to change, with editions of books usually having a cadence of a decade or more.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#89

Post by contagiousflow »

tptacek wrote: post #86
contagiousflow wrote: post #85

Earlier quoted context omitted.

> what processes do you have set up to backstop automated decisionmaking? The companies releasing these models actively encourage the act of automated decision making by them. The entire value proposition is the automation of decisions and knowledge work. It's rare to find a use case for them that isn't offboarding your thinking and therefore agency

The entire value proposition of the computer industry is the automation of decisions and knowledge work. We are and always have been in the business of automating away people's jobs.

I reckon we agree more than we disagree, but there is a dichotomy of expansive and contractive technologies. Much of the computer industry has given more agency, choice, and knowledge to people.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#90

Post by baggy_trough »

whatshisface wrote: post #44
taeric wrote: post #32

Earlier quoted context omitted.

I confess I laughed harder at the Grok comment than I wish I had. Sad to remember that some strawmen are given life and promoted by people. Actively.

I had a good laugh when Haiku's thinking summarization referred to mayor Mamdani as a, quote, "known anti-Zionist." :-) Probably a good thing to remember is that the value added in RLHF is not partly biased, or biased, but itself bias. (Context: I asked it to write fake Reddit comments, because I was curious about how realistic they could be. The colorful phrase occurred during its reasoning about the requested subje…

Is there something strange or funny about that?
Post reply on HN