Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLM warnings Google fired Timnit Gebru over have all come true

tumblr.com

Post reply on HN
Print view View original

101–110 of 124 posts

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#101

Post by tptacek »

timmg wrote: post #93
tptacek wrote: post #45

Earlier quoted context omitted.

Yes, it would be extremely bad if the statistical weight of the total corpus of training data caused a system using an LLM to make decisions about extending credit to offer worse terms (say) to women.

> sing an LLM to make decisions about extending credit to offer worse terms (say) to women. In general, or if it isn't the correct answer? Like: young men pay more for car insurance than young women (today). This is based on statistical models. Should they be outlawed? I think that is a very interesting question (but they aren't, today). If the LLM was in charge, would it be wrong for it to charge young men more? Sho…

It would obviously be very bad if those decisions were being made based on the statistical weight of the training corpus of a general large language model.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#102

Post by tptacek »

plastic-enjoyer wrote: post #95
tptacek wrote: post #67

Earlier quoted context omitted.

An example of the loading the term "stochastic" has to Gebru: the paper goes on at some length about how the coherence of ChatGPT responses is in part a product of human pattern-matching instinct, that we're primed to see coherent responses whether or not there's truly a communicative intent behind what we're reading. That insinuation hasn't held up at all! It is not a failure mode of modern frontier models (or the l…

> So what does it tell us that an LLM is "stochastic"? About as much as we could glean from the knowledge that the signaling in the computer systems we happen to be using right now is "electronic". It's an interesting fact about the world, but not something especially helpful to make predictions from. I think we've been talking past each other. The term “parrot” may do a disservice to AI, I think, however, that one c…

I don't think we're talking past each other, I just think we're struggling to find a disagreement. All I'm saying is that anti-AI advocates (and the Gebru paper, by implication) refer to the stochasticity of LLMs as a core limitation, and that's a category error.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#103

Post by krapp »

morpheos137 wrote: post #72
krapp wrote: post #62

Earlier quoted context omitted.

It's incredibly depressing that you believe arguing about semicolons is more important than argument about human beings, power hierarchies, prejudice and the way these are encoded and expressed by the systems we create and use to influence and control society, but I guess it takes all kinds.

its incredibly depressing ostensibly intelligent people get depressed about others having different points of view or set up fallacies of the excluded middle / xor fallacies where not warranted.

They aren't expressing a point of view, they're engaging in lazy performative cynicism. It's incredibly depressing so few people here can tell the difference.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#104

Post by whatshisface »

baggy_trough wrote: post #90
whatshisface wrote: post #44

Earlier quoted context omitted.

I had a good laugh when Haiku's thinking summarization referred to mayor Mamdani as a, quote, "known anti-Zionist." :-) Probably a good thing to remember is that the value added in RLHF is not partly biased, or biased, but itself bias. (Context: I asked it to write fake Reddit comments, because I was curious about how realistic they could be. The colorful phrase occurred during its reasoning about the requested subje…

Is there something strange or funny about that?

In English, the word "known" is generally placed in sentences like, "known sympathizer," more often than in "known Democrat." Compare, "suspected," contrast the more neutral, "is an."

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#105

Post by morpheos137 »

krapp wrote: post #103
morpheos137 wrote: post #72

Earlier quoted context omitted.

its incredibly depressing ostensibly intelligent people get depressed about others having different points of view or set up fallacies of the excluded middle / xor fallacies where not warranted.

They aren't expressing a point of view, they're engaging in lazy performative cynicism. It's incredibly depressing so few people here can tell the difference.

kinda meta lol but my incredible depression was performatively cynical.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#106

Post by flashman »

tptacek wrote: post #27
skupig wrote: post #23

Earlier quoted context omitted.

There has been plenty of research that shows LLMs encode social biases. It seems pretty obvious even before looking at the research that training on the whole internet will end up encoding widely-held social biases and stereotypes. https://arxiv.org/pdf/2508.07111 https://github.com/angl1n/social-bias-llm-vlm

Have you read through the sources on that Github link? It's a set of sociology cites establishing that bias exists (something no serious person ever disputed), followed by a couple papers showing mechanistic descriptions of how bias could propagate through an LLM. The paper you call out specifically takes last-generation open-weights models and attempts to trick them into revealing biases through their level of confi…

> Claude and ChatGPT are not indoctrinating into the manosphere users asking about discounted cash flow formulae.

You're defining an extremely narrow case and then saying bias is irrelevant within it. At the risk of Godwin's Law that's kind of like saying it's okay if my accountant is a Nazi as long as they only ever have conversations about accountancy.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#107

Post by tptacek »

flashman wrote: post #106
tptacek wrote: post #27

Earlier quoted context omitted.

Have you read through the sources on that Github link? It's a set of sociology cites establishing that bias exists (something no serious person ever disputed), followed by a couple papers showing mechanistic descriptions of how bias could propagate through an LLM. The paper you call out specifically takes last-generation open-weights models and attempts to trick them into revealing biases through their level of confi…

> Claude and ChatGPT are not indoctrinating into the manosphere users asking about discounted cash flow formulae. You're defining an extremely narrow case and then saying bias is irrelevant within it. At the risk of Godwin's Law that's kind of like saying it's okay if my accountant is a Nazi as long as they only ever have conversations about accountancy.

This reply would make sense if the only words you read in my comment were these 16, but in fact that response to your rebuttal is contained in the sentences adjacent to it in the paragraph.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#108

Post by em-bee »

timmg wrote: post #93
tptacek wrote: post #45

Earlier quoted context omitted.

Yes, it would be extremely bad if the statistical weight of the total corpus of training data caused a system using an LLM to make decisions about extending credit to offer worse terms (say) to women.

> sing an LLM to make decisions about extending credit to offer worse terms (say) to women. In general, or if it isn't the correct answer? Like: young men pay more for car insurance than young women (today). This is based on statistical models. Should they be outlawed? I think that is a very interesting question (but they aren't, today). If the LLM was in charge, would it be wrong for it to charge young men more? Sho…

young men pay more for car insurance than young women (today). This is based on statistical models. Should they be outlawed?

EU has outlawed them. their argument is that differentiation is only valid if the difference is the actual cause and not merely statistical correlation.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#109

Post by jauco »

j16sdiz wrote: post #48

I am not sure what I should think of AI reinforced discrimination. Some sensitive traits (e.g. Race) have high correlation with something we want to estimate (eg crime rate, credit score). The same traits can be correlated with thousands of different other attributes. For example, to estimate the risk of loan default, (mathematically) i can use a) race b) zip code c) 3 or 4 seemingly unrelated attributes, but still h…

Discrimination is just another word for “treating differently”. The discrimination that we generally disallow is the one where it relates to humans and where they are treated differently based on attributes they have no control over. That were either an accident of birth or faith (which is special cased as something you should not put pressure on).

When estinating a loan default, even of 99 people with a purple skin color default on a loan, the hundredth should not be expected to default on the loan just because of the skin color. Both because this is scientifically wrong (it’s not the skin color that causes them to default. There’s a confounding variable) and because it would put someone in a position that they can never get out of.

So the answer to your question is simple: you make a model where the attributes are causal factors for loan default. And you might need to special case attributes that are an accident of birth but that list is finite (listed in the law) and short and generally constructed to exclude strong causal variables.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#110

Post by insane_dreamer »

simonw wrote: post #57

> Amazon's hiring algorithm penalized resumes that contained the word "women" in any context. Healthcare risk scoring algorithms used by major US hospitals were found to systematically underestimate the medical needs of Black patients. Apple Card's credit algorithm gave wives credit lines 10x lower than their husbands for the same financial profile. The Amazon hiring story is from 2018: https://www.reuters.com/articl…

Right, but they were about models, and therefore useful to projections about what even more powerful models (LLMs) might do. The author should have made that clear.
Post reply on HN