Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLM warnings Google fired Timnit Gebru over have all come true

tumblr.com

Post reply on HN
Print view View original

61–70 of 124 posts

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#61

Post by tptacek »

ipython wrote: post #53
laweijfmvo wrote: post #5

The warnings: > The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language. > The second warning was about bias amplification. The paper documented in detail that internet-scale training data contains systematic overrepresentation of dominant viewpoints…

When I developed my first red-teaming exercise for breaking AI agents about 12 months ago, I developed a trivial health care app to demonstrate how to prompt inject a model to get it to disclose information it should not (of course, the demonstrated mitigation in the workshop is to secure the data outside of the model's ability to influence/reason, rather than relying on the model to implement access control). I buil…

What context did you set up? Did you set the expectation that it was a reference monitor for security/safety decisions? Did you imply a specific cast of characters, only revealing the existence of a female-coded doctor deep into the context? You can get this kind of result from bias, but you can also get it from implicit search constraint-solving.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#62

Post by krapp »

everdrive wrote: post #47
skupig wrote: post #23

Earlier quoted context omitted.

There has been plenty of research that shows LLMs encode social biases. It seems pretty obvious even before looking at the research that training on the whole internet will end up encoding widely-held social biases and stereotypes. https://arxiv.org/pdf/2508.07111 https://github.com/angl1n/social-bias-llm-vlm

It's incredibly depressing that the concept of "bias" has been shrunken down to solely mean "bad attitudes about an ethnic or gender ground" (and perhaps on the right, "bad attitudes about conservatives") Bias could mean so, so many other things. Was the amyloid hypothesis incorrect? How should we use semicolons? How do you know when meetings waste more time than not? etc. People understand the world via mental short…

It's incredibly depressing that you believe arguing about semicolons is more important than argument about human beings, power hierarchies, prejudice and the way these are encoded and expressed by the systems we create and use to influence and control society, but I guess it takes all kinds.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#63

Post by Fricken »

timmg wrote: post #42
skupig wrote: post #23

Earlier quoted context omitted.

There has been plenty of research that shows LLMs encode social biases. It seems pretty obvious even before looking at the research that training on the whole internet will end up encoding widely-held social biases and stereotypes. https://arxiv.org/pdf/2508.07111 https://github.com/angl1n/social-bias-llm-vlm

> There has been plenty of research that shows LLMs encode social biases. At the risk of stepping into a hornets nest: is that different than "knowledge"? Or maybe, what would it mean if an LLM had no social biases? (Would we ever agree that was the case?)

[dead]

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#64

Post by plastic-enjoyer »

Legend2440 wrote: post #38
plastic-enjoyer wrote: post #33

Earlier quoted context omitted.

Are they, though? I think what LLMs proved is that proving theorems, following instructions and solving complex problems - intelligent behaviour - does not need any kind of understanding, but only ability to recombine things in a stochastic matter. Which basically just means that these things weren't as special as people had thought.

I think you have already decided that LLMs cannot possibly understand. Therefore anything they do must not have required understanding in the first place. It's circular logic.

> I think you have already decided that LLMs cannot possibly understand.

Well, maybe you should stop thinking.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#65

Post by Enginerrrd »

taeric wrote: post #26
wesleywt wrote: post #14

Earlier quoted context omitted.

Why should the person identifying the problem provide a solution? This doesn't make sense.

If the criticism can't distill up from "bad things could happen", it just isn't useful to keep paying people to come up with that kind of critique. And it isn't like we stopped paying attention to these concerns, is it? Nor were they completely blind siding us at the time. The question was largely of what to do about them.

It’s pretty common in the security world to have a red team and a blue team. There is overlap in the skillset for both, but there are good reasons to have separate people develop each team, and we wouldn’t expect people to have a talent for both.

Ideally, we like it if the red team can suggest solutions, but that’s not always their job or expertise and I’ve rarely if ever heard someone express the sentiment you are within that context by suggesting a really good red team person isn’t useful if they can’t fix the holes they find.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#66

Post by anonymousiam »

Why did Darren O'Connor think it was necessary to mention that Timnit Gebru is black? It has no bearing at all on the content. Would it be appropriate for all articles everywhere to mention the race of everybody cited? If not, then why is it okay here?

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#67

Post by tptacek »

plastic-enjoyer wrote: post #60
tptacek wrote: post #39

Earlier quoted context omitted.

We've clearly crossed a threshold at which "stochastic" is no longer doing the work Gebru (and, more importantly, the acolytes of this paper; I shouldn't tar Gebru with what they've done with the work) expected it to do. Lots of important processes are stochastic, including at some levels human thought itself. Advocates who deploy the term "stochastic" seem to believe it impeaches the technology, which is kind of emb…

> We've clearly crossed a threshold at which "stochastic" is no longer doing the work What do you mean?

An example of the loading the term "stochastic" has to Gebru: the paper goes on at some length about how the coherence of ChatGPT responses is in part a product of human pattern-matching instinct, that we're primed to see coherent responses whether or not there's truly a communicative intent behind what we're reading. That insinuation hasn't held up at all! It is not a failure mode of modern frontier models (or the last several generations of models) that they routinely collapse into gibberish revealing the messages they've sent to be meaningless the whole time.

Nonetheless, despite the fact that GPT 4o could reliably solve randomly generated multivariable calculus problems, these systems are at bottom still fundamentally stochastic at least in their kernels (you could have a philosophical debate about how stochastic the entire training process is given how dependent it is on RL). So what does it tell us that an LLM is "stochastic"? About as much as we could glean from the knowledge that the signaling in the computer systems we happen to be using right now is "electronic". It's an interesting fact about the world, but not something especially helpful to make predictions from.

I think Gebru --- or at least, the abstraction of Gebru I formed in my head after reading this one paper --- is probably surprised by that outcome. Surprise is good and healthy! The acolytes, though, who Gebru is not responsible for, are something worse than surprised.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#68

Post by morpheos137 »

laweijfmvo wrote: post #5

The warnings: > The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language. > The second warning was about bias amplification. The paper documented in detail that internet-scale training data contains systematic overrepresentation of dominant viewpoints…

people need to define what "understand" means before they argue about it. example, I as human do not understand what: "The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language," even means outside some circular folk definition of "understand." what does it mean operationally if llm fluency is lacking in "understanding?" if the fluency is deep, context adaptive and general or at least very broad, where is the functional deficit? with regard to affirming bias or median opinion this is probably true with regard to one shot prompts but the the extent rhlf does not constrain the llm to a point of view and to the extent it can adapt its "fluency" to user inputs llms are perfectly capable of generating niche ideological content. Rhlf to the extent it constrains this constrains user freedom.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#69

Post by taeric »

Enginerrrd wrote: post #65
taeric wrote: post #26

Earlier quoted context omitted.

If the criticism can't distill up from "bad things could happen", it just isn't useful to keep paying people to come up with that kind of critique. And it isn't like we stopped paying attention to these concerns, is it? Nor were they completely blind siding us at the time. The question was largely of what to do about them.

It’s pretty common in the security world to have a red team and a blue team. There is overlap in the skillset for both, but there are good reasons to have separate people develop each team, and we wouldn’t expect people to have a talent for both. Ideally, we like it if the red team can suggest solutions, but that’s not always their job or expertise and I’ve rarely if ever heard someone express the sentiment you are w…

Right, but if one of my teams, red or blue, was just saying "the other teams could be flawed", I would probably push for a new makeup for that team?

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#70

Post by tptacek »

Enginerrrd wrote: post #65
taeric wrote: post #26

Earlier quoted context omitted.

If the criticism can't distill up from "bad things could happen", it just isn't useful to keep paying people to come up with that kind of critique. And it isn't like we stopped paying attention to these concerns, is it? Nor were they completely blind siding us at the time. The question was largely of what to do about them.

It’s pretty common in the security world to have a red team and a blue team. There is overlap in the skillset for both, but there are good reasons to have separate people develop each team, and we wouldn’t expect people to have a talent for both. Ideally, we like it if the red team can suggest solutions, but that’s not always their job or expertise and I’ve rarely if ever heard someone express the sentiment you are w…

This is true but it's worth pointing out that the currency of red teaming is the POC, and the authors of the Stochastic Parrots paper don't have one.
Post reply on HN