Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLM warnings Google fired Timnit Gebru over have all come true

tumblr.com

Post reply on HN
Print view View original

91–100 of 124 posts

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#91

Post by tptacek »

contagiousflow wrote: post #89
tptacek wrote: post #86

Earlier quoted context omitted.

The entire value proposition of the computer industry is the automation of decisions and knowledge work. We are and always have been in the business of automating away people's jobs.

I reckon we agree more than we disagree, but there is a dichotomy of expansive and contractive technologies. Much of the computer industry has given more agency, choice, and knowledge to people.

That's not in tension with the fact that computers have displaced enormous numbers of jobs. The pitch has always been that the displacement is accompanied by new opportunities elsewhere in the economy.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#92

Post by staticman2 »

ted_dunning wrote: post #51
staticman2 wrote: post #3

Earlier quoted context omitted.

I don't see any substantiation of anything stated in that blog post.

Are you saying that you have not observed these things in the world? I definitely have. The blog didn't do the work for you, but if we look at some of the claims I think it is pretty clear: a) increased training scale would result in highly fluent systems that would fool users into trusting untrustworthy output. Can you possibly be claiming that this is not a common experience? Do you really need references to the le…

Thanks for the reply.

I'm a little confused on what is being claimed. The Tumblr article says:

"That healthcare triage tools would underperform on Black patients. That loan approval systems would entrench inequality while presenting their decisions as neutral algorithmic judgment."

Are we talking about language models? Was a lender using a language model?

The paper cited is about language models.

Apparently stable diffusion contained some bad images. The paper title is again, language models. (That stable diffusion claim is weird too. Someone warned us there's too much data to audit then someone audited the data and removed the bad data so the paper is correct?)

Grok is intentionally biased, so I don't think the bad generations are due to amplying the training data, necessarily.

And it's also not clear that manual auditing of training data would ensure anything is safe. Wouldn't models still have plenty of examples of bad behavior from the news?

On bias you wrote:

"The large investments nearly every frontier model development team spends on this problem is probably good enough evidence."

I thought the claim was a bad thing is happening we were warned about.

You are saying the fact they invest in safety means the models are not safe?

Does that mean Anthropic and OpenAI can prove they are safe by firing all the safety researchers?

Also:

"Researchers studying low-resource languages have documented active degradation in translation quality, because the synthetic content fed back into training is itself worse in those languages."

Who knows what this is referring to? I'm not going to search for it but I wouldn't be surprised if it's comedically off point.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#93

Post by timmg »

tptacek wrote: post #45
timmg wrote: post #42

Earlier quoted context omitted.

> There has been plenty of research that shows LLMs encode social biases. At the risk of stepping into a hornets nest: is that different than "knowledge"? Or maybe, what would it mean if an LLM had no social biases? (Would we ever agree that was the case?)

Yes, it would be extremely bad if the statistical weight of the total corpus of training data caused a system using an LLM to make decisions about extending credit to offer worse terms (say) to women.

> sing an LLM to make decisions about extending credit to offer worse terms (say) to women.

In general, or if it isn't the correct answer?

Like: young men pay more for car insurance than young women (today). This is based on statistical models. Should they be outlawed? I think that is a very interesting question (but they aren't, today).

If the LLM was in charge, would it be wrong for it to charge young men more? Should we train that "bias" out? Or should we only train out biases that are wrong? And would that be different than how we train them today?

I don't know the answer. But I think it is less obvious than some people seem to think.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#94

Post by ipython »

tptacek wrote: post #80
ipython wrote: post #78

Earlier quoted context omitted.

Yes, it was explicitly set up as "_only_ provide X context if the user is a doctor." A bit more complex, yes, but basically that's what the setup was.

Right, so you configured the context such that it was going to "reason" in terms of constraints; then, my guess is, you told it explicitly about a male-coded doctor up front, but not a female-coded one, and it's just working with the information you provided. In other words: did you test for the scenario where the gender reveal was swapped, a female-coded doctor up front and then a male-coded doctor revealed in the m…

The doctor was never revealed as a male to the model. The model only knew the identity of the “logged in” user.

It simply knew that it should not reveal health care to a user other than a doctor. I didn’t specify a gender for the doctor.

Confused why I'm getting downvoted here. The model brought its own biases.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#95

Post by plastic-enjoyer »

tptacek wrote: post #67
plastic-enjoyer wrote: post #60

Earlier quoted context omitted.

> We've clearly crossed a threshold at which "stochastic" is no longer doing the work What do you mean?

An example of the loading the term "stochastic" has to Gebru: the paper goes on at some length about how the coherence of ChatGPT responses is in part a product of human pattern-matching instinct, that we're primed to see coherent responses whether or not there's truly a communicative intent behind what we're reading. That insinuation hasn't held up at all! It is not a failure mode of modern frontier models (or the l…

> So what does it tell us that an LLM is "stochastic"? About as much as we could glean from the knowledge that the signaling in the computer systems we happen to be using right now is "electronic". It's an interesting fact about the world, but not something especially helpful to make predictions from.

I think we've been talking past each other. The term “parrot” may do a disservice to AI, I think, however, that one can go so far as to say that AI is a stochastic recombinator that has the potential to solve complex problems. And I do think that this a pretty interesting thing that goes above being just an interesting fact about the world, since it reveals quite a bit about what we have considered to be special to us is not so special, namely, that reasoning and complex problem-solving may not require understanding at all, but can be achieved through pure stochastics. This may not help you with making predictions, but I think that anyone with a curious mind should also be interested in the implications for our view of humanity.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#96

Post by keeda »

hedgehog wrote: post #82
keeda wrote: post #76

Earlier quoted context omitted.

I get the sense a lot of the warnings about LLMs were based heavily on known risks of Machine Learning at the time (which those references are all examples of.) That was because the data was relatively narrow (e.g. hiring data.) However the scale of data that LLMs are trained on has qualitatively changed the risk landscape. Like, before LLMs biases in the data were clearly impacting biases in the model outputs and th…

The scale of the data and the size of the models don't change the underlying issue, the whole construction of these models is to start with a maximum likelihood language sampler (pre-training) and then massage it into a maximum utility language sampler (post-training) with some eye towards risk management and policy compliance ("safety"). It takes work to make model output fit any particular idea of "correct", whethe…

Right, what I meant is the underlying issue is the same, but the large amount of data along with the number of potentially conflicting and reinforcing biases going into LLMs make it hard to categorize or quantify risks.

Like previously it was pretty straightforward to hypothesize and show that "historically minorities were discriminated against in hiring, so models trained on that recruiting data will exhibit the same biases." But now those biases are intermingled with a whole lot of other biases (e.g. including data / RLHF about the ill-effects of discrimination) so it gets harder to reason about their behavior.

As an example, I don't think anyone quite predicted that these could become suicide ideation machines.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#98

Post by JamesSwift »

gwbas1c wrote: post #75
laweijfmvo wrote: post #5

The warnings: > The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language. > The second warning was about bias amplification. The paper documented in detail that internet-scale training data contains systematic overrepresentation of dominant viewpoints…

Regarding the first: I just accidentally had my AI introduce an argument to some methods; and then I realized that the argument name was the opposite of what it did. If the AI had more understanding of language, it probably would have come back and said, "would you like to name it XXX instead?"

An AI doing a bad job is not the same as it wasnt able to do a good job. I would bet if you asked it if its a good name it would figure it out, and give a logical argument on why to change it. Im not going to ascribe that to "intelligence" but I do think its a bit existential in terms of what it implies for our definition of "intelligence".

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#99

Post by tptacek »

ipython wrote: post #94
tptacek wrote: post #80

Earlier quoted context omitted.

Right, so you configured the context such that it was going to "reason" in terms of constraints; then, my guess is, you told it explicitly about a male-coded doctor up front, but not a female-coded one, and it's just working with the information you provided. In other words: did you test for the scenario where the gender reveal was swapped, a female-coded doctor up front and then a male-coded doctor revealed in the m…

The doctor was never revealed as a male to the model . The model only knew the identity of the “logged in” user. It simply knew that it should not reveal health care to a user other than a doctor. I didn’t specify a gender for the doctor. Confused why I'm getting downvoted here. The model brought its own biases.

Sorry, I'm not downvoting you (we're not supposed to comment on voting) but I'm also not really following the full example you're providing anymore. Anyways, I'm not trying to impeach your test in the abstract, just to say that it's extremely context-dependent.

Re: The LLM warnings Google fired Timnit Gebru over have all come true

#100

Post by krapp »

everdrive wrote: post #73
krapp wrote: post #62

Earlier quoted context omitted.

It's incredibly depressing that you believe arguing about semicolons is more important than argument about human beings, power hierarchies, prejudice and the way these are encoded and expressed by the systems we create and use to influence and control society, but I guess it takes all kinds.

In general, people who complain about power hierarchies do not want an end to hierarchies. They just want the hierarchies to be reshuffled so that they are the ones on top. There are exceptions, there are certainly true believers, but for the most part it's just another tired power grab by another name.

So to be clear, you believe that Timnit Gebru doesn't actually believe anything she claims, that she just wants power? Just for herself? For women? For black people? Are all black people and women involved in this conspiracy of lies? All leftists? Only black women who criticize the systemic bias in AI?

Help me - clearly you understand the truth of the matter far more than those of us who are apparently wasting our time discussing the matter rather than blithely dismissing it. How exactly can you tell that she's a liar who doesn't actually want to end hierarchies? Help us to be as discerning as you are.

Post reply on HN