Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A.I. Doesn't Get Black Twitter

inverse.com

Post reply on HN
Print view View original

31–40 of 105 posts

Re: A.I. Doesn't Get Black Twitter

#31

Post by viraptor »

wRastel27 wrote: post #15

Is this a result of the AI being written by a bunch of non-black people? It seems to me that this problem is more of a reflection of the people working on the code.

I expect it's mostly a reflection on the learning data in language processing. Almost any corpus of text you can easily find for experimenting is close to standard English. Dialects very rarely appear in published texts unless it's just to point out that a specific person being quoted uses that dialect.

Choose a student of any colour. When they need a text corpus for research, do you think they'll reach for a different, known, large text collection that matches their background?

Re: A.I. Doesn't Get Black Twitter

#32

Post by visarga »

ams6110 wrote: post #24
panarky wrote: post #10

Earlier quoted context omitted.

> Isn't the ideal system to not assume anything about the content it's analyzing ... Because Google no longer just searches for literal strings of text, ranked by links. They now infer your meaning and find results that match your intent -- even if the exact search string doesn't exist in the search results. For more on this, read about Hinton's work on "thought vectors" [1]. So if Google can't determine the meaning…

I can't say I would think that Google does a particularly good job at interpreting the intent of my search queries. I often need to reformulate them to get what I'm after. But I doubt that my searches are similar to those of the general public so that is understandable.

I get the same feeling - that Google is often missing the meaning and giving me unrelated keyword matches.

Re: A.I. Doesn't Get Black Twitter

#33

Post by gugagore »

wyager wrote: post #30
panarky wrote: post #5

This is an interesting ethical challenge. If the training corpus for a machine learning model contains stereotypes and biases, then the output of the model will reflect those prejudices. When that model is used in large-scale applications, it will not just repeat those biases, it will amplify them. It propagates biased language, which feeds back into the model in a self-reinforcing vicious cycle of increasing bias. S…

> If the training corpus for a machine learning model contains stereotypes and biases, then the output of the model will reflect those prejudices. Depends what you mean by "the corpus". Let's take the example of loan applications. Maybe the training data contains credit scores, biographical information, and personal essays about the applicants from loan officers. Let's say loan officers tend to use more negative lang…

You are only talking about supervised learning.

And the bias people are probably talking about is bias in the data, not bias in the "input", as you put it.

See http://arxiv.org/abs/1607.06520 for a counterexample to your point.

Re: A.I. Doesn't Get Black Twitter

#34

Post by gugagore »

panarky wrote: post #5

This is an interesting ethical challenge. If the training corpus for a machine learning model contains stereotypes and biases, then the output of the model will reflect those prejudices. When that model is used in large-scale applications, it will not just repeat those biases, it will amplify them. It propagates biased language, which feeds back into the model in a self-reinforcing vicious cycle of increasing bias. S…

Yes! Here's an example: http://arxiv.org/abs/1607.06520

Re: A.I. Doesn't Get Black Twitter

#35

Post by protomyth »

panarky wrote: post #10
bmmayer1 wrote: post #6

Earlier quoted context omitted.

Especially as it relates to this: "This means that blogs or websites that employ African-American language could actually be pushed down in search results because of Google’s language processing." Isn't the ideal system to not assume anything about the content it's analyzing, but to base the model on behavioral patterns instead? In the case of Google ranking, either a site has traffic/low bounce/backlinks/social cred…

> Isn't the ideal system to not assume anything about the content it's analyzing ... Because Google no longer just searches for literal strings of text, ranked by links. They now infer your meaning and find results that match your intent -- even if the exact search string doesn't exist in the search results. For more on this, read about Hinton's work on "thought vectors" [1]. So if Google can't determine the meaning…

"They now infer your meaning and find results that match your intent -- even if the exact search string doesn't exist in the search results."

Bing seems to do the same thing. I really wish there were a grep for the web because both engines get it wrong even when I try to exclude results.

Re: A.I. Doesn't Get Black Twitter

#36

Post by alphonsegaston »

panarky wrote: post #5

This is an interesting ethical challenge. If the training corpus for a machine learning model contains stereotypes and biases, then the output of the model will reflect those prejudices. When that model is used in large-scale applications, it will not just repeat those biases, it will amplify them. It propagates biased language, which feeds back into the model in a self-reinforcing vicious cycle of increasing bias. S…

Since most languages have dialects of similar "standard" form divergence, it's an NLP problem that would have to be addressed anyway. While slang might be hard to keep up with, the grammatical patterns of AAVE are well-documented and understood by linguists, and there's plenty of data to model.

The issue of prejudice stems from refusing to treat AAVE as a valid input to an AI, mirroring the racism in American culture that similarly devalues and excludes it. Output can conform to some approximation of "standard" form without being problematic, following the pattern of interaction between standard and dialect speakers. As a speaker of the standard, it's gauche and derisive to imitate or refuse or comprehend a dialect speaker, not to respond in your own form.

Re: A.I. Doesn't Get Black Twitter

#37

Post by gugagore »

a3n wrote: post #16
wRastel27 wrote: post #15

Is this a result of the AI being written by a bunch of non-black people? It seems to me that this problem is more of a reflection of the people working on the code.

There may be some of that, but I think it has more to do with AI still being young, and it's easier to work with less ambiguous text for learning. That and everyday speech move much faster than formal writing. Now that my son has gone off to college and is less accessible as a resource, I expect to almost totally lose touch with the speech of "these kids today."

Do you think written Standard English is less ambiguous than any other English, or written language? Because I don't see why that would be the case.

Re: A.I. Doesn't Get Black Twitter

#38

Post by WhoBeI »

panarky wrote: post #5

This is an interesting ethical challenge. If the training corpus for a machine learning model contains stereotypes and biases, then the output of the model will reflect those prejudices. When that model is used in large-scale applications, it will not just repeat those biases, it will amplify them. It propagates biased language, which feeds back into the model in a self-reinforcing vicious cycle of increasing bias. S…

Was reading the article and, in true wiki fashion, click the links that seem relevant to get more context. As is often the case I ended up in some other corner of the internet watching interviews with Stephen Pinker. Something in all that input triggered a thought when I read your comment:

What is really the difference between spelling a word wrongly and using a non-dictionary word to express something. We except one as "slang" but will criticize the other because they don't comply with established rules. Isn't that what slang is? A break from established rules.

Anyways, just a thought, because I agree with you; if you publish - proofread. I just find it interesting to throw a Socrates argument at myself from time to time.

Btw and somewhat of topic.. this was the Pinker interview that was most interesting:

https://www.youtube.com/watch?v=egU0dxzFKAQ

Re: A.I. Doesn't Get Black Twitter

#39

Post by yongjik »

bmmayer1 wrote: post #6
panarky wrote: post #5

This is an interesting ethical challenge. If the training corpus for a machine learning model contains stereotypes and biases, then the output of the model will reflect those prejudices. When that model is used in large-scale applications, it will not just repeat those biases, it will amplify them. It propagates biased language, which feeds back into the model in a self-reinforcing vicious cycle of increasing bias. S…

Especially as it relates to this: "This means that blogs or websites that employ African-American language could actually be pushed down in search results because of Google’s language processing." Isn't the ideal system to not assume anything about the content it's analyzing, but to base the model on behavioral patterns instead? In the case of Google ranking, either a site has traffic/low bounce/backlinks/social cred…

> why is Google attempting to 'read' and analyze the content itself?

So that when you search for "cheesecake recipes", you don't get a well-sourced, high-traffic article from a respectable website describing in detail how the new business initiative by Cheesecake Factory is a recipe for disaster.

* Disclaimer: I pulled that example out of thin air. There's actually no such article.

Re: A.I. Doesn't Get Black Twitter

#40

Post by sevenless »

panarky wrote: post #5

This is an interesting ethical challenge. If the training corpus for a machine learning model contains stereotypes and biases, then the output of the model will reflect those prejudices. When that model is used in large-scale applications, it will not just repeat those biases, it will amplify them. It propagates biased language, which feeds back into the model in a self-reinforcing vicious cycle of increasing bias. S…

This is the same question as 'What values should we impose on society?' And that is an irreducible conflict. It's politics.

Programming a computer to perform a task is a powerful way to reveal hidden assumptions. What the problem of machine learning on human data reveals, is there is no objectivity. Our most 'objective' models will simply learn and then reinforce biases and prejudices and cause harm to people. We can't build objective, neutral, value-free models when it comes to human behavior, because humans change their behavior in response to the models. When we reject stereotypes we are imposing a set of values, just as much as when we embrace them. Machine learning forces us to confront the fact that this is exactly what we are doing.

There's a kind of philosophical crisis going on here. We need an entirely new way to think about the limits of objectivity in human sciences, and how to create ethical models in the presence of feedback between science and society. The language of objective physical science doesn't work.

Post reply on HN