Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’

time.com

Post reply on HN
Print view View original

81–90 of 111 posts

Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’

#81

Post by dev1n »

ramblenode wrote: post #76

An interesting point touched on in the article is the extent to which feedback from a predictive algorithm contributes to its later predictions being biased. If police are targeting a specific neighborhood because it is flagged as being a high crime area, then there is likely to be a higher arrest rate for innocent people too, i.e. a higher false positive rate. Unless there is a good way of differentiating false and…

For further reading on the topic of fairness in machine learning I would like to point to this [1].

[1]: https://jeremykun.com/2015/07/13/what-does-it-mean-for-an-al...

Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’

#82

Post by GFK_of_xmaspast »

burgreblast wrote: post #62
hood_syntax wrote: post #48

Earlier quoted context omitted.

> So having more cops in a high-crime neighborhood is somehow a bad thing, because the presence of more police actually confirms the fact that there is more of a need of police in this neighborhood? Did you understand what you quoted? The author is saying that citations for petty offences (which practically everyone commits with some non-zero frequency, intentionally or not) have a higher chance of affecting future p…

> petty offences (which practically everyone commits with > some non-zero frequency, This isn't Josef K in Kafka's Der Process. Most people don't have any trouble avoiding arrestable behavior. Lawlessness is not a virtue. >ignorance of suffering Ads for bad for-pay universitites are good to remind us of... suffering? And sorry, but even "awareness of suffering" isn't virtuous either. It's what you DO about what you k…

> arrestable behavior

For example, "breathing while black."

Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’

#84

Post by thaw13579 »

yummyfajitas wrote: post #42
muraiki wrote: post #26

Earlier quoted context omitted.

Big data somehow being unbiased and impartial is a myth. The algorithms are still a product of humans. One notable example is a Google algorithm for labeling objects in photos that tagged black people as gorillas [1]. Now I don't have particular insight into the development process behind this algorithm, but this result may have occurred because the training data used did not include enough photos of black people. An…

What evidence do you have that one specific classification error resulting in a tweetstorm was the result of bias at all? I've been mislabeled as a tank in at least one image processing demo (admittedly, this was well before the era of deep learning). Image classification errors happen. Also, it's a bit strange that you suggest ML algos are racist against Asians. You do know that Asians are wildly overrepresented amo…

There was that twitter chatbot that learned some pretty awful things pretty quickly:

http://www.theverge.com/2016/3/24/11297050/tay-microsoft-cha...

Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’

#85

Post by IncRnd »

ramblenode wrote: post #76

An interesting point touched on in the article is the extent to which feedback from a predictive algorithm contributes to its later predictions being biased. If police are targeting a specific neighborhood because it is flagged as being a high crime area, then there is likely to be a higher arrest rate for innocent people too, i.e. a higher false positive rate. Unless there is a good way of differentiating false and…

One could legitimately say that learning of patterns is the process of bias creation.

Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’

#87

Post by IncRnd »

guelo wrote: post #45
bko wrote: post #40

> [algorithms] decide who gets access to credit and who pays higher insurance premiums, as well as who will receive online advertising for luxury handbags versus who’ll be targeted by predatory ads for for-profit universities. As opposed to what? Subjective decisions made by humans based on biases and personal preferences with no semblance of reason or fairness? > for example, who lives in an area targeted by crime f…

> the presence of more police actually confirms the fact that there is more of a need of police in this neighborhood? If we were to enforce all laws we would need massively more police in every neighborhood. When police are sent to flood a poor neighborhood they end up ticketing poor residents for a bunch of "violations" that they would also find if they went to the rich neighborhoods. But if they ever actually did i…

There have been zero shootings in the area where I live. More police patrols will not increase the number of shootings.

It makes sense for police to be where serious crime occurs.

Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’

#88

Post by yummyfajitas »

thaw13579 wrote: post #75
yummyfajitas wrote: post #33

Earlier quoted context omitted.

This is simply not true. If we train a mortgage engine on redlining data, it will only reproduce the red line if it's predictive. I.e., the algorithm will only reproduce the red line if the red line is statistically valid . I discuss this in detail here, with numerical examples: https://www.chrisstucchio.com/blog/2016/alien_intelligences_... The fact is that machine learning takes biased inputs and produces unbiased…

I appreciate this analysis, but it's important to note that not everyone wants unbiased model output. Historically, redlining was intentional, and it took laws to stop it. A major risk of automating mortgage engines is that redlining could easily be reinstated (in the form of model parameters) in a way that is both hard to detect and easily deniable.

There are two possible reasons for redlining:

1) I hate black people more than I like money, but blacks do pay back their loans, so I'll redline even if it costs me money.

2) I like money and am neutral towards blacks, but blacks are deadbeats so redlining gets me more money.

(There are also 2 other cases. "I love/am neutral towards blacks and they pay back their loans", which results in no disparate impact. Also "I hate blacks and they don't pay back their loans" which is equivalent to (2).)

Statistical algorithms solve (1), yet somehow disparate impact isn't fixed. (2) is unspeakable to the modern left - only racists could say that - yet statistics continually reproduces conclusion (2). So obviously the problem must be with the algorithms.

Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’

#89

Post by qrendel »

ThePhysicist wrote: post #12

There is a lot of interesting research on the societal impact and the (potential) problems caused by the use of large-scale data analysis and artificial intelligence, e.g. by Kate Crawford at MSFT Research. Last year I gave a talk on this at 32C3 where I tried to elucidate some of the problems that can pop up when using data analysis to automate things that were previously done by humans: https://www.youtube.com/watc…

> What we really need therefore (IMHO) is to teach ethics to people that work with data

Studies seem to indicate that studying and teaching ethics does not make you more inclined towards ethical behavior.[1] On top of that, from personal experience in business ethics courses, "ethical" has an extremely suspicious equivalence with "the thing which limits the company's exposure to liability."

[1] http://www.faculty.ucr.edu/~eschwitz/SchwitzPapers/BehEth-14...

Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’

#90

Post by rz2k »

ramblenode wrote: post #76

An interesting point touched on in the article is the extent to which feedback from a predictive algorithm contributes to its later predictions being biased. If police are targeting a specific neighborhood because it is flagged as being a high crime area, then there is likely to be a higher arrest rate for innocent people too, i.e. a higher false positive rate. Unless there is a good way of differentiating false and…

Could that be a personnel training problem? The data in such a case would be saying there is a high number of crime victims, even if that number is correlated with the number of people committing crimes. And unless there's a corpse, the reports generally come from victims, not surveillance of criminals committing acts.

Officers making themselves visible on the street and checking to make sure people are safe and finding out from them where dangers might come from, is different than directly treating everyone on the street like they're potential criminals.

Post reply on HN