On a related note, a Princeton professor and some colleagues just published a blog post and a research paper documenting that "language contains human biases, and so will machines trained on language corpora:" https://news.ycombinator.com/item?id=12356111
Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’
31–40 of 111 posts
Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’
#32"she shows how the algorithms [...] are based on exactly the sort of shallow and volatile type of data sets that informed those faulty mortgage models in the run up to 2008" so the fault is of the technology, right?
Well there is a paradox here, if the models were faulty, then why was every bank scrambling to offload the mortgages to someone else? If the models said the mortgages were sound, why didn't the banks want to hold onto them as a cash cow? Could it be the data spoke but no-one wanted to hear what it said?
They were using the best quality models to make the most perfect shit sandwiches. The deli just overproduced.
Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’
#33If we train a mortgage engine on Redlining [0] data, it will then red line. Algorithms are not somehow "fair" because there isn't directly a human in the path. They are as "human as we are" but only proxy and at much lower fidelity. Not to Godwin the thread ... but big data has been used for immoral purposes since well, big data [1]. The even larger danger, is that people who don't know better, will simply "train a m…
This is simply not true. If we train a mortgage engine on redlining data, it will only reproduce the red line if it's predictive. I.e., the algorithm will only reproduce the red line if the red line is statistically valid.
I discuss this in detail here, with numerical examples: https://www.chrisstucchio.com/blog/2016/alien_intelligences_...
The fact is that machine learning takes biased inputs and produces unbiased outputs, and this is a pretty normal occurrence. If I tell you that my algorithm to target advertisements treated `displayedInterest x isMobile` with 15% more weight than `displayedInterest x isDesktop` because it corrected for bias induced by latency on mobile connections, you'd think nothing of it.
Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’
#34Earlier quoted context omitted.
Interesting. Do you have a source?
Here's an example, and the study cited here has been replicated in many American cities.[0] For the petty fines and eroded trust, this is from an advisory letter sent from the Justice Deparment to the judiciary of all fifty states: “Individuals may confront escalating debt; face repeated, unnecessary incarceration for nonpayment despite posing no danger to the community; lose their jobs; and become trapped in cycles…
Your link doesn't work.
Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’
#35"she shows how the algorithms [...] are based on exactly the sort of shallow and volatile type of data sets that informed those faulty mortgage models in the run up to 2008" so the fault is of the technology, right?
Well there is a paradox here, if the models were faulty, then why was every bank scrambling to offload the mortgages to someone else? If the models said the mortgages were sound, why didn't the banks want to hold onto them as a cash cow? Could it be the data spoke but no-one wanted to hear what it said?
Depends on which model you mean.
The banks were holding higher tranches of the securitised mortgage pools. While those pools had traditional levels of defaults, the tranches had their expected value. As the defaults accelerated beyond historic levels, they plummeted in value. In both cases the same model said the instrument had/didn't have value. As these instruments are fair value accounted the fact that they were still bringing in cash didn't help any.
However, as the mortgage market started to go south, the default models underestimated the level of defaults, particularly in the equity release market. This underestimation was endemic in the industry at the time, from the banks to the regulators and ratings agencies.
There is, of course, a very strong argument to say that the ignorance was willful, everyone was on quite a nicer earner and reacted late to the evidence as it started appearing.
So there wasn't really a paradox. The banks were holding tranches while the expected defaults were within historic levels. Once it became clear that that wasn't a good predictor anymore, they tried to limit the effect but it turned out that so many participants were in the same boat that there weren't enough buyers.
I have to say I'm not entirely sure what this has to do with ad analytics but I'm not writing a book on the subject so maybe that's not too surprising.
Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’
#36If we train a mortgage engine on Redlining [0] data, it will then red line. Algorithms are not somehow "fair" because there isn't directly a human in the path. They are as "human as we are" but only proxy and at much lower fidelity. Not to Godwin the thread ... but big data has been used for immoral purposes since well, big data [1]. The even larger danger, is that people who don't know better, will simply "train a m…
This is simply not true. If we train a mortgage engine on redlining data, it will only reproduce the red line if it's predictive. I.e., the algorithm will only reproduce the red line if the red line is statistically valid . I discuss this in detail here, with numerical examples: https://www.chrisstucchio.com/blog/2016/alien_intelligences_... The fact is that machine learning takes biased inputs and produces unbiased…
> This is simply not true. If we train a mortgage engine on redlining data, it will only reproduce the red line if it's predictive. I.e., the algorithm will only reproduce the red line if the red line is statistically valid.
If you train the algorithm to make the same decisions humans did (which is reasonably common when the humans are considered to be effective experts), it will make the same decisions humans did.
If you're comparing outcomes then you'll do better, but even then bias can affect outcomes - if members of group A was sold higher-interest mortgages than equivalent members of group B, then your dataset will show that group B has higher default rates and your algorithm will learn this, even though the difference was only there due to human bias.
Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’
#37Earlier quoted context omitted.
Disallowing the use of race as a signal in machine learning That sounds like a great idea, but it isn't as simple as it appears. Most systems won't have "race" directly encoded as a feature, but that is insufficient. See slide 22 from [1], further in depth discussion discussion from [2] or just this feature (which many systems could automatically discover): Feature6578 = Loc=EastOakland && Income I don't know what th…
The problem is deciding which features are reasonable to include. Yes, location and income can probably predict race to a certain precision, and if you're building a race-detector-by-proxy, you'd want to include those. But what if you're evaluating eligibility for a mortgage? Certainly location and income are critical. A black person in East Oakland with an income below $10k is going to struggle with a mortgage, but…
The problem is actually different.
Race is often very predictive. Blacks don't perform worse simply because they have income http://ftp.iza.org/dp8733.pdf http://www.mindingthecampus.org/2010/09/the_underperformance... https://randomcriticalanalysis.wordpress.com/2015/05/16/on-c... https://randomcriticalanalysis.wordpress.com/2015/11/22/on-t...
If you were right, then the machine learning algorithm would quickly determine that race doesn't matter. Redundant encoding, direct encoding, etc would be irrelevant - the algorithm would ignore it as noise.
The problem is that the algorithm is supposed to uncover hidden patterns that predict loan default. But then there is one specific hidden pattern that is socially and legally taboo to detect, yet also highly predictive.
(Incidentally, if you can solve this problem in lending, it's easily a unicorn startup.)
Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’
#38Earlier quoted context omitted.
This is simply not true. If we train a mortgage engine on redlining data, it will only reproduce the red line if it's predictive. I.e., the algorithm will only reproduce the red line if the red line is statistically valid . I discuss this in detail here, with numerical examples: https://www.chrisstucchio.com/blog/2016/alien_intelligences_... The fact is that machine learning takes biased inputs and produces unbiased…
> This is simply not true. If we train a mortgage engine on redlining data, it will only reproduce the red line if it's predictive. I.e., the algorithm will only reproduce the red line if the red line is statistically valid. If you train the algorithm to make the same decisions humans did (which is reasonably common when the humans are considered to be effective experts), it will make the same decisions humans did. I…
But you don't train the algorithm to make the same decision humans did. You train it to maximize your objective function.
I algorithmically trade the stock market. I don't train my model by asking whether it trades the way I would. The whole point is for it to do a better job than me! I train the model to maximize my returns [1] in backtests and simulations - if it trades differently than I do, so much the better!
[1] More precisely, volatility adjusted returns, and I also penalize model complexity to avoid overfitting.
Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’
#39If we train a mortgage engine on Redlining [0] data, it will then red line. Algorithms are not somehow "fair" because there isn't directly a human in the path. They are as "human as we are" but only proxy and at much lower fidelity. Not to Godwin the thread ... but big data has been used for immoral purposes since well, big data [1]. The even larger danger, is that people who don't know better, will simply "train a m…
This is simply not true. If we train a mortgage engine on redlining data, it will only reproduce the red line if it's predictive. I.e., the algorithm will only reproduce the red line if the red line is statistically valid . I discuss this in detail here, with numerical examples: https://www.chrisstucchio.com/blog/2016/alien_intelligences_... The fact is that machine learning takes biased inputs and produces unbiased…
> The fact is that machine learning takes biased inputs and produces unbiased outputs
That is not necessarily true, because it assumes that the biased inputs have no influence on the outputs.
Suppose the training data is a bank that has historically given higher-interest loans to minorities. Those loans will default more often, and so the algorithm could "correctly" conclude that minorities default more often.
Cause and effect are very hard for an algorithm to distinguish when it's the clustered input variables that are linked together for some opaque reason. The fact is, both statements in this example would be true: "higher interest is correlated with more defaults" and "minority borrowers are correlated with defaults." The algorithm doesn't know one from the other.
Sure, more training data could help the algorithm distinguish between the two, but there may not be enough training data that shows the inverse -- low-interest loans given to minorities -- if the historical data is biased enough.
Re: Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’
#40> [algorithms] decide who gets access to credit and who pays higher insurance premiums, as well as who will receive online advertising for luxury handbags versus who’ll be targeted by predatory ads for for-profit universities.
As opposed to what? Subjective decisions made by humans based on biases and personal preferences with no semblance of reason or fairness?
> for example, who lives in an area targeted by crime fighting algorithms that add more police to his neighborhood because of higher violent crime rates will necessarily be more likely to be targeted for any petty violation, which adds to a digital profile that could subsequently limit his credit, his job prospects, and so on.
So having more cops in a high-crime neighborhood is somehow a bad thing, because the presence of more police actually confirms the fact that there is more of a need of police in this neighborhood?
> this technology was actually siloing people into online gated communities where they no longer had to even acknowledge the existence of the poor,
So targeted advertising to the needs of people is bad because... I can't even make sense of this one.