Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Viral Tweet About Apple Card Leads to Goldman Sachs Probe

bloomberg.com

Post reply on HN
Print view View original

61–70 of 180 posts

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#61

Post by MBCook »

matthewmacleod wrote: post #15
mc32 wrote: post #3

Maybe these CC issuers should just take gender out of the equation and solely base their credit extension on economic and behavioral data points.

That would be the obvious first step and an absolute minimal requirement. The problem is that as these systems get more black-box-y there is little to stop them making predictions that are affected by inferred hidden characteristics. Like, “we don’t take race or ethnicity into account, but our insurance algorithm says that if you have black hair and brown eyes and a name commonly found in black communities and you li…

The chances there is a line in the algorithm that says ‘women get less’ is basically 0%.

It’s got to be inferences like you said. And those are so much harder to spot.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#62

Post by toomuchtodo »

xenihn wrote: post #56
toomuchtodo wrote: post #43

Earlier quoted context omitted.

To regulators, sure. Entirely reasonable. To your average Joe? No. This is already done to prevent redlining when originating mortgages. Disclaimer: Work in financial services in risk management, interface with regulators. Opinions are my own.

What's your argument for opposing full exposure, other than the potential impact on your personal livelihood?

The model is not your property. Do you plan on releasing all of your company’s code on Github gratis? Unlikely.

As long as the model doesn’t violate the law, you have no right to it, unless you’re a regulator. Regulators and oversight bodies get full (audited and logged) access.

And while it’s none of your business, I don’t need my job, or a job at all. I work for projects I enjoy.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#65

Post by taway87 »

slg wrote: post #6

>“Any algorithm, that intentionally or not results in discriminatory treatment of women or any other protected class of people violates New York [and federal] law.” I worry about this a lot with the growing importance of algorithms and machine learning. You can't just not actively program the thing to discriminate and assume that is enough. You have to specifically program it to not discriminate.

So, "credit realism", to match "IQ realism"?

I see this a lot, and the background assumption seems to be "in the real world, minority group A is actually riskier, dumber, or objectively worse in some other way, so in order to comply with anti-discrimination, we have to introduce special cases."

Maybe instead we should start with the assumption that women are NOT riskier, dumber, or objectively worse, and fix the likely bug, instead?

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#66

Post by xtacy »

matthewmacleod wrote: post #36
jacquesm wrote: post #26

Earlier quoted context omitted.

It won't have that line in their code. What it very likely will have is some Bayesian algo or maybe if they're really high tech some ML black box that will have gender as one of its inputs. And it probably shouldn't have that input. The whole idea behind decisions like these is that they have to be explainable and ML especially does not lend itself well for that.

Note that an ML box doesn’t even need gender as an input. Name alone probably gets you 90% of the way there.

That points to the core of the issue. "Fairness" in ML algorithms can be hard to define and assess.

It's easy to say "omit gender from the model", but the real issue here has to do with the _causal_ pathways between your input variables and the output variable.

Since ML mostly works by exploiting correlations between the input and output variables, omitting gender doesn't mean gender's influence is removed. You'll have to omit all the causal pathways from gender -> the output, effectively "d-separating" [1] gender from the output. Whether that's practical or not depends on how well we understand the data generating process.

[1] http://bayes.cs.ucla.edu/BOOK-2K/d-sep.html

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#67

Post by purple_ducks »

The response to codinghorror's "then don't use the Apple Card? Solution seems obvious" is on point:

> This is such a shallow, disappointing take. If we relegate all responsibility for discrimination to the individuals discriminated against, nothing is going to change! Individual action against structural problems is INSUFFICIENT.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#68

Post by rhexs »

jacquesm wrote: post #16

Good for him. Of course if he had been a nobody this would never have gotten traction but typically DHH is found on the right side of issues like these and I commend him for speaking out. This is likely one of very many such instances and let's hope that insurance companies, banks and other financial service providers take note.

Speaking out about what? The guy appears to be claiming that a company is discriminating against women with no evidence aside from a single anecdote. Seems like he's some sort of rich 1%er founder, probably not approved under any sort of standard model the typical applicant will fit into, and thus has a crazy high limit.

Is there anything more to this than typical twitter mock indignation and outrage?

Now, if his wife was enormously wealthy before they met and this is the outcome, I suppose that would raise some eyebrows!

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#69

Post by disiplus »

School-Cotton wrote: post #31
jacquesm wrote: post #20

Earlier quoted context omitted.

No, but neither is there anything that would justify giving a spouse such a very different limit especially considering that they file joint tax returns and her credit score is better than his (see article). It would seem that his conclusion is warranted absent evidence to the contrary, the difference is too large to explain in ways that make any sense.

Maybe I am missing something, but wouldn't it make perfect sense if they have dramatically different income? I think even if two people have their property in common (and if the algorithm even knows about that), it is still not unreasonable to believe that there is a higher probability of the one with higher income paying off his or her loans.

i don't think that income is better then historical data of you paying on time.

i have 3x the pay of my wife but she pays always on time and i get overdue notices because i forget or don't care.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#70

Post by anamexis »

School-Cotton wrote: post #31
jacquesm wrote: post #20

Earlier quoted context omitted.

No, but neither is there anything that would justify giving a spouse such a very different limit especially considering that they file joint tax returns and her credit score is better than his (see article). It would seem that his conclusion is warranted absent evidence to the contrary, the difference is too large to explain in ways that make any sense.

Maybe I am missing something, but wouldn't it make perfect sense if they have dramatically different income? I think even if two people have their property in common (and if the algorithm even knows about that), it is still not unreasonable to believe that there is a higher probability of the one with higher income paying off his or her loans.

Income is generally supplied by the applicant at the time of application, and they presumably both used the same household income number.
Post reply on HN