Viral Tweet About Apple Card Leads to Goldman Sachs Probe

I mean, don't you think it's kind of a strawman to assume anyone thinks machines are explicitly sexist? It seems much more likely that whatever method they use to do feature selection on this obviously high-dimensional data just happens to end up picking something that was a proxy for gender.

You can be implicitly biased without being aware of it. This is true for both humans and algorithms.

I've heard anecdotally a number of times that actuaries peg men with higher car insurance rates. I'm only just learning that there's anything like this in the credit system to prevent gender bias. Does something like that protection exist for auto insurance?

As a side note, Idon't really understand the thread's appeal to credit scores here, considering that the TransUnion rating system is supposed to be the inferior one that Apple is looking to replace.

But most of all, I'm shocked by the number of people here outraged by Apple's behavior who will not even be considering switching off their iPhone (Twitter OP included, hell, he even posted a screenshot of his recurring TransUnion payment still being served via Apple Pay). I actually happen to agree with OP, but this blaise attitude towards real customer complaints has kept me from using Apple products for years.

> One data point really isn't enough to draw a conclusion

My wife's credit history is longer and historically her score higher because I used no revolving credit until recently, and not using credit cards counts against you. Also married filing jointly...

For the Apple Card, I was given her limit several times over.

I did notice that the expected limit is shown before the hard pull on credit. This means they've got a pretty good idea before getting the latest credit report.

> GS ... probably put a ton of diligence into their risk model.

To your point about the risk model, I generally think the credit score the bureaus give us is wrong, while I think the decision GS made on the card limits is probably plausible ... if there's some chance or probability we might split up.

For instance, we work in different states, so data patterns might look like we are already separated? I also don't know if she put her income or household income. My income has been several times hers since long before we got together.

Rather than gender bias, I would imagine that given the probability of divorces at a certain age, executive level, and income bracket, that would put a thumb on the scale for ... what if you were not married filing jointly? Who makes more? What cash payment can they carry without going broke? Weighted that way, we should have different limits, and it's not a gender thing. This is the kind of correlation humans might not come up with, but ML probably would.

If this is purely actuarial, the decision might be correct, while not feeling moral.

I haven't read the whole thread, but I am seeing him bitch about her credit score being higher -- from a single agency. There are other agencies which may have different info.

I know nothing about this particular credit card. I'm not as up on credit stuff as I once was (and the world has changed a lot since then). But when I was a homemaker, I had a credit card in my name with a much higher limit than my husband had on any of his cards. That's not exactly the norm.

There are various factors that go into this. He's not wrong to suggest that there is a very big problem with employees having no idea what went wrong. I'm less confident that it is reasonable to infer gender is the entire explanation.

If she applied after him getting his application approved, this might very well explain her tiny line of credit: the algorithm didn't look at him and her alone in isolation, but at both at the same time, saw a certain combined income, saw he already has a large line of credit issued and concluded "I am not giving them another 3000 bucks or whatever, they already got that, so let it be 50 bucks or something, the minimum we can give to a customer, because really they together used all the credit we're willing to extend already when he got his card approved".

If she applied first, she might have gotten the big chunk, and he might have ended up with the tiny chunk.

Or it might be that the algorithm training data was just biased against women, which is entirely possible as well.

The black-box-ness, even to employees, of course, is a huge problem.