Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Viral Tweet About Apple Card Leads to Goldman Sachs Probe

bloomberg.com

Post reply on HN
Print view View original

31–40 of 180 posts

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#31

Post by School-Cotton »

jacquesm wrote: post #20
rndgermandude wrote: post #13

Other than him being very outraged and claiming gender-based discrimination again and again, is there actually anything to suggest the algorithm is biased on the gender axis, as opposed to any other axis? Just to be clear: I am not dismissing the possibility that the algorithm (meaning the training data, really) is gender biased, it just isn't clear from what I have seen in the tweet storm that this is necessarily th…

No, but neither is there anything that would justify giving a spouse such a very different limit especially considering that they file joint tax returns and her credit score is better than his (see article). It would seem that his conclusion is warranted absent evidence to the contrary, the difference is too large to explain in ways that make any sense.

Maybe I am missing something, but wouldn't it make perfect sense if they have dramatically different income?

I think even if two people have their property in common (and if the algorithm even knows about that), it is still not unreasonable to believe that there is a higher probability of the one with higher income paying off his or her loans.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#32

Post by markdown »

raldi wrote: post #8
markdown wrote: post #4

[flagged]

Heinemeier Hansson is his two-word last name. It's customary to include a person's first name and full last time in an article. See also: Eddie van Halen

Thanks. I've very familiar with van and von used like this, but Heinemeier seemed like a middle name.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#34

Post by xenihn »

function_seven wrote: post #19
rndgermandude wrote: post #13

Other than him being very outraged and claiming gender-based discrimination again and again, is there actually anything to suggest the algorithm is biased on the gender axis, as opposed to any other axis? Just to be clear: I am not dismissing the possibility that the algorithm (meaning the training data, really) is gender biased, it just isn't clear from what I have seen in the tweet storm that this is necessarily th…

> is there actually anything to suggest the algorithm is biased on the gender axis, as opposed to any other axis? None of us are allowed to know that. That's the big problem here. They delegate the decision to a black box that cannot be questioned. When the process is set up like that, I think it's fair to assume the worst case scenarios and put the burden on the company to prove otherwise. Adverse inference is a sen…

Black-box algorithms that affect your livelihood are bad, and I don't understand how anyone can be opposed to full transparency.

I don't care what the context is. Whether its credit ratings, job applications, or college admissions.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#35

Post by sgerenser »

rndgermandude wrote: post #13

Other than him being very outraged and claiming gender-based discrimination again and again, is there actually anything to suggest the algorithm is biased on the gender axis, as opposed to any other axis? Just to be clear: I am not dismissing the possibility that the algorithm (meaning the training data, really) is gender biased, it just isn't clear from what I have seen in the tweet storm that this is necessarily th…

I haven't seen any actual evidence other than his single datapoint of him vs. his wife's experience. Of course its possible that the algorithm is designed to give lower credit limits to women, but I find it unlikely. My wife and I have applied for dozens of cards over the past 10+ years and I've consistently found that once you're score is above some reasonable threshold, the biggest predictor of what credit limit you will be offered are the credit limits on the OTHER credit cards reporting to your report. Maybe he already has much higher limits on other cards?

I started using credit cards a few years before my wife, and despite us both having excellent credit the first time she started applying for cards she was getting limits around $1K or $2K while at that point I was around 10x that. But after a few years of getting more cards and requesting limit increases, both of us now have roughly the same (fairly ridiculous) limits across all of our cards. Admittedly I don't have any cards issued by Goldman Sachs bank, but I can't imagine their algorithm would be much different than Amex, Citi, Chase, etc.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#36

Post by matthewmacleod »

jacquesm wrote: post #26
themgt wrote: post #17

I would take DHH's interpretation of this with a large grain of salt. He seems to assume that living in a "community property state" will be taken into account (presumably then considering his wife to have parity with his income/score) by the credit limit algorithm, but we have no way to know that that's the case. He also adds "It gets even worse. Even when she pays off her ridiculously low limit in full, the card wo…

It won't have that line in their code. What it very likely will have is some Bayesian algo or maybe if they're really high tech some ML black box that will have gender as one of its inputs. And it probably shouldn't have that input. The whole idea behind decisions like these is that they have to be explainable and ML especially does not lend itself well for that.

Note that an ML box doesn’t even need gender as an input. Name alone probably gets you 90% of the way there.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#37

Post by ofcourseianal »

zrail wrote: post #14
dghughes wrote: post #9

Hopefully anything linked to gender is looked into for example car insurance? Men typically signifigantly more than women for car insurance.

Car insurers can point at actuarial data to back up their rating decisions.

Maybe GS can too?

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#38

Post by raldi »

markdown wrote: post #32
raldi wrote: post #8

Earlier quoted context omitted.

Heinemeier Hansson is his two-word last name. It's customary to include a person's first name and full last time in an article. See also: Eddie van Halen

Thanks. I've very familiar with van and von used like this, but Heinemeier seemed like a middle name.

Nope. Google his family; they're all HH's.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#39

Post by xenihn »

tootie wrote: post #25

One data point really isn't enough to draw a conclusion and I actually doubt it's true. GS is extremely scrupulous when it comes to avoiding liability and they probably put a ton of diligence into their risk model. It's possible there's some emergent gender bias but we'd need a better show of proof.

What we really need is the ability to force organizations to expose their models.
Post reply on HN