Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Viral Tweet About Apple Card Leads to Goldman Sachs Probe

bloomberg.com

Post reply on HN
Print view View original

81–90 of 180 posts

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#81

Post by justapassenger »

slg wrote: post #6

>“Any algorithm, that intentionally or not results in discriminatory treatment of women or any other protected class of people violates New York [and federal] law.” I worry about this a lot with the growing importance of algorithms and machine learning. You can't just not actively program the thing to discriminate and assume that is enough. You have to specifically program it to not discriminate.

Implementation details of law and enforcement matter a lot, but in general, I agree with a spirit of it.

As world depends more and more on algorithms, we need to have more security around them. Especially as algorithms are used a lot as cost cutting, so reaching competent human to appeal error is becoming harder and harder.

If we don’t put more scrutiny around tech, we may end up living in kafkaesque world.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#82

Post by matthewmacleod »

esotericn wrote: post #54

It strikes me that longer term, we're just going to end up playing a cat and mouse game with people continually inventing different categories that businesses aren't permitted to 'discriminate' on. Today it might be gender and race. That makes a lot of sense, because the alternative is to further entrench what are basically inheritances. But aren't we just going to rattle on through and have the algorithms discover (…

That’s not a great argument though - essentially “we can’t prevent an ML model from being biased so we should embrace it.”

Western society has mostly accepted the idea that—outside of some specific cases—we should avoid building systems and processes that systematically discriminate against people on the basis of a selection of characteristics which have historically attracted it. The exact application of this concept, the interpretation of it, and the boundaries of discrimination or protected characteristics will continue to be subject to gray areas and refinement. The rules are not perfect.

But I do think a better solution to the problem (“we have implemented a whizz-bang new technology which is inherently subject to bias”) is to either fix or discard that technology, rather than discarding the concepts of equalities regulation and civil rights.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#83

Post by [deleted] »

xenihn wrote: post #77
toomuchtodo wrote: post #62

Earlier quoted context omitted.

The model is not your property. Do you plan on releasing all of your company’s code on Github gratis? Unlikely. As long as the model doesn’t violate the law, you have no right to it, unless you’re a regulator. Regulators and oversight bodies get full (audited and logged) access. And while it’s none of your business, I don’t need my job, or a job at all. I work for projects I enjoy.

If you're really financially independent, that's even more incentive for you to support exposure and transparency. People are being harmed by these models. You would lose nothing, and others would benefit. >As long as the model doesn’t violate the law, you have no right to it, unless you’re a regulator. If you have no realistic alternative to being judged by a model, you absolutely should have a right to it. Maybe no…

[deleted]

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#84

Post by [deleted] »

slg wrote: post #6

>“Any algorithm, that intentionally or not results in discriminatory treatment of women or any other protected class of people violates New York [and federal] law.” I worry about this a lot with the growing importance of algorithms and machine learning. You can't just not actively program the thing to discriminate and assume that is enough. You have to specifically program it to not discriminate.

[deleted]

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#85

Post by rightbyte »

xtacy wrote: post #66
matthewmacleod wrote: post #36

Earlier quoted context omitted.

Note that an ML box doesn’t even need gender as an input. Name alone probably gets you 90% of the way there.

That points to the core of the issue. "Fairness" in ML algorithms can be hard to define and assess. It's easy to say "omit gender from the model", but the real issue here has to do with the _causal_ pathways between your input variables and the output variable. Since ML mostly works by exploiting correlations between the input and output variables, omitting gender doesn't mean gender's influence is removed. You'll ha…

You can simulate it with artificial querries with the same data except gender and name and see if women get less or more.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#86

Post by dijit »

jyrkesh wrote: post #74
kyrieeschaton wrote: post #57

I guess it worked for him to throw a hissy fit, but there's absolutely no reason to actually believe they're conditioning on gender. It could be "it's a new product, we randomly assign credit limits to see how it affects behavior". It could be "it's a community property state and we're overexposed to this household if we give the second card the same limit as the first". It could, realistically, be almost anything ex…

I've heard anecdotally a number of times that actuaries peg men with higher car insurance rates. I'm only just learning that there's anything like this in the credit system to prevent gender bias. Does something like that protection exist for auto insurance? As a side note, Idon't really understand the thread's appeal to credit scores here, considering that the TransUnion rating system is supposed to be the inferior…

For car insurance? This is incredibly well known and not at all anecdotal. There was even a popular car insurance firm in the UK that only gave women car insurance if they were women called “Sheilas Wheels” which touted lower fees than “other insurance companies that have to also cater to men”.

https://youtu.be/GzNJh1o84-E

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#87

Post by sparkling »

minimaxir wrote: post #2

The original Twitter thread is still being updated, and it's a doozy. This is not an isolated incident. https://twitter.com/dhh/status/1193240508845510656 https://twitter.com/dhh/status/1193242909111398401

Jikes. I appreciate DHH for his open source contributions, but this is straight up embarrassing. The sample size for his claims here is exactly 1. There are probably hundreds of data points that go into the algo that determines the initial credit limit. It could be literally anything, from the number of existing CCs in her name, her credit history, whatever.

Stop crying, stop bitching, grow up.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#88

Post by TAForObvReasons »

dijit wrote: post #86
jyrkesh wrote: post #74

Earlier quoted context omitted.

I've heard anecdotally a number of times that actuaries peg men with higher car insurance rates. I'm only just learning that there's anything like this in the credit system to prevent gender bias. Does something like that protection exist for auto insurance? As a side note, Idon't really understand the thread's appeal to credit scores here, considering that the TransUnion rating system is supposed to be the inferior…

For car insurance? This is incredibly well known and not at all anecdotal. There was even a popular car insurance firm in the UK that only gave women car insurance if they were women called “Sheilas Wheels” which touted lower fees than “other insurance companies that have to also cater to men”. https://youtu.be/GzNJh1o84-E

Even though it wasn't a protected class, GEICO (Government Employees Insurance Company) originally started as an auto insurance company specifically for federal government employees, a pool perceived to be less risky than the driving population

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#89

Post by r99g7 »

And my wife's Apple Card limit is 20x higher than mine. Are they're discriminating against men too‽

If I was as rich as DHH, who makes millions per year, I would cancel my credit cards and refuse to do business with companies like Goldman Sachs entirely.

I wouldn't give a crap about 2% cashback and I don't know why he does.

And despite the fact that he's probably wrong about the entire complaint, the least he could do is cancel his cards in protest. Instead, he seems to have accepted the "bribe" (his word) quite willingly.

Re: Viral Tweet About Apple Card Leads to Goldman Sachs Probe

#90

Post by w4 »

DoreenMichele wrote: post #78
minimaxir wrote: post #2

The original Twitter thread is still being updated, and it's a doozy. This is not an isolated incident. https://twitter.com/dhh/status/1193240508845510656 https://twitter.com/dhh/status/1193242909111398401

I haven't read the whole thread, but I am seeing him bitch about her credit score being higher -- from a single agency. There are other agencies which may have different info. I know nothing about this particular credit card. I'm not as up on credit stuff as I once was (and the world has changed a lot since then). But when I was a homemaker, I had a credit card in my name with a much higher limit than my husband had…

> I haven't read the whole thread, but I am seeing him bitch about her credit score being higher -- from a single agency. There are other agencies which may have different info.

If you read the thread, he was specifically told the Apple Card uses Transunion, which is why they checked with Transunion, and found her score was higher on that report: https://twitter.com/dhh/status/1192945415538106369?s=21

Post reply on HN