Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

unite.ai

Post reply on HN
Print view View original

31–40 of 91 posts

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#31

Post by graycat »

Some more details would be nice.

So a loan can be good or bad. The loan officer can rate it as good or bad. And the software can rate it as good or bad.

To evaluate the situation, would like more than the "95%":

(1) Would like to see the arithmetic that yielded the "95%". (2) Would like to know the rate (probability) of false positives, when the software said the loan was good but it wasn't. (3) Would like to know the rate (probability) of the false negatives when the software said the loan was bad but it was good.

And, really, when the software and the loan officer disagreed, what were the ratings of the applications and when was the software correct and when, the officer correct.

Finally, what was the average cost per mistake for the false positives and for the false negatives. E.g., a false negative could cost the bank some business but a false positive could cost them $millions.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#32

Post by fancyfredbot »

If I run N different machine learning models over the same data, and each has some random error in fitting the objective function, then I pick the one which matches the validation data best, isn't there a danger of picking the one which was "luckiest" with the random errors? Presumably for large N that's a real problem? How do people account for that?

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#33

Post by notahacker »

platers wrote: post #23
Ekaros wrote: post #17

Loan decision is relatively straightforward algorithmic process. I see no reason why AI is needed or wouldn't give similar answers.

Shouldn't the sentence be flipped in that case? I see no reason why expensive human is needed or wouldn't give similar answers.

The absence of human picking up obvious discrepancies the model isn't trained on or doesn't accept as inputs is an order of magnitude or three less expensive than making more bad loans (of the size/rate that aren't already determined by a basic credit check). Also, checking the loan matches the bank's credit criteria isn't the only task a bank employee performs during the course of their employment, which likely includes quite a few the AI is utterly terrible at, like talking to people.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#34

Post by edmundsauto »

DrRavenstein wrote: post #25

This headline and article are horrible and misrepresent the problem and the outcome. The paper is about the manual process of re-assigning a credit score on a scale of 1 to 15 based on other customer criteria. Really the fact that this process exists at all shows that their initial credit scoring approach is flawed or too simplistic. The argument of "just replace it with an if statement" does not hold up in this scen…

Plus, systemic risk of a repeatable exploitation is more likely without humans in the loop. Making a bad loan for $1M is bad, but if “attackers” can repeatedly prove until they get a bad risk $1B loan, it becomes business shattering.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#35

Post by ant6n »

forgingahead wrote: post #6

minimum_credit_score = Loan.joins(:borrower).where("loans.default = true").average("borrowers.credit_score") if loan_applicant.credit_score > minimum_credit_score decision = "approve" else decision = "reject" end

You want to use the average credit scores of all defaulting loans as threshold? That seems really low, you're setting your bank up for a lot of defaults. But then there's a lot of selection bias in your data -- presumably your bank has been denying loans to people with bad scores, so over time you your minimum credit score is the upward inching average inside the cracks between safe and denied loads.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#36

Post by mjburgess »

fancyfredbot wrote: post #32

If I run N different machine learning models over the same data, and each has some random error in fitting the objective function, then I pick the one which matches the validation data best, isn't there a danger of picking the one which was "luckiest" with the random errors? Presumably for large N that's a real problem? How do people account for that?

https://en.wikipedia.org/wiki/Cross-validation_(statistics)

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#37

Post by mjburgess »

Accuracy is almost never the right statistic. Here, profitability is. As soon as you realise the target statistic applies a human-goal-value-weighting to the prediction outcomes, it should be easier to see why ML systems are often unsuited to the task they're set.

Here, those 5% of cases are likely business-ending or business-making: precisely because they arent naively routine.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#38

Post by kqr »

hooande wrote: post #4

This isn't surprising. Most 'AI' comes down to correlating a small number of variables (one or two) with a prediction target. The real benefit of any form of machine learning is detecting functional relationships between variables (ie, "when this AND NOT that OR this"). It's just that these relationships don't provide a real benefit in 99/100 real world use cases. In the case of a loan, if the credit score is high th…

Yup. This has been shown over and over by Meehl and that gang.

> The real benefit of any form of machine learning is detecting functional relationships between variables (ie, "when this AND NOT that OR this"). It's just that these relationships don't provide a real benefit in 99/100 real world use cases.

And the real problem with humans, ironically enough, is that due to narrative fallacy, confirmation bias, etc., humans vastly overvalue the contributions of these special circumstances.

If a dumb 2--3 variable rule predicts your friend will go to the movies in the weekend, you are likely to go, "Yeah, maybe. But she complained of that headache this morning, maybe there's something deeper there that also makes them not want to go to the movies."

In most cases, you'll be wrong to override the dumb rule.

So why do people do this? Well, the original movie-going prediction might only have been a 55 % shot. So 45 % of the time, your friend won't go to the movies, headache or not. But if they don't, you'll think you made the right call considering that headache.

If they do go, you'll end up thinking, "Right, of course. They had the headache but their friend really wanted to go so of course they would endure it."

In other words, you'll think of a way to frame your incorrect call as the right one. (Hindsight bias.)

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#39

Post by kqr »

nikanj wrote: post #14
ssivark wrote: post #2

https://astralcodexten.substack.com/p/heuristics-that-almost...

I was just coming here to post the same link. The 95% accuracy should be compared to the general approval rate - does the fancy AI beat a simple return true?

Yup. I always advocate that the first model one should build is the one that constantly returns the most common answer.

That way, you can compare the more fancy stuff to something to see whether you're really improving.

You also get to evaluate the economic gains of the more sophisticated model against the development, maintenance, and data costs of it, compared to the dumb one.

(Another good baseline is returning a random historic result in proportion to how often it occurs. It sometimes helps against exploit attempts at the expense of data.)

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#40

Post by Brian_K_White »

Title is oddly framed. What is interesting or useful about merely not-quite predicting what a human will do?

Do the AI's 5% discrepency picks perform better or worse than the human's picks?

The title is worded to suggest the AI can, or is very close, do the human's job, which could totally be so.

But it could also be that the AI loses 5% vs the human, and the bank only makes 5% on loans in the first place, and so losing 5% of them is like totally erasing the entire income of the bank (from loans) which makes the AI a complete Hindenberg, rather the opposite of the implication from the title.

Post reply on HN