Live data from Hacker News

‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

unite.ai

51–60 of 91 posts

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#51
post #44

I don't think you need AI. Like in Italy the total of your loans cannot exceed 1/3rd of your monthly salary, as simple as that.

Does this mean most Italians rent their homes for life and a small fraction purchase their homes in cash?

No, we have some of the highest home ownerships in the world.

1/3rd of salary is more than enough to buy you a home. That's enough to get a 200k loan for 30 years while making average salary (1700 net euros).

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#52
post #17

Loan decision is relatively straightforward algorithmic process. I see no reason why AI is needed or wouldn't give similar answers.

Every bank is already using a data driven model for credit decisions too. I am sure that is the major input for the human decision already.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#53

How does "simple AI" compare with an old fashioned small table lookup and a few "if" statements?

Decision trees (the base learner in random forests and gradient boosted decision trees) are effectively nested “if” statements, and predictions from tree ensembles (RF and GBDT) are effectively averages of predictions from those nested “if” statements.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#54
post #44

Earlier quoted context omitted.

Does this mean most Italians rent their homes for life and a small fraction purchase their homes in cash?

No, we have some of the highest home ownerships in the world. 1/3rd of salary is more than enough to buy you a home. That's enough to get a 200k loan for 30 years while making average salary (1700 net euros).

Aha you mean the interest cannot exceed 1/3 of salary, not the principal? That makes more sense.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#55

Title is oddly framed. What is interesting or useful about merely not-quite predicting what a human will do? Do the AI's 5% discrepency picks perform better or worse than the human's picks? The title is worded to suggest the AI can, or is very close, do the human's job, which could totally be so. But it could also be that the AI loses 5% vs the human, and the bank only makes 5% on loans in the first place, and so los…

Absolutely! There is not enough data here to signify the actual performance of the algorithm.

Without more data it's not clear if the algorithm is actually effective - e.g. if the approval rate of these loans is 80% for instance, a formula of "always return true" could be said to be '80% accurate compared to human decisions'.

And if the AI makes the same decisions as a human 95% of the time, but then makes horrific errors the other 5% of the time, it wouldn't be an appropriate replacement for the human. This is the same issue with self driving - it doesn't matter if you make great decisions 99% of the time if 1% of the time you decide to drive into a wall (until you have got your error down to a place where it's safer on aggregate than a human).

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#56

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Expert_system

Difference is that random decision forests learn the rules for themselves, they don't have to be programmed in manually by experts. They're one of the most performant "pre-deep-learning" machine learning models and a sensible baseline for many ML tasks before you go out and buy a $1500 GPU

The ability to program approval rules and understand why decisions have been made (and then tweak the ruleset based on economic/statistical analysis) is a feature for these sort of organizations, not a limitation.

There will be elements of AI which are useful, but ultimately banks will want to know why a certain decision was made, and want to incorporate their own economic calculations and forecasts into the model.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#57

If I run N different machine learning models over the same data, and each has some random error in fitting the objective function, then I pick the one which matches the validation data best, isn't there a danger of picking the one which was "luckiest" with the random errors? Presumably for large N that's a real problem? How do people account for that?

https://en.wikipedia.org/wiki/Cross-validation_(statistics)

Cross validation doesn't solve that problem. As the Wikipedia article says: "The variance of F* can be large.[26][27] For this reason, if two statistical procedures are compared based on the results of cross-validation, the procedure with the better estimated performance may not actually be the better of the two procedures (i.e. it may not have the better value of EF). Some progress has been made on constructing confidence intervals around cross-validation estimates,[26] but this is considered a difficult problem. "

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#58

This headline and article are horrible and misrepresent the problem and the outcome. The paper is about the manual process of re-assigning a credit score on a scale of 1 to 15 based on other customer criteria. Really the fact that this process exists at all shows that their initial credit scoring approach is flawed or too simplistic. The argument of "just replace it with an if statement" does not hold up in this scen…

Plus, systemic risk of a repeatable exploitation is more likely without humans in the loop. Making a bad loan for $1M is bad, but if “attackers” can repeatedly prove until they get a bad risk $1B loan, it becomes business shattering.

Do they remove the human in the loop? That doesn't seem like a smart idea. A model would be good just for suggestions.

Humans are both biased and with high variance (not to mention corruptible), but the algorithm can benefit from much better scrutiny and ensure uniform application of the criteria. If a human overrides, then they got to have a good reason.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#59
post #56

Earlier quoted context omitted.

Difference is that random decision forests learn the rules for themselves, they don't have to be programmed in manually by experts. They're one of the most performant "pre-deep-learning" machine learning models and a sensible baseline for many ML tasks before you go out and buy a $1500 GPU

The ability to program approval rules and understand why decisions have been made (and then tweak the ruleset based on economic/statistical analysis) is a feature for these sort of organizations, not a limitation. There will be elements of AI which are useful, but ultimately banks will want to know why a certain decision was made, and want to incorporate their own economic calculations and forecasts into the model.

Decision forests don't provide that explainability though, how can you interpret averages of hundreds of decision trees?

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#60
post #54

Earlier quoted context omitted.

No, we have some of the highest home ownerships in the world. 1/3rd of salary is more than enough to buy you a home. That's enough to get a 200k loan for 30 years while making average salary (1700 net euros).

Aha you mean the interest cannot exceed 1/3 of salary, not the principal? That makes more sense.

More likely to be based on the payment (principal plus interest) than on the interest alone.
Post reply on HN