Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

unite.ai

Post reply on HN
Print view View original

61–70 of 91 posts

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#61

Post by sokoloff »

ant6n wrote: post #35
forgingahead wrote: post #6

minimum_credit_score = Loan.joins(:borrower).where("loans.default = true").average("borrowers.credit_score") if loan_applicant.credit_score > minimum_credit_score decision = "approve" else decision = "reject" end

You want to use the average credit scores of all defaulting loans as threshold? That seems really low, you're setting your bank up for a lot of defaults. But then there's a lot of selection bias in your data -- presumably your bank has been denying loans to people with bad scores, so over time you your minimum credit score is the upward inching average inside the cracks between safe and denied loads.

It depends heavily on the interest rate as well. If the interest rate is high enough, you can show a profit even with a fair number of loans sent to collections.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#62

Post by IgorPartola »

sdoering wrote: post #15
hooande wrote: post #4

This isn't surprising. Most 'AI' comes down to correlating a small number of variables (one or two) with a prediction target. The real benefit of any form of machine learning is detecting functional relationships between variables (ie, "when this AND NOT that OR this"). It's just that these relationships don't provide a real benefit in 99/100 real world use cases. In the case of a loan, if the credit score is high th…

When dealing with stakeholders in big corporations I learned that they absolutely needed AI, machine learning and data science as capabilities in the projects that they started. Even if data was only very tangentially related to the core product/project. If so, for example one needed to datascience the shit out of project metadata to create a project efficiency self optimization loop. Or what ever crap one could come…

I am unclear as to whether what you are saying is that they need data science to justify their projects because when they say that an AI told them that they need $X extra for the project it carries more weight than when they say they did the analysis by hand or if the data science actually provided value in this case.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#63

Post by jacquesm »

sdoering wrote: post #29
roenxi wrote: post #24

Earlier quoted context omitted.

Yes. But there is also an interesting failure mode - people don't bother applying if they don't think they'll get accepted. So presumably even just a "YES" will be a fairly good model of what actually happens without necessarily capturing the real model that the bank is using. A 95% accuracy rate isn't necessarily very good because of this effect (it might be, it just isn't certain).

My SO used to work in a bank in the business loan securities back office. Believe me. Lots of people and businesses apply even if they themselves see no realistic chance of receiving a loan. And in private non business customers there are even a significant amount that have less than nothing to their name, living of welfare and still don't see, why they should not receive a loan from the bank for the newest iPhone. O…

Excuse my snarkiness, but having actually worked for a bank and seeing the inside of the loan department from an IT perspective, the bulk of that translates into: banks will allow the people that are already wealthy to leverage to make them even more wealthy while working hard to keep those already down firmly in their place. Not to mention to take advantage of them by charging them exorbitant fees for the little bit of credit that is extended to them.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#64

Post by jsemrau »

Brian_K_White wrote: post #40

Title is oddly framed. What is interesting or useful about merely not-quite predicting what a human will do? Do the AI's 5% discrepency picks perform better or worse than the human's picks? The title is worded to suggest the AI can, or is very close, do the human's job, which could totally be so. But it could also be that the AI loses 5% vs the human, and the bank only makes 5% on loans in the first place, and so los…

It takes a long while to validate this statistically though as default cohorts move through the lending cycle.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#65

Post by mvc »

sdoering wrote: post #29
roenxi wrote: post #24

Earlier quoted context omitted.

Yes. But there is also an interesting failure mode - people don't bother applying if they don't think they'll get accepted. So presumably even just a "YES" will be a fairly good model of what actually happens without necessarily capturing the real model that the bank is using. A 95% accuracy rate isn't necessarily very good because of this effect (it might be, it just isn't certain).

My SO used to work in a bank in the business loan securities back office. Believe me. Lots of people and businesses apply even if they themselves see no realistic chance of receiving a loan. And in private non business customers there are even a significant amount that have less than nothing to their name, living of welfare and still don't see, why they should not receive a loan from the bank for the newest iPhone. O…

> why they should not receive a loan from the bank for the newest iPhone

Excuse my snarkiness but as someone who's been poor and needed a phone, I'd have got a loan to buy a 2nd hand phone if anyone would've given me one. Trouble is, both the companies providing the phones, and those providing the loans seem to prefer that I get the brand new one.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#66

Post by mjburgess »

fancyfredbot wrote: post #57
mjburgess wrote: post #36

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Cross-validation_(statistics)

Cross validation doesn't solve that problem. As the Wikipedia article says: "The variance of F* can be large.[26][27] For this reason, if two statistical procedures are compared based on the results of cross-validation, the procedure with the better estimated performance may not actually be the better of the two procedures (i.e. it may not have the better value of EF). Some progress has been made on constructing conf…

Well the historical data {(x, y)...} is assumed to be distributed according to the true distribution, such that y = t(x) where t is the true function which maps x to y. Of course, in many situations, no such function exists (ie., there are genuinely ambigious xs, such that t(x) cannot produce a single y -- consider an ambigous cat/dog picture).

If we sweep models f1,...fn across the validation set ... and choose max() of scores() of f1..fn on V, we get f*.

Now if your issue is that f* might be an "unlucky draw", you're right. But there is no statistical way of fixing this -- if we know (via experiment, etc.) what the true distribution is, we can meaasure |f* - t| -- but if we knew this, we wouldnt bother finding f*.

If you want a mechanism to mitigate these problems, there is one main one: the scientific method. To test whether f* poorly reflects t, go and do some experiemnts. If you can't, then you wont know.

(Hence: there is no way of doing science via "mere statistics". It is the experimental conditions constructed by concept-laden, in-the-world, experimenters which are able to obtain the sequence of datasets needed to give confidence to any given model. ML is therefore not able to know anything, its "conclusions" enterily derivative-of, and limited-by, human experimentation. The intelligence occurs in the experimental design, when that's done, everything else is "stamp collecting").

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#67

Post by lumost »

I had an uncle who worked on a rural independent regional bank. While this was a while ago, the bank could and would use a variety of factors including references, propensity towards substance issues and others to make a loan.

I wouldn’t doubt that if you sat down, and created a data set/data collection scheme to gather this data you could make an algorithm to closely mirror the outcomes of a loan decision. However as a human, the loan officer might simply add a new criteria as desired.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#69

Post by sdoering »

mvc wrote: post #65
sdoering wrote: post #29

Earlier quoted context omitted.

My SO used to work in a bank in the business loan securities back office. Believe me. Lots of people and businesses apply even if they themselves see no realistic chance of receiving a loan. And in private non business customers there are even a significant amount that have less than nothing to their name, living of welfare and still don't see, why they should not receive a loan from the bank for the newest iPhone. O…

> why they should not receive a loan from the bank for the newest iPhone Excuse my snarkiness but as someone who's been poor and needed a phone, I'd have got a loan to buy a 2nd hand phone if anyone would've given me one. Trouble is, both the companies providing the phones, and those providing the loans seem to prefer that I get the brand new one.

I am not talking about reasonable needs here. I am talking about people who are Cleary feeling entitled to receive loans they know the can never repair and would not even be willing to try.

I myself had to live on 345 Euros a month for quite some time. And even longer was living below what Germany considers the amount of monthly income that one would qualify to not be called poor ("Armutsgrenze"). I know the feeling and hear you. The only difference is that I could not even get a contract as no telco would find me creditworthy enough. Needed to do prepaid and see how I would get any old mobile.

Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy

#70

Post by sdoering »

jacquesm wrote: post #63
sdoering wrote: post #29

Earlier quoted context omitted.

My SO used to work in a bank in the business loan securities back office. Believe me. Lots of people and businesses apply even if they themselves see no realistic chance of receiving a loan. And in private non business customers there are even a significant amount that have less than nothing to their name, living of welfare and still don't see, why they should not receive a loan from the bank for the newest iPhone. O…

Excuse my snarkiness, but having actually worked for a bank and seeing the inside of the loan department from an IT perspective, the bulk of that translates into: banks will allow the people that are already wealthy to leverage to make them even more wealthy while working hard to keep those already down firmly in their place. Not to mention to take advantage of them by charging them exorbitant fees for the little bit…

I am with you. Shitty entitled customers calling female tellers whores that need to be f**ed to loosen up a bit (happened more than once) do not absolve banks from their abysmal policies. Or the system that not only enables but sometimes regulates/necessitates these policies.

I am not siding with banks in general here. I was just exemplifying one aspect to show how it is not always black and white.amd how there can be unreasonable customers.

[Edit typos]

Post reply on HN