How does "simple AI" compare with an old fashioned small table lookup and a few "if" statements?
https://en.wikipedia.org/wiki/Random_forest
‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy
11–20 of 91 posts
Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy
#12If you want to impress me, tell me also the results of these: - Performance against a fair dice - Performance against a group of humans trained to predict bank managers' loan decisions
i d be more interested if the ai made a better financial decision than the managers
Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy
#13Earlier quoted context omitted.
https://en.wikipedia.org/wiki/Random_forest
https://en.wikipedia.org/wiki/Expert_system
Difference is that random decision forests learn the rules for themselves, they don't have to be programmed in manually by experts. They're one of the most performant "pre-deep-learning" machine learning models and a sensible baseline for many ML tasks before you go out and buy a $1500 GPU
Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy
#14https://astralcodexten.substack.com/p/heuristics-that-almost...
I was just coming here to post the same link. The 95% accuracy should be compared to the general approval rate - does the fancy AI beat a simple return true?
Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy
#15This isn't surprising. Most 'AI' comes down to correlating a small number of variables (one or two) with a prediction target. The real benefit of any form of machine learning is detecting functional relationships between variables (ie, "when this AND NOT that OR this"). It's just that these relationships don't provide a real benefit in 99/100 real world use cases. In the case of a loan, if the credit score is high th…
When dealing with stakeholders in big corporations I learned that they absolutely needed AI, machine learning and data science as capabilities in the projects that they started. Even if data was only very tangentially related to the core product/project. If so, for example one needed to datascience the shit out of project metadata to create a project efficiency self optimization loop. Or what ever crap one could come up with.
Else they would not be able to secure funding for the idea from upper management because they had learned that 'data is the new oil'.
The second VS part always needed/needs to be how this project enables the product to be (or become) a platform where the company can the run value added services on top (value adding for the company, not the customer).
Imagine a car being a platform and you can book additional horse power for the upcoming trip. Or change the background image for the instrument panel. Or pay extra to natively have access to your Spotify playlists. And then datascience the user data to recommend even more stuff resulting in $$$ (by imagination from management and fuelled by slide decks created by some junior strategists that have never even owned a car but presented by the client account lead in a lush retreat).
Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy
#16As bank I'd look into these 4.x % of loans where the machine learning disagrees.
This headline feels like 20 years old though, except for calling it "AI".
Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy
#17Loan decision is relatively straightforward algorithmic process. I see no reason why AI is needed or wouldn't give similar answers.
Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy
#18I don't think you need AI. Like in Italy the total of your loans cannot exceed 1/3rd of your monthly salary, as simple as that.
Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy
#19https://astralcodexten.substack.com/p/heuristics-that-almost...
I was just coming here to post the same link. The 95% accuracy should be compared to the general approval rate - does the fancy AI beat a simple return true?
The logistic regression would likely pick up on that if it were a stratification issue.
Re: ‘Simple’ AI can anticipate bank managers’ loan decisions to over 95% accuracy
#20If you want to impress me, tell me also the results of these: - Performance against a fair dice - Performance against a group of humans trained to predict bank managers' loan decisions
The article is a bit rubbish. They're not predicting on the binary "would we give these people a loan or not" but they're predicting manual corrections fo credit scores by bank managers. It's a 15 way classification problem (1 is low score, 15 is high). The data is distributed in a bell-curve like way with the most people in the 6 or 7 bracket.
From the paper:
> As is typical in machine learning we also report the Accuracy p-value computed from a one-sided test (Kuhn et al., 2008) which compares the prediction accuracy to the "no information rate", which is the largest class percentage in the data (23.85%).
So fair dice 23.85%, model 95%.
That said I bet a human who had read the banking rules and regulations and recommendations on lending could easily match this performance.