Financial Statement Analysis with Large Language Models
papers.ssrn.com
Financial Statement Analysis with Large Language Models
1–10 of 219 posts
Re: Financial Statement Analysis with Large Language Models
#2[dead]
Re: Financial Statement Analysis with Large Language Models
#3If that's a thing, does it become a thing learning how to write a "poisoned" statement that misleads an LLM but is still factual?
Re: Financial Statement Analysis with Large Language Models
#4I recalling seeing a LinkedIn post by Greg Diamos at Lamini, they shared analysis of earnings calls. There are links on HuggingFace and GitHub, here they are:
- https://huggingface.co/datasets/lamini/earnings-calls-qa
- https://huggingface.co/datasets/lamini/earnings-raw
- https://github.com/lamini-ai/lamini-earnings-calls/tree/main
Re: Financial Statement Analysis with Large Language Models
#5If that's a thing, does it become a thing learning how to write a "poisoned" statement that misleads an LLM but is still factual?
It is already been done as quants analyze company statements for the last decade at least, counting positive and negative words, etc...
Re: Financial Statement Analysis with Large Language Models
#6If that's a thing, does it become a thing learning how to write a "poisoned" statement that misleads an LLM but is still factual?
Maybe, but it sounds hard if there are multiple LLMs out there that people might use to analyze such text. Tricking multiple LLMs with a certain poisonous combination of words and phrases sounds a lot like having a file that hashes to the same hash from different hashing techniques. Theoretically possible but actually practically impossible.
Re: Financial Statement Analysis with Large Language Models
#7It would've been interesting to compare models with larger context windows, e.g. Gemini with 1m+ tokens and Claude Opus. Otherwise, the title maybe should've been Financial Statement Analysis with GPT-4.
Re: Financial Statement Analysis with Large Language Models
#8It would've been interesting to compare models with larger context windows, e.g. Gemini with 1m+ tokens and Claude Opus. Otherwise, the title maybe should've been Financial Statement Analysis with GPT-4.
The study only captured the financial statements. I am unsure what a larger context window would buy you.
Re: Financial Statement Analysis with Large Language Models
#9Figure 3 on p.40 of the paper seems to show that their LLM based model does not statistically significantly outperform a 3 layer neural network using 59 variables from 1989.
This figure compares the prediction performance of GPT and quantitative models based on machine learning. Stepwise Logistic follows Ou and Penman (1989)’s structure with their 59 financial predictors. ANN is a three-layer artificial neural network model using the same set of variables as in Ou and Penman (1989). GPT (with CoT) provides the model with financial statement information and detailed chain-of-thought prompts. We report average accuracy (the percentage of correct predictions out of total predictions) for each method (left) and F1 score (right). We obtain bootstrapped standard errors by randomly sampling 1,000 observations 1,000 times and include 95% confidence intervals.Re: Financial Statement Analysis with Large Language Models
#10If that's a thing, does it become a thing learning how to write a "poisoned" statement that misleads an LLM but is still factual?
As someone alluded to, the narrative that management drives has been examined and studied many times over. What is management saying, what are they not saying, what are they saying but not loudly, what did they say before that they no longer speak about. There are insights to glean but nothing that is giving you an unknown edge. Sentiment analysis and the like go back well into the late 80s, early 90s.