Financial Statement Analysis with Large Language Models

Was going to point out the same. Glad to have the paper to read but I don't think the findings are significant.

The area where I see this making the most transformational change is by enabling average citizens to ask meaningful questions about the finances of their local government. In Cook County, Illinois, there are hundreds of local municipalities and elected authorities, all of which are producing monthly financial statements. There is not enough citizen oversight and rarely any media attention except in the most egregious cases (e.g. the recent drama in Dolton, IL, where the mayor is stealing millions in plain view of the citizens).

Not to mention, as somebody who works in quant trading doing ml all day on this kind of data. That ann benchmark is nowhere near state of the art.

People didn't stop working on this in 1989 - they realised they can make lots of money doing it and do it privately.

That is not a benefit. If you use a tool like this to try to compete with sophisticated actors (e.g. all major firms in the capital markets space) you will lose every time.

So the history of this type of research as I know it was that we

- started to diff the executives statements from one quarter to another. Like engineering projects alot of this is pretty standard so the starting point is the last doc. Diffing allowed us to see what the executives added and thought was important and also showed what they removed. This worked well and for some things still does, this is what a warrant canary does, but stopped generating much alpha around 2010ish.

- simple sentiment. We started to count positive and negative words to build a poor mans sentiment analysis that could be done very quickly upon doc release to trade upon. worked great up until around 2013ish before it started to be gamed and even bankruptcy notices gave positive sentiment scores by this metric.

- sentiment models. Using proper models and not just positive and negative word counts we built sentiment models to read what the executives were saying. This worked well until about 2015/2016ish in my world view as by then executives carefully wrote out their remarks and had been coached to use only positive words. Worked until twitter killed the fire hose, and wasn't very reliable as reputable news accounts kept getting hacked. I remember i think AP new's account got hacked and reported a bombing at the white house that screwed up a few funds.

You also had Anne Hathaway news pushing up Berkshire Hathaway's share price type issues in this time period.

- there was a period here where we kept the same technology but used it everywhere from the twitter firehose to news articles to build a realtime sentiment model for companies and sectors. Not sure it generates much alpha due to garbage in, garbage out and data cleaning issues.

- LLMs, with about GPT2 we could build models to do the sentiment analysis for us, but they had to be built out of foundational models and trained inhouse due to context limitations. Again this has been gamed by executives so alot of the research that I know of is now targeted at ingesting the Financials of companies and being able to ask questions quickly without math and programming.

ie what are the top 5 firms in the consider discretionary space that are growing their earnings the fastest while not yet raising their dividends and whose share price hasn't kept up with their sectors average growth.