Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Financial Statement Analysis with Large Language Models

papers.ssrn.com

Post reply on HN
Print view View original

31–40 of 219 posts

Re: Financial Statement Analysis with Large Language Models

#31

Post by TechDebtDevin »

yaj54 wrote: post #21
foota wrote: post #12

Earlier quoted context omitted.

I agree this isn't earth shattering, but I think the benefit here is that it's a general solution instead of one trained on financial statements specifically.

agreed. most people can't create a custom tailored finance statement model. but many people can write the following sentence: "analyze this financial statement and suggest a market strategy." and if that sentence performs as well as an (albeit old) custom model, and is likely to have compound improvements in its performance over time with no changes to the instruction sentence...

No

Re: Financial Statement Analysis with Large Language Models

#32

Post by 01100011 »

Top story on HN because we all secretly think we can be the next Jim Simons when in reality we're a few months away from posting loss porn to /r/WSB.

If standardized LLM models are used to analyze statements, expect the statements to be massaged in ways that produce more favorable results from the LLM.

Re: Financial Statement Analysis with Large Language Models

#33

Post by bossyTeacher »

flourpower471 wrote: post #15
antimatter15 wrote: post #9

Figure 3 on p.40 of the paper seems to show that their LLM based model does not statistically significantly outperform a 3 layer neural network using 59 variables from 1989. This figure compares the prediction performance of GPT and quantitative models based on machine learning. Stepwise Logistic follows Ou and Penman (1989)’s structure with their 59 financial predictors. ANN is a three-layer artificial neural networ…

Not to mention, as somebody who works in quant trading doing ml all day on this kind of data. That ann benchmark is nowhere near state of the art. People didn't stop working on this in 1989 - they realised they can make lots of money doing it and do it privately.

>People didn't stop working on this in 1989 - they realised they can make lots of money doing it and do it privately.

Mind elaborating?

Re: Financial Statement Analysis with Large Language Models

#34

Post by hnthrowaway6543 »

primitivesuave wrote: post #13

The area where I see this making the most transformational change is by enabling average citizens to ask meaningful questions about the finances of their local government. In Cook County, Illinois, there are hundreds of local municipalities and elected authorities, all of which are producing monthly financial statements. There is not enough citizen oversight and rarely any media attention except in the most egregious…

Let's say LLMs work exactly as advertised in this case: you go into the LLM, say "find corruption in these financial reports", and it comes back with some info about the mayor spending millions on overpriced contracts with a company run by his brother. What then? You can post on Twitter, but unless you already have a following it's shouting into the void. You can go to your local newspapers, they'll probably ignore you; if they do pay attention, they'll write an article which gets a few hundred hits. If the mayor acknowledges it at all, they'll slam it as a political hit-piece, and that's the end of it. So your best chance is... hope really hard it goes viral, I guess?

This isn't meant to be overly negative, but exposing financial corruption is mostly about information control; I don't see how LLMs help much here. Even if/when you find slam-dunk evidence that corruption is occurring, it's generally very hard to provide evidence in a way that Joe Average can understand, and assuming you are a normal everyday citizen, it's extremely hard to get people to act.

As a prime example, this bit on the SF "alcohol rehab" program[0] went semi-viral earlier this week; there's no way to interpret $5 million/year spent on 55 clients as anything but "incompetence" at best and "grift and corruption" at worst. Yet there's no public outrage or people protesting on the streets of SF; it's already an afterthought in the minds of anyone who saw it. Is being able to query an LLM for this stuff going to make a difference?

[0] https://www.sfchronicle.com/politics/article/sf-free-alcohol...

Re: Financial Statement Analysis with Large Language Models

#35

Post by ecjhdnc2025 »

yaj54 wrote: post #21
foota wrote: post #12

Earlier quoted context omitted.

I agree this isn't earth shattering, but I think the benefit here is that it's a general solution instead of one trained on financial statements specifically.

agreed. most people can't create a custom tailored finance statement model. but many people can write the following sentence: "analyze this financial statement and suggest a market strategy." and if that sentence performs as well as an (albeit old) custom model, and is likely to have compound improvements in its performance over time with no changes to the instruction sentence...

But it can't come up with a particularly imaginative strategy; it can only come up with a mishmash of existing stuff it has seen, equivocate, or hallucinate a strategy that looks clever but might not be.

So it all needs checking. It's the classic LLM situation. If you're trained enough to spot the errors, the analysis wouldn't take you much time in the first place. And if you're not trained enough to spot the errors...

And let's say it does work. It's like automated exchange betting robots. As soon as everyone has access to a robot that can exploit some hidden pattern in the data for a tiny marginal gain, the price changes and the gain collapses.

So if everyone has the same access to the same banal, general analysis tools, you know what's going to happen: the advantage disappears.

All in all, why would there be any benefits from a generalised model?

Re: Financial Statement Analysis with Large Language Models

#36

Post by SavageBeast »

bossyTeacher wrote: post #33
flourpower471 wrote: post #15

Earlier quoted context omitted.

Not to mention, as somebody who works in quant trading doing ml all day on this kind of data. That ann benchmark is nowhere near state of the art. People didn't stop working on this in 1989 - they realised they can make lots of money doing it and do it privately.

>People didn't stop working on this in 1989 - they realised they can make lots of money doing it and do it privately. Mind elaborating?

Speaking for myself and likely others with similar motivations, yes we can "figure it out" and publish something to show our work and expand the field of endeavor with our findings - OR - we can figure something profitable out on our own and use our own funds to trade our strategies with our own accounts.

Anyone who has figured out something relatively profitable isn't telling anyone how they did it.

Re: Financial Statement Analysis with Large Language Models

#37

Post by foota »

_se wrote: post #17
foota wrote: post #12

Earlier quoted context omitted.

I agree this isn't earth shattering, but I think the benefit here is that it's a general solution instead of one trained on financial statements specifically.

That is not a benefit. If you use a tool like this to try to compete with sophisticated actors (e.g. all major firms in the capital markets space) you will lose every time.

We come up with all sorts of things that are initially a step backwards, but that lead to eventual improvement. The first cars were slower than horses.

That's not to suggest that Renaissance is going to start using Chat GPT tomorrow, but maybe in a few years they'll be using fine tuned versions of LLMs in addition to whatever they're doing today.

Even if it's not going to compete with the state of the art models for something, a single model capable of many things is still useful, and demonstrating domains where they are applicable (if not state of the art) is still beneficial.

Re: Financial Statement Analysis with Large Language Models

#39

Post by dpflan »

bossyTeacher wrote: post #33
flourpower471 wrote: post #15

Earlier quoted context omitted.

Not to mention, as somebody who works in quant trading doing ml all day on this kind of data. That ann benchmark is nowhere near state of the art. People didn't stop working on this in 1989 - they realised they can make lots of money doing it and do it privately.

>People didn't stop working on this in 1989 - they realised they can make lots of money doing it and do it privately. Mind elaborating?

Seems simple: Why share your effective strategies in an industry full of competition and those striving to gain a competitive edge?

Re: Financial Statement Analysis with Large Language Models

#40

Post by melenaboija »

bossyTeacher wrote: post #33
flourpower471 wrote: post #15

Earlier quoted context omitted.

Not to mention, as somebody who works in quant trading doing ml all day on this kind of data. That ann benchmark is nowhere near state of the art. People didn't stop working on this in 1989 - they realised they can make lots of money doing it and do it privately.

>People didn't stop working on this in 1989 - they realised they can make lots of money doing it and do it privately. Mind elaborating?

> Mind elaborating?

I am assuming, he/she minds a lot.

Post reply on HN