Financial Statement Analysis with Large Language Models • Hacker News Board

Re: Financial Statement Analysis with Large Language Models

#151

Post by mulmen » Sat, May 25, 2024, 1:10 AM UTC

hnthrowaway6543 wrote: post #34

primitivesuave wrote: post #13

The area where I see this making the most transformational change is by enabling average citizens to ask meaningful questions about the finances of their local government. In Cook County, Illinois, there are hundreds of local municipalities and elected authorities, all of which are producing monthly financial statements. There is not enough citizen oversight and rarely any media attention except in the most egregious…

Let's say LLMs work exactly as advertised in this case: you go into the LLM, say "find corruption in these financial reports", and it comes back with some info about the mayor spending millions on overpriced contracts with a company run by his brother. What then? You can post on Twitter, but unless you already have a following it's shouting into the void. You can go to your local newspapers, they'll probably ignore y…

Why would you report financial crimes to Twitter? If your LLM uncovers financial crimes you should contact regulators and prosecutors. They're both incentivized to do something about it.

Re: Financial Statement Analysis with Large Language Models

#152

Post by ds_opseeker » Sat, May 25, 2024, 1:46 AM UTC

jasonfarnon wrote: post #125

SavageBeast wrote: post #36

Earlier quoted context omitted.

Speaking for myself and likely others with similar motivations, yes we can "figure it out" and publish something to show our work and expand the field of endeavor with our findings - OR - we can figure something profitable out on our own and use our own funds to trade our strategies with our own accounts. Anyone who has figured out something relatively profitable isn't telling anyone how they did it.

Why not then publish the strategies once outmoded, or are they in fact published? Can I go see somewhere what strategies big funds used in the 90s to make bank, which presumably no longer offer a competitive advantage? The way I can go see what computer exploits/hacks used to work when they were still secret? Maybe it's just what I know, but I can't help but think the "strategies" are a lot like security exploits--so…

> Why not then publish the strategies once outmoded

Because then your competition knows which strategies don't work, and also what types of strategies you work on.

Don't leak information.

Re: Financial Statement Analysis with Large Language Models

#153

Post by benreesman » Sat, May 25, 2024, 1:50 AM UTC

flourpower471 wrote: post #15

antimatter15 wrote: post #9

Figure 3 on p.40 of the paper seems to show that their LLM based model does not statistically significantly outperform a 3 layer neural network using 59 variables from 1989. This figure compares the prediction performance of GPT and quantitative models based on machine learning. Stepwise Logistic follows Ou and Penman (1989)’s structure with their 59 financial predictors. ANN is a three-layer artificial neural networ…

Not to mention, as somebody who works in quant trading doing ml all day on this kind of data. That ann benchmark is nowhere near state of the art. People didn't stop working on this in 1989 - they realised they can make lots of money doing it and do it privately.

I never traded consistently and successfully but I did do a startup with a seasoned quant trader with the ambition of using bigger models to generate novel alpha. We mopped the floor with the academics who publish but that is whiffle ball compared to a real prop outfit that lasts.

Not having made it big myself I obviously don’t know the meta these days, but last I had any inside baseball, the non-stationarity and friction just kill you on trying to get fancy as opposed to just nailing it on the fundamentals.

Extreme execution quality is a game, people make money in both traditional liquidity provision and agency execution by being fast as hell and managing risk well.

Individual signals that are individually somewhat mundane but composed well via straightforward linear-ish regressions is a game: people get (ever decaying) alpha out of bright ideas (and rotate new signals in).

And I’m sure that LLMs have started playing a role, there’s a legitimate capability increase in spite of the dubious production-worthiness.

But as a blind wager, I bet prop trading is about what it was 5 years ago on better gear: elite execution (no pun intended) on known-good ways to generate alpha.

Re: Financial Statement Analysis with Large Language Models

#154

Post by bbor » Sat, May 25, 2024, 2:08 AM UTC

Oh damn, I thought this was about real finances, turns out it’s just part of that weird “property” thing they do in NYC. I hope someone’s working on feeding their Ledger files into a structured language model…

More substantively, LLMs are for linguistic tasks. That’s why I’m super super bullish (heh) on llms for decoding EEG data, and incredibly bearish on their ability to accurately model a corporation’s asset flow. I just don’t see how the confounding variables / motivating forces would be at all linguistic. This is basically using LLMs for super-advanced arithmetic

Re: Financial Statement Analysis with Large Language Models

#155

Post by bbor » Sat, May 25, 2024, 2:10 AM UTC

01100011 wrote: post #32

Top story on HN because we all secretly think we can be the next Jim Simons when in reality we're a few months away from posting loss porn to /r/WSB. If standardized LLM models are used to analyze statements, expect the statements to be massaged in ways that produce more favorable results from the LLM.

I can’t wait until there’s warnings on stock market apps like cigarettes and lottery tickets. Well actually I guess there are no warnings on lotto tickets, probably for the exact same reason as why the government doesn’t protect people from being scammed by hedge funds with way more info than they have: the government needs that revenue.

Re: Financial Statement Analysis with Large Language Models

#156

Post by vharuck » Sat, May 25, 2024, 2:11 AM UTC

apwell23 wrote: post #20

primitivesuave wrote: post #13

The area where I see this making the most transformational change is by enabling average citizens to ask meaningful questions about the finances of their local government. In Cook County, Illinois, there are hundreds of local municipalities and elected authorities, all of which are producing monthly financial statements. There is not enough citizen oversight and rarely any media attention except in the most egregious…

> citizens to ask meaningful questions about the finances of their local government. is there a demand for this. I live in cook country. I really don't want to ask these questions. Not sure what I get out of asking these questions other than anger and frustration.

Most people won't care most of the time. But if the local government cuts the budget for something you like and says "We couldn't find the money," you may care that year.

Re: Financial Statement Analysis with Large Language Models

#157

Post by FabHK » Sat, May 25, 2024, 2:29 AM UTC

richrichie wrote: post #132

chollida1 wrote: post #91

Earlier quoted context omitted.

> the way to calculate the price of an Option/Derivative hasn't changed in my understanding for 20/30 years That’s not true. It is true that the black scholes model was found in the 70s but since then you have - stochastic vol models - jump diffusion -local vol or Dupire models - levy process - binomial pricing models all came well After the initial model was derived. Also a lot of work in how to calculate vols or pr…

Very few of the fancy models are actually used. Dupire's non parametric model has been the industrial work horse for a long time. Heston like SV's and Jump diffusions promised a lot and did not work in practice (calibration, stability issues). Some form of local stochastic models get used for certain products. In general, it is safe to say that Black-Scholes and its deterministic extension local vol have held up well…

Not only that, but Dupire’s local vol, stochastic vol (Heston in rates, or on the equity side models that combine local vol with a stoch vol component to calibrate to implied vols perfectly) and jump diffusion were basically in production 15 years ago.

Since the GFC it’s not about crazy new products (on derivatives desks), but it’s about getting discounting/funding rates precisely right (depending on counterparty, collateral and netting agreements, onshore/offshore, etc), and about compliance and reporting.

Re: Financial Statement Analysis with Large Language Models

#158

Post by FabHK » Sat, May 25, 2024, 2:30 AM UTC

JumpCrisscross wrote: post #119

posting_mess wrote: post #73

Earlier quoted context omitted.

Quantitative trading is simply the act of trading on data, fast or slowly, but I'll grant you for the more sophisticated audience there is a nuance between "HFT" and "Quant" trading. To be "super right" you just have to make money over a timeline, you set, according to your own models. If I choose a 5 year timeline for a portfolio, I just have to show my portfolio outperforming "your preferred index here" over that t…

> the way to calculate the price of an Option/Derivative hasn't changed in my understanding for 20/30 years Not true. Most of the magic happens in estimating the volatility surface, BSM's magic variable. But I've also seen interesting work in expanding the rates components. All this before we get into the drift functions.

While the industry has changed substantially since the GFC, all foundational derivatives models were basically in place back then.

Re: Financial Statement Analysis with Large Language Models

#159

Post by helsinkiandrew » Sat, May 25, 2024, 2:32 AM UTC

nebula8804 wrote: post #53

chollida1 wrote: post #19

So the history of this type of research as I know it was that we - started to diff the executives statements from one quarter to another. Like engineering projects alot of this is pretty standard so the starting point is the last doc. Diffing allowed us to see what the executives added and thought was important and also showed what they removed. This worked well and for some things still does, this is what a warrant…

I have no window into this world but I am curious if you know anything about the techniques that investors used to short or just analyze Tesla stock during the production hell of 2017-2020? It was an interesting window in ways that firms use to measure as much of the company as they can from the outside. In fact was there any other stock that was as heaving watched during that time? Looking back at that era it seemed…

Most companies aren’t obsessed enough with shortens to try and hide poor results from analysis that will be exposed in 3 months anyway. There’s always ways around them - number of cars registered, number of delivery trucks visiting, time for delivery on website, how much overtime is being worked etc.

Re: Financial Statement Analysis with Large Language Models

#160

Post by rgmerk » Sat, May 25, 2024, 2:40 AM UTC

primitivesuave wrote: post #13

The area where I see this making the most transformational change is by enabling average citizens to ask meaningful questions about the finances of their local government. In Cook County, Illinois, there are hundreds of local municipalities and elected authorities, all of which are producing monthly financial statements. There is not enough citizen oversight and rarely any media attention except in the most egregious…

“There are hundreds of local municipalities and elected authorities.”

That’s the root of your problem. Too many governments, not enough attention available to keep them accountable.