Reading SEC filings using LLMs
beatandraise.com
Reading SEC filings using LLMs
1–10 of 78 posts
Re: Reading SEC filings using LLMs
#2I have been working on getting ChatGPT to answer questions that equity research analysts, investors would like to get from SEC filings. The application uses a combination of hybrid text search and LLMs for completion and does not rely much on embedding based distance searches.
A core assumption underlying this is that LLMs are already pretty good and will continue to get better at reading texts. If provided with the right thing to read, they will do very well on 'reading comprehension'.
Open ended writing is more susceptible to errors, especially in questions related to finance. For e.g google's revenues are just as likely to be 280.2 billion vs 279 billion in a probabilistic model that guesses the next part of the sentence - Google's revenues for FY 2022 are ....
So this leaves us with the main problem to solve; Serving the right texts to the LLM aka text search.
Once the right text is served, we can generate any pretty much anything in the text, Income statements, ceo comments, accounts payable on the fly, For e.g try - `can you get me Nvidia and AMD's income statement from March 2020 ?` as in here. https://imgur.com/gallery/H8Vfd5X A few more examples, Apple's sales in China, Google's revenue by quarter: https://imgur.com/a/oCCay3o
Currently, the application supports ~8k companies that are registered with the SEC. Pdfs are still work in progress, so tesla etc don't work as well.
The stack is Nextjs on Supabase. So Postgres's inbuilt text search does a lot of heavy lifting.
If one thinks of the bigger picture, we can extend/improve this to pdfs and the entire universe of stocks and more. a.k.a a big component of what CapitalIQ, Factset, Bloomberg and Reuters do can now be generated on the fly accurately for a fraction of the cost.
Generating graphs with gross margin increasing etc are just one step further and stuff like EV/Ebitda, yet another step further, as one can call a stock pricing api for each date of the report.
I would guess a number of LLM applications follow a similar process, ask a question --> LLM converts to query --> datalakes/bases --> searching and serving texts --> answer. Goes without saying, I would appreciate any feedback, especially from those who are building stuff that looks architecturally similar :) !
Re: Reading SEC filings using LLMs
#3I have been working on getting ChatGPT to answer questions that equity research analysts, investors would like to get from SEC filings. The application uses a combination of hybrid text search and LLMs for completion and does not rely much on embedding based distance searches. A core assumption underlying this is that LLMs are already pretty good and will continue to get better at reading texts. If provided with the…
I've been trying something similar with parliamentary debates. They're long winded, often full of empty speech, and a chore to read.
The LLMs are able to hone in on the details and provide interesting responses like "What questions were asked of the minister that they failed to address" and "What should the opposition leader have mentioned in their response that the minister would have found difficult to answer"
Crucially, they're well transcribed: https://api.oireachtas.ie/
Re: Reading SEC filings using LLMs
#4I have been working on getting ChatGPT to answer questions that equity research analysts, investors would like to get from SEC filings. The application uses a combination of hybrid text search and LLMs for completion and does not rely much on embedding based distance searches. A core assumption underlying this is that LLMs are already pretty good and will continue to get better at reading texts. If provided with the…
One of the advantages of data from CapIQ /Refinitiv is that you're not just pulling data from a single report but rather data has been curated across time from multiple historical reports so that historical income statements, balance sheets, footnote data etc spanning many years can be generated.
When you say that generating graphs of gross margin, EV/Ebitda is just one step further, are you talking about generating those based in just a single report's information or are you combining information from multiple years to for example show gross margin trends over 10 years and EV/TTM Ebitda?
Re: Reading SEC filings using LLMs
#5I have been working on getting ChatGPT to answer questions that equity research analysts, investors would like to get from SEC filings. The application uses a combination of hybrid text search and LLMs for completion and does not rely much on embedding based distance searches. A core assumption underlying this is that LLMs are already pretty good and will continue to get better at reading texts. If provided with the…
Neat idea!
How do you parse the PDFs?
Re: Reading SEC filings using LLMs
#6I have been working on getting ChatGPT to answer questions that equity research analysts, investors would like to get from SEC filings. The application uses a combination of hybrid text search and LLMs for completion and does not rely much on embedding based distance searches. A core assumption underlying this is that LLMs are already pretty good and will continue to get better at reading texts. If provided with the…
One of the advantages of data from CapIQ /Refinitiv is that you're not just pulling data from a single report but rather data has been curated across time from multiple historical reports so that historical income statements, balance sheets, footnote data etc spanning many years can be generated. When you say that generating graphs of gross margin, EV/Ebitda is just one step further, are you talking about generating…
I am talking about comparing multiple reports, i.e gross margin trends over 10 years and EV/TTM Ebitda etc across several reports. Currently only financials are possible, but the ratios depend on stock prices, so we are working on that !
You can think of it like this, you now have an army of readers that can go through tables really quickly.
FactSet, CapIQ etc use a combination of automation/manual entry and fit these tables into a homogenized schema so that they can be saved, compared etc. So if you want to get Apple's Greater China sales from 2020, you would be lucky if they decided to create an item for that. https://imgur.com/a/bp2hb7n
Here are two examples, AMD's revenues and AMD's revenue outlook using beatandraise.com https://imgur.com/a/61jqiUk I doubt you can get AMD's own outlook on CapIQ for e.g
Context sizes mean getting 100s of reports on one call is not possible, but multiple iterations will still do the trick. So in effect, you can actually create a dataset like FactSet for a lot lower cost, more comprehensive and can be customized to what the user wants, if you see my point... :)
Re: Reading SEC filings using LLMs
#7I have been working on getting ChatGPT to answer questions that equity research analysts, investors would like to get from SEC filings. The application uses a combination of hybrid text search and LLMs for completion and does not rely much on embedding based distance searches. A core assumption underlying this is that LLMs are already pretty good and will continue to get better at reading texts. If provided with the…
Neat idea! How do you parse the PDFs?
Apache Tika has worked well for me in the past, ended up running it on an AWS Lambda
Re: Reading SEC filings using LLMs
#8I have been working on getting ChatGPT to answer questions that equity research analysts, investors would like to get from SEC filings. The application uses a combination of hybrid text search and LLMs for completion and does not rely much on embedding based distance searches. A core assumption underlying this is that LLMs are already pretty good and will continue to get better at reading texts. If provided with the…
I've been trying something similar with parliamentary debates. They're long winded, often full of empty speech, and a chore to read. The LLMs are able to hone in on the details and provide interesting responses like "What questions were asked of the minister that they failed to address" and "What should the opposition leader have mentioned in their response that the minister would have found difficult to answer" Cruc…
I think one thing you can try is to figure out what lies where, so chunking arbitrarily will not work as well as chunking with headings for e.g
For e.g, if the question is: "What should the opposition leader have mentioned in their response that the minister would have found difficult to answer."
An embedding based search will find it fairly difficult to match this against a text. Based on my experience, you have to figure out what is a nonanswer first (i don't think that's easy but gpt4 is very good at a lot of language based stuff.)
you can try Q1: question A1: Answer then prompt GPT, do you think A1 answers the question and then save it. And then, Q1: question, A1: Answer, Q2: Follow up based on the following questions, do you think A1 answers Q1 and then save it to a db.
You can then augment it in the code with our own knowledge of how politicians lie, using certain words etc :) to improve what gpt4 might miss...
Re: Reading SEC filings using LLMs
#9I have been working on getting ChatGPT to answer questions that equity research analysts, investors would like to get from SEC filings. The application uses a combination of hybrid text search and LLMs for completion and does not rely much on embedding based distance searches. A core assumption underlying this is that LLMs are already pretty good and will continue to get better at reading texts. If provided with the…
Neat idea! How do you parse the PDFs?
Pdf parsing was more tedious that I would have liked at this stage so I stuck to the SEC which requires that companies file in a text format :) so that helped.
I used poppler on a digital ocean droplet, but the sheer variety of company pdfs especially european companies, some of which have to be OCRed, meant results were not really uniform. GPT still does very well, but not as well as on text documents directly. So in short, this is next on the list...