Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reading SEC filings using LLMs

beatandraise.com

Post reply on HN
Print view View original

51–60 of 78 posts

Re: Reading SEC filings using LLMs

#51

Post by nnechm »

alxdrlitreev wrote: post #49

We've recently made COFIN AI — an AI based on ChatGPT that reads hundreds of pages of SEC filings such as 10Ks and 10Qs and checks investor calls to assist you with investment research and strategy evaluation. Pretty much similar (almost the same) to something you did. Check this out: https://cofinapp.com Let me know what you think!

Thanks for letting me know, i guess you use a different strategy of querying by company and by document. I see this when I try to get Apple's rest of asia pacific revenues for e.g

I guess it really depends on how your targeted user would use it... :)

https://imgur.com/a/uEgLUpz

Re: Reading SEC filings using LLMs

#52

Post by nnechm »

nnechm wrote: post #2

I have been working on getting ChatGPT to answer questions that equity research analysts, investors would like to get from SEC filings. The application uses a combination of hybrid text search and LLMs for completion and does not rely much on embedding based distance searches. A core assumption underlying this is that LLMs are already pretty good and will continue to get better at reading texts. If provided with the…

A few things on prompting: 1. Get me google cloud revenues fails, because somehow gpt4 thinks i am talking about an entity called google cloud and not google :) 2. So in order to fix it, you can either ask for Get me google's cloud revenues or get me google cloud revenues from google's results...

As you can see, inspite of all the training in the real world, gpt4 thinks google cloud is more of an entity than google is, based on that question :)

Re: Reading SEC filings using LLMs

#54

Post by Der_Einzige »

dcsan wrote: post #22
anotherhue wrote: post #3

Earlier quoted context omitted.

I've been trying something similar with parliamentary debates. They're long winded, often full of empty speech, and a chore to read. The LLMs are able to hone in on the details and provide interesting responses like "What questions were asked of the minister that they failed to address" and "What should the opposition leader have mentioned in their response that the minister would have found difficult to answer" Cruc…

Could you ask an LLM if they were persuaded by a speaker of one side of a debate as a method of evaluation? Ie the bot's before and after opinion based on fine-tuning with the pro and con arguments? I was also thinking about a society of bots type application where you could have autonomous bot researchers, debaters, judges and audience. Would be interesting to feed in the topics and grab some popcorn

Relevant useful dataset: https://paperswithcode.com/dataset/debatesum

Re: Reading SEC filings using LLMs

#55

Post by halflings »

manuelabeledo wrote: post #53

I may be missing something, but looking at the examples in the website, isn't this just a fancy worded search? What kind of data can be extracted from these filings, that is not available using "traditional" methods?

Yes, this looks like a (hopefully) better information retrieval system than running CTRL+F over a PDF.

Nothing wrong with that :) information retrieval is hard. And CTRL+F won't render a table for queries like "what is the % increase of revenue over the last 3 quarter"-type queries.

Re: Reading SEC filings using LLMs

#56

Post by gen220 »

Beware Gell-Mann amnesia when assessing how useful LLMs are for investing research. Don't forget the instinctual response you have to LLMs' supposed abilities to write software projects, when assessing LLMs' abilities to comprehend and extract the interesting/relevant parts of a 10-Q/K.

IME, the most useful software tools in investing are analogous to the most useful software tools in software-building: the IDEs, libraries and frameworks that make an increasingly-expert human more effective.

Unfortunately, there aren't any long-term-valuable shortcuts to attaining expertise in any domain, but there are an increasing number dead ends that really look like shortcuts.

Re: Reading SEC filings using LLMs

#57

Post by nnechm »

manuelabeledo wrote: post #53

I may be missing something, but looking at the examples in the website, isn't this just a fancy worded search? What kind of data can be extracted from these filings, that is not available using "traditional" methods?

That's a good question, for e.g if you want to get Microsoft's revenues by product category or apple's revenues in greater China. https://imgur.com/a/LdQkt7j

You can also do this across time with constraints on context limits ...

What is a traditional method, would you search the string or would you find the named entity (using NLP) and look for the entity ? or do you mean a ctrl +F in the document ?

I guess the premise of LLMs, is that we have an intelligence that can read and write, so it can do this in an automated way in different applications. In this case, I am trying to automate and speed up the investment research process, along the way, we can create our own dataset of financial data that can be generated on the fly as necessary.

Also, how would you extract an income statement from the text, also in the same img using a traditional method ? I personally find it magical it can do that, it knows where it ends and so on...

Re: Reading SEC filings using LLMs

#58

Post by nnechm »

halflings wrote: post #55
manuelabeledo wrote: post #53

I may be missing something, but looking at the examples in the website, isn't this just a fancy worded search? What kind of data can be extracted from these filings, that is not available using "traditional" methods?

Yes, this looks like a (hopefully) better information retrieval system than running CTRL+F over a PDF. Nothing wrong with that :) information retrieval is hard. And CTRL+F won't render a table for queries like "what is the % increase of revenue over the last 3 quarter"-type queries.

Yes, information retrieval is hard :) A lot of people ask for 'can you get Apple's revenues by product category for all of 2020', how do you get the smallest piece of text that has the most information about product category and revenues ?I can get a lot of text, but that would mean I probably run over the context limit and so on :)

Re: Reading SEC filings using LLMs

#59

Post by nnechm »

gen220 wrote: post #56

Beware Gell-Mann amnesia when assessing how useful LLMs are for investing research. Don't forget the instinctual response you have to LLMs' supposed abilities to write software projects, when assessing LLMs' abilities to comprehend and extract the interesting/relevant parts of a 10-Q/K. IME, the most useful software tools in investing are analogous to the most useful software tools in software-building: the IDEs, lib…

I think you are referring to the more hyped version where somehow LLMs can figure out how to get the most relevant information from 10Ks and do most of the investment process. My mental model is simply that we have an assistant, an amazing assistant that can read pretty well, and we can use them in the investment process.

Re: Reading SEC filings using LLMs

#60

Post by jatinshah »

The main use case highlighted here seems to be retrieving quantitative data from financial reports and then doing sophisticated analysis (trendlines, forecasting etc) using ChatGPT (basically what’s currently done in Excel).

I doubt any finance professional would want to move from Excel to ChatGPT for these use cases.

A secondary issue is that numbers in financial reports need to be standardized before they can be used for analysis - say comparing ROE of a stock with industry average. That’s not possible with numbers from raw reports.

Post reply on HN