Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Bedrock AI (YC S21) – Using ML to identify red flags in SEC filings

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 109 posts

Re: Launch HN: Bedrock AI (YC S21) – Using ML to identify red flags in SEC filings

#41

Post by piesauce »

paisible wrote: post #39
kbennatti wrote: post #26

Earlier quoted context omitted.

Ooh I love MTL. That's the first place I lived in Canada. Great q. We used SEC enforcement actions related to fraud as our gold label (fairly common practice in academia). Really key thing here is that you need to be careful about what years you're using for training because if you include years that are too late in the fraud cycle, you end up with significant target leakage e.g. the filings we'll say, "we're being i…

Thanks for the quick answer! Follow-up around this as it's a space I'm actively working in: did you use or build any tools for the labeling process, or was it Excel? :D Also, do you ultimately see/position your solution as an AI-powered exploration tool that allows humans to derive better insights, faster (but where the NLP side of things is simply to assist in this discovery process), or do you see the models (and r…

Our annotation process has been manual so far, but we are working on building something to make it more efficient :-) We see our solution as an AI-powered assistant for qualitative research that makes the job of an analyst much easier, and don't see it as 'replacing' humans for the foreseeable future.

Re: Launch HN: Bedrock AI (YC S21) – Using ML to identify red flags in SEC filings

#42

Post by piesauce »

ZeroCool2u wrote: post #40

I've worked with 10-K's and 8-K's extensively for the purposes of using them for NLP. This is extremely arduous work and a clear winner in terms of profitable ideas, so kudos to the team for the launch, this is really impressive. Perhaps this is giving a bit too much away in terms of the secret sauce, but would love if you could talk a bit about how you handle the wild disparities in the structure of the documents. D…

Thanks for the kind words! We don't use XBRL at all. We did try it initially, but it was wildly inconsistent across companies. I think one of the things that worked well for us was that we spent a lot of time at the initial stages of the pipeline (efficient sentence and word tokenization, span detection), that bode well for our models later on.

Re: Launch HN: Bedrock AI (YC S21) – Using ML to identify red flags in SEC filings

#43

Post by piesauce »

naturalauction wrote: post #34
kbennatti wrote: post #6

Earlier quoted context omitted.

Great q. Sino-Forest is an out-of-sample test so our models didn't technically "learn" from it. That said, very valid comment. Historical testing only goes so far. Assessing whether our algorithms work in deployment has been cool. Check out some of our live, in deployment examples here - https://bedrock.substack.com/p/bedrock-ai-vs-activist-shorts

> https://bedrock.substack.com/p/bedrock-ai-vs-activist-shorts How often are companies rated with a risk factor this high. As in does a risk factor in the 80s mean that fraud is extremely likely or is it just notifying humans that this filing might be worth reading over with a fine-toothed comb.

Less than 10 percent of companies have a risk score above 80. Our historical testing shows that around 1/3rd of companies with scores above 80 turn out to be fraudulent. (It is hard to test this so this might be an overestimate)

Re: Launch HN: Bedrock AI (YC S21) – Using ML to identify red flags in SEC filings

#45

Post by kbennatti »

riku_iki wrote: post #44

> in the company’s disclosures including buying and selling from companies controlled by their directors How did you find that companies are controlled by their directors?

It's disclosed in their filings e.g. "Among the vendors were a director of the Company and an entity controlled by such director"

At lot of egregious things are disclosed on page 101 of a filing but they get missed because these filings are so long and deathly boring

Re: Launch HN: Bedrock AI (YC S21) – Using ML to identify red flags in SEC filings

#46

Post by bambax »

> Most public company data is unstructured and textual.

This is surprising to me.

Wouldn't it be possible to make a very short list of, say, 12 blunt questions that would help flag fraud?

Of course the company could be lying in the answers. (If it was found to be lying in any one answer then it would automatically be flagged as "high risk".)

But this may not even be needed, since what you seem to be saying is that the information about fraud is there in plain sight, yet hard to see, because it is drowned in fluff and periphrases.

Is there an opportunity for a private company, say a rating company, to make and distribute such a questionnaire? Or does it already exist in some form?

Re: Launch HN: Bedrock AI (YC S21) – Using ML to identify red flags in SEC filings

#47

Post by htrp »

mynegation wrote: post #15

How would you address would-be filers from using your own product from iterating on their wording until the red flags are removed?

Realistically they probably don't need to do that. The filings are usually pretty heavily lawyered up and the lawyers are pretty lazy on language updates.

Re: Launch HN: Bedrock AI (YC S21) – Using ML to identify red flags in SEC filings

#48

Post by bambax »

fritzi wrote: post #9

There have been several popular books about famous fraudsters. I suspect that you come across some facts that would be interesting (if not profitable) not just to institutional investors but to the average nerd, or maybe someone looking for an idea for the next bestseller.

"Lying For Money" by Dan Davis has several chapters about financial fraud. It's a great read.

Re: Launch HN: Bedrock AI (YC S21) – Using ML to identify red flags in SEC filings

#49

Post by somberi »

kbennatti wrote: post #38
Seabiscuit wrote: post #36

Interesting site. I currently work at a hedge fund, but have a small dose of NLP in my academic background, so it's always interesting to see concepts like this come out. Two questions: - Are you using EDGAR's 'Facts' function? It seems to make SEC Filings a lot more like structured text than they have been previously, but I haven't seen really convincing tools developed to use it yet - How/do you ever see yourself i…

Thanks for website edit! Fixed. We don't rely on XBRL for parsing. It's not very consistent/reliable and its mostly for numeric content. We've definitely considered integrating ratios both into our dashboard. It isn't a current priority because ratios are already well supported elsewhere.

Asking to learn - ex-Algo here. May I ask where else ratios are represented. Thanks and all the best for your launch. Very useful service.

From experience, I would suggest you have way for the manager of a fund or a desk or a bank, to see the usefulness. You will have good pull from the line staff, but selling to the managers is the hard part.

Re: Launch HN: Bedrock AI (YC S21) – Using ML to identify red flags in SEC filings

#50

Post by piesauce »

somberi wrote: post #49
kbennatti wrote: post #38

Earlier quoted context omitted.

Thanks for website edit! Fixed. We don't rely on XBRL for parsing. It's not very consistent/reliable and its mostly for numeric content. We've definitely considered integrating ratios both into our dashboard. It isn't a current priority because ratios are already well supported elsewhere.

Asking to learn - ex-Algo here. May I ask where else ratios are represented. Thanks and all the best for your launch. Very useful service. From experience, I would suggest you have way for the manager of a fund or a desk or a bank, to see the usefulness. You will have good pull from the line staff, but selling to the managers is the hard part.

Thanks for your wishes! For ratios, We love Alpha Vantage. They are great value for money.
Post reply on HN