The anatomy of an ML-powered stock picking engine • Hacker News Board

Re: The anatomy of an ML-powered stock picking engine

#81

Post by muggermuch » Wed, Sep 28, 2022, 3:41 AM UTC

darawk wrote: post #79

This was a very enjoyable read. I built a nearly (architecturally) identical system a few years back that also had to be scrapped for different reasons. This brought back a lot of memories. The sanity checks, the index reconstitution issues, dealing with the insanity of security identification and tracking through time. The fun cases are the ones where it's not even clear what the right answer truly is, e.g. company…

Thank you, and ha! An emphatic yes to all the points you raised! It's especially daunting when there are multiple vendors with incompatible point-in-time hygiene setups, a situation I faced at the beginning of setting up Didact.

Also, this was really my first time with equities - my professional trading career was derivatives-focused - both listed (CME) and OTC (FX forwards/swaps). I think I lost the first few months simply trying to reorient my style of thinking.

Water under the bridge I guess.

Re: The anatomy of an ML-powered stock picking engine

#82

Post by Straw » Wed, Sep 28, 2022, 4:34 AM UTC

gbasin wrote: post #37

If your predictions are good, I'd be happy to get you $100 million in assets to manage. It's very unlikely that your predictions are good...

I've been thinking about trying to build something from scratch with a similar spirit, but very different methods, but I also doubt that my predictions will be any good without far more time investment than I have available.

As you say you have expertise in the area, any chance I could ask you for advice or how to decide if its worth trying?

Re: The anatomy of an ML-powered stock picking engine

#83

Post by ackbar03 » Wed, Sep 28, 2022, 4:36 AM UTC

HFguy wrote: post #50

chollida1 wrote: post #35

Someone asked about how difficult it is to get outside investment.... It's usually very difficult and it takes a lot of money to run a proper fund. Let's say you raise $50M. You can maybe charge 1 and 20,meaning you get 1% of assets each year for running the fund and 20% of profits. 1% of $50M( and keep in mind this is a large raise for someone without a track record on the sell side or inside another fund) give you…

This is actually way too optimistic. Your first 1-2 seed investors will: - Only pay 1 and 10 (1% fixed fee and 10% of PNL) - They will also get ownership of the actual fund management firm and will get that in the form of 20% of REVENUE (not equity, revenue, think about that) This is one reason new fund formation is way down. The economics are bad for years. Know a bunch of HF people that started vc-backed tech firms…

I guess if you have a reasonably profitable strategy it probably just makes more sense to run it with your own money? I guess the only reason why you would want to trade as a hedge fund is if you want to scale up, but most strategies aren't really that scale-able anyways from what I understand, since when you start trading in any real significant size you start moving the markets.

Re: The anatomy of an ML-powered stock picking engine

#84

Post by muggermuch » Wed, Sep 28, 2022, 4:51 AM UTC

defrost wrote: post #77

muggermuch wrote: post #2

Hi, fellow HN'ers! Author here, please let me know if you have any questions or thoughts!

EDGAR filings (structured text) is an area unto itself, I see you've limited yourself to quarterlies. Across any market area (eg: mineral resources) there are thousands of documents released daily across multiple exchanges (via EDGAR, SEDAR, etc) ranging from two line advisories, to 4,000 page technical reports on projects | acquisitions, alongside the usual quarterly | yearly annual reports, etc. There's plenty to d…

This is a very insightful remark, thank you.

I focused on 10-Qs for the EDGAR filings module as you rightly pointed out - it seemed to be a good balance between implicit information and usefulness of the data. TBH I didn't actually investigate the other (many) patterns.

Having said that, I have really enjoyed Kai Wu's research from Sparkline Capital (https://www.sparklinecapital.com/), especially his extraction of the innovation factor from EDGAR filing texts. He's appeared in numerous podcasts, and they have all been super useful to listen to. Maybe someday when I re-investigate EDGAR filings and go further, I might target these signals you talk about here.

Re: The anatomy of an ML-powered stock picking engine

#85

Post by defrost » Wed, Sep 28, 2022, 5:07 AM UTC

muggermuch wrote: post #84

defrost wrote: post #77

Earlier quoted context omitted.

EDGAR filings (structured text) is an area unto itself, I see you've limited yourself to quarterlies. Across any market area (eg: mineral resources) there are thousands of documents released daily across multiple exchanges (via EDGAR, SEDAR, etc) ranging from two line advisories, to 4,000 page technical reports on projects | acquisitions, alongside the usual quarterly | yearly annual reports, etc. There's plenty to d…

This is a very insightful remark, thank you. I focused on 10-Qs for the EDGAR filings module as you rightly pointed out - it seemed to be a good balance between implicit information and usefulness of the data. TBH I didn't actually investigate the other (many) patterns. Having said that, I have really enjoyed Kai Wu's research from Sparkline Capital ( https://www.sparklinecapital.com/ ), especially his extraction of…

You're welcome.

14+ years back a small group of West Australians put together what became

https://www.spglobal.com/marketintelligence/en/campaigns/met...

which was based upon integrating (GIS and regular DB) every daily mineral lease record across the accessable globe together with every publicly filed document across the relevant stock markets (AU, TSX, South Africa, London, etc) using (and updating|refing) templated patterns that appear in various classes of forms .. I would assume that territory has been revisited with better ML techniques.

Re: The anatomy of an ML-powered stock picking engine

#86

Post by sbierwagen » Wed, Sep 28, 2022, 5:44 AM UTC

notacop31337 wrote: post #52

gbasin wrote: post #37

If your predictions are good, I'd be happy to get you $100 million in assets to manage. It's very unlikely that your predictions are good...

It's very unlikely that you're able to get OP $100 million in assets...

"Inadequate Equilibria".

If you picked a HN comment at random, then the person who made that comment are overwhelmingly unlikely to raise $100MM at the drop of a hat. Picking a HN user at random won't do it either.

But there's not a lot of finance-related submissions on HN. The people in the comments may be unusual. And off the top of my head there certainly is an existence proof for at least a few investors on the news site for Y Combinator, a VC fund, such as pg or sama. There are obviously other VC people on here as well.

Re: The anatomy of an ML-powered stock picking engine

#87

Post by madmax108 » Wed, Sep 28, 2022, 6:30 AM UTC

jjoonathan wrote: post #69

Joel_Mckay wrote: post #60

Earlier quoted context omitted.

Some simply build a portfolio by copying those who can't be charged for violating market rules. Not sure why some folks find this strategy so controversial. =) Congress member holdings report: http://clerk.house.gov/public_disc/financial-search.aspx Senate member holdings report: https://efdsearch.senate.gov/search/

Is there an ETF yet?

Unsure if this was a joke Q, but the answer may just be yes:

"Two proposed exchange-traded funds would mimic stock trades made by members of Congress and their spouses. If approved, the ETFs would track trades by Democrats and Republicans, under tickers NANC and CRUZ. "

https://markets.businessinsider.com/news/etf/stocks-etfs-nan...

Re: The anatomy of an ML-powered stock picking engine

#88

Post by maCDzP » Wed, Sep 28, 2022, 6:41 AM UTC

alpineidyll3 wrote: post #55

My heart goes out to this author, but you can tell even by his first table that he doesn't quite understand the mathematics of financial markets, the purpose of a hedge fund, how they grow etc. 1) It's plain by quickly looking at the allocation of capital in investment firms, that AUM is not made by performance; it's marketing. At best people invest when they believe a person is connected to inside information. Sayin…

I found your comments about rediscovering Kelly et al interesting. Could you recommend some textbooks that describes what you are referring to? If there are good overviews of the subject?

Re: The anatomy of an ML-powered stock picking engine

#89

Post by rohitb91 » Wed, Sep 28, 2022, 7:08 AM UTC

adamsmith143 wrote: post #21

muggermuch wrote: post #7

Earlier quoted context omitted.

Thank you for appreciating the article; I tried to disclose all that I could! 1. Yes, I did put my own money in it (low 6 figures). 2. It went as described in the article - for the capital I allocated to Didact, I beat the market (SPY) by ~20% since inception. 3. If I understand your question correctly, this would be the equivalent of the payoff on an optimal lookback option ( https://en.wikipedia.org/wiki/Lookback_o…

>2. It went as described in the article - for the capital I allocated to Didact, I beat the market (SPY) by ~20% since inception. This seems extremely hard to believe. You should be running a multi-billion $ Quant fund if this is the case. The idea that you would try to push this as a newsletter rather than just taking investor money and becoming a billionaire literally makes the story seem farcical.

Definitely possible doing so many things. Following trend and just being in DXY or short SPY. It's a super short time-frame. Anything can happen. Trust test is 10 year + horizons.

Re: The anatomy of an ML-powered stock picking engine

#90

Post by TrackerFF » Wed, Sep 28, 2022, 8:25 AM UTC

chollida1 wrote: post #35

Someone asked about how difficult it is to get outside investment.... It's usually very difficult and it takes a lot of money to run a proper fund. Let's say you raise $50M. You can maybe charge 1 and 20,meaning you get 1% of assets each year for running the fund and 20% of profits. 1% of $50M( and keep in mind this is a large raise for someone without a track record on the sell side or inside another fund) give you…

Hardest part is raising those $50M - even more so if you have zero professional experience in high-finance. Getting a foot inside investment banking, hedge funds, private equity, etc. is extremely competitive to say the least.

I think the best shot for any outsider programmer would be to seek (and team up with) those finance professionals that are already thinking about exiting to start their own funds, and in the need of some technical partner...but even then, you're also competing against experienced devs. already in the field.

In the end, it is just really, really difficult for outsiders to just enter this sector, if they have any hopes of working with any substantial amount of capital.

I guess the better option would be to make some product you can sell as SaaS to the masses, or figure out how to manage thousands and thousands of low-$ investors.