The anatomy of an ML-powered stock picking engine

No. Because to train the sentiment model you need estimable distributions.

Thanks for sharing! Really curious about ML stock market models as they seem extremely difficult to outperform the market consistently over time.

A few questions:

1) Were these stock picks for major stocks / ETFs? Or small market cap stocks?

2) How many people were subscribed to your newsletter?

3) What do you estimate the impact was of creating a “self-fulfilling prophecy” of entering a position and then recommending your subscribers take the same position?

4) Do you think your asset mix outperformed the market by picking high risk / high reward stocks in a bull market? Or picking safe stocks in a bear market? In other words, how do you know the engine wasn’t biased towards the market trend that happened to play out? For example, if I have a basket of tech stocks that would typically outperform in a bull market and flip a coin to buy or short them and guess right, I could outperform the market by chance. How did you account for this?

5) Have you backtested the engine to see what it would have returned in previous years? (Obviously on unseen data, rather than data it used for training)

"steadily beating the S&P 500 for over a year on a weekly basis"

Can be achieved by chance alone.

If not chance, I can give you a strategy that would be highly likely to achieve such a result: it would take a lot of risk though!

I love it when people post this stuff to HN. Naive people try it, loose a bundle to market makers, then go back to their day job.

EDGAR filings (structured text) is an area unto itself, I see you've limited yourself to quarterlies.

Across any market area (eg: mineral resources) there are thousands of documents released daily across multiple exchanges (via EDGAR, SEDAR, etc) ranging from two line advisories, to 4,000 page technical reports on projects | acquisitions, alongside the usual quarterly | yearly annual reports, etc.

There's plenty to do parsing common forms for generic changes (board members, board member share changes, etc) and market regime specifics (exploration property aquisition) and trends (series of related aquisitions) for those that like the weeds.

Some might argue that 'understanding' these patterns lead the changes in stock price movements, and give insight wrt weathering short term changes for longer term returns.

This was a very enjoyable read. I built a nearly (architecturally) identical system a few years back that also had to be scrapped for different reasons. This brought back a lot of memories. The sanity checks, the index reconstitution issues, dealing with the insanity of security identification and tracking through time.

The fun cases are the ones where it's not even clear what the right answer truly is, e.g. company A spins out company B, and then 5 years later they re-merge. Who's time series and associated data is "the" canonical one? The data vendors often try to give their answers to this question, but maybe their answers don't make sense for your analysis.

Then there's the fact that a lot of vendors don't really do point in time correctly. They like to go back and helpfully revise data points for you that they or the company initially misreported. This is all well and good except that if you were trading for real, you wouldn't have known the correct information at the time, and so any backtest based on the updated information will be invalid. Vendors are a bit better now about providing true point in time data sets, or at the very least accurately describing when they are/aren't doing this. But we had a few cases where they said they were, but they definitely weren't.