Live data from Hacker News

Machine learning for financial prediction

robotwealth.com

51–60 of 84 posts

Re: Machine learning for financial prediction

#51
post #11
post #8

Really successful traders spend their obtaining insider information, not massaging public data. It stands to reason that an ensemble of technical trading methods would regress towards the mean.

Using insider information is illegal. If you mean insider information then this is total nonsense. Sure, there are a few that do illegal things (and inevitably get caught since there is so much monitoring going on).

[deleted]

Re: Machine learning for financial prediction

#52
post #19
post #12

Earlier quoted context omitted.

This is exactly true. I had the pleasure of having a very short sit down with one of the world's most successful traders and he said this in not so many words. It was more subtle than inside information. He implied that he could actually influence the outcome and hurried me on from that point. Mind blown.

Sounds like Billions is not that far off...

Never heard of it.

Re: Machine learning for financial prediction

#53
post #12

Earlier quoted context omitted.

This is exactly true. I had the pleasure of having a very short sit down with one of the world's most successful traders and he said this in not so many words. It was more subtle than inside information. He implied that he could actually influence the outcome and hurried me on from that point. Mind blown.

Please do share more if you have anything.

It seemed like the game is not necessarily rigged, but information is power. Regular traders are playing for table scraps compared to the really huge people who play politics, not markets.

For example, http://priceonomics.com/the-trade-of-the-century-when-george...

What did Soros know about what would happen and what did he actually cause to happen? The story sounds nice on the surface (I made a multi-billion dollar bet on a quote from some guy - lol), but I wouldn't be surprised if there was some back channelling going on.

Re: Machine learning for financial prediction

#56
post #2

If anyone else is getting errors when loading the page, here's the google cached version http://webcache.googleusercontent.com/search?q=cache:-ciyXfS...

More tools for ML financial prediction

http://52.11.211.67/recommend/app/hidden_connections?query=h...

http://52.11.211.67/recommend/historical-trends/index-contra...

Re: Machine learning for financial prediction

#57

It's just so ridiculously easy to overfit these models, and so so many ways to shoot yourself in the foot as a result. For example, "I split the data set into 5 random segments and then trained a model on 4 of the 5 segments and then tested it on 5th." Such data is serially correlated (it's not good old iid) so already it looks like you have poisoned the test set with information from the training set. The hard part…

This is one of the better responses. Issues that arise: low # of data points at macro timescale, time series data (and local correlation between individual data points) making it hard to extract training/testing sets, and the overarching structural shifts in the market over time that invalidate older data (depending on context).

Re: Machine learning for financial prediction

#58
post #10
post #5

Never understood why anyone would spend time creating any trading method given even if it did work (possible, but unlikely) the SEC would audit you and then leak how you were making the outperforming returns. Welcome any thoughts, in part because legally beating the market is possible, just don't get the SEC & OPSEC aspect.

The problem isn't coming up with an algorithm that works (i.e. more wins than losses). The difficulty is gaining confidence in your algo and determining when to move from paper trading to actual trading. You run into counter-intuitive things while training a neural net, for example. You'd think more training data would be good, but when training neural nets, you actually want to use as little data as possible while s…

> The problem isn't coming up with an algorithm that works (i.e. more wins than losses).

An algorithm that works would also include the ability to limit losses. An algorithm might be correct 9 out of 10 times, but may lose more in a single transaction than what it earned in those 9 winning transactions.

Re: Machine learning for financial prediction

#59
post #35

Former professional investment manager here... The biggest problem with things like this, which almost nobody talks about in the context of investing, is publication bias. 100 people try to develop a profitable trading algorithm. 1 comes up with one that looks great on back-tests at a 1% confidence (in other words, exactly what you'd expect from random chance alone over 100 trials). That person writes an article/pitc…

> 100 people try to develop a profitable trading algorithm.... It's much worse than this with machine learning approaches. Imagine a million people trying to find a profitable algo, all on your laptop, and you are choosing the best one out of all of those. If you are used to pen-and-paper trading strategies, or even excel spreadsheets, machine learning is just a completely different level to this. And probably how it…

The type of approach used has limited effect on survivorship bias, what matters is the number of people employing different approaches and the size of the effect. So if machine learning approaches can produce real results, the data will show this. Survivorship bias is real, but it is not the full story.

Re: Machine learning for financial prediction

#60
If you can actually reliably generate alpha from a model like this there is no point of running the strategy yourself. There are any number of hedge funds that will sign you on, let you keep all of the IP you develop, and give you 10-12% of any returns you generate. That sounds small, but it's mitigated by the fact that you will have access to potentially billions of dollars in capital to trade if your strategy has the capacity for it. So you get 10% of a much bigger pie, with way less downside risk. Plus you get access to all their internal trading systems, execution services, data feeds, etc, which are usually orders of magnitude better than what an individual has access to.
Post reply on HN