Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine learning for financial prediction

robotwealth.com

Post reply on HN
Print view View original

41–50 of 84 posts

Re: Machine learning for financial prediction

#41

Post by CPLX »

egwor wrote: post #11
aj7 wrote: post #8

Really successful traders spend their obtaining insider information, not massaging public data. It stands to reason that an ensemble of technical trading methods would regress towards the mean.

Using insider information is illegal. If you mean insider information then this is total nonsense. Sure, there are a few that do illegal things (and inevitably get caught since there is so much monitoring going on).

It's only illegal if you are an insider or got it from an insider who realized a gain on it. There's a lot of misconceptions here about what insider trading actually is.

Re: Machine learning for financial prediction

#42

Post by zepto »

dsacco wrote: post #24
falsestprophet wrote: post #13

Earlier quoted context omitted.

Exactly, when I heard drugs usage was a problem in some communities I too knew that was impossible because selling and using drugs is illegal. Sure, there are a few that do illegal things (and inevitably get caught since there is so much monitoring going on).

Your middlebrow dismissal doesn't work here because the parent didn't say it's not a problem or that it doesn't happen. The parent is refuting a point that successful traders exclusively become that way by trading on illegally obtained information.

The point being refuted is a straw man - nobody said anything about 'exclusively'.

Re: Machine learning for financial prediction

#43

Post by dsacco »

zepto wrote: post #42
dsacco wrote: post #24

Earlier quoted context omitted.

Your middlebrow dismissal doesn't work here because the parent didn't say it's not a problem or that it doesn't happen. The parent is refuting a point that successful traders exclusively become that way by trading on illegally obtained information.

The point being refuted is a straw man - nobody said anything about 'exclusively'.

No it isn't. The original comment might not have said "all successful traders spend their time...insider information" but the implication is there as it was stated. Given what the grandparent comment replied with, it appears I wasn't the only one who inferred that message.

Re: Machine learning for financial prediction

#44

Post by kasey_junk »

aj7 wrote: post #8

Really successful traders spend their obtaining insider information, not massaging public data. It stands to reason that an ensemble of technical trading methods would regress towards the mean.

You probably mean "information that others do not have". That is not the same thing as "insider information".

Re: Machine learning for financial prediction

#45

Post by mathgenius »

It's just so ridiculously easy to overfit these models, and so so many ways to shoot yourself in the foot as a result.

For example, "I split the data set into 5 random segments and then trained a model on 4 of the 5 segments and then tested it on 5th." Such data is serially correlated (it's not good old iid) so already it looks like you have poisoned the test set with information from the training set.

The hard part is not "feature engineering" or "ensemble methods", the hard part is controlling the entropy that you feed these things because they are voracious monsters and will absolutely eat all of it.

Re: Machine learning for financial prediction

#46

Post by zepto »

dsacco wrote: post #43
zepto wrote: post #42

Earlier quoted context omitted.

The point being refuted is a straw man - nobody said anything about 'exclusively'.

No it isn't. The original comment might not have said "all successful traders spend their time...insider information" but the implication is there as it was stated. Given what the grandparent comment replied with, it appears I wasn't the only one who inferred that message.

Just because more than one person infers something doesn't mean it's really there.

Re: Machine learning for financial prediction

#47

Post by lordnacho »

nxzero wrote: post #16
lordnacho wrote: post #7

Earlier quoted context omitted.

Why would the SEC audit you? Just by random chance there are many people outperforming the market. They can't audit you just for outperforming. If they do audit you, how will they discover how you are generating your trading decisions? Their remit is to make sure you aren't doing something illegal. There's no reason they would understand what you were doing in anything other than a superficial way. Also, something ca…

Few years back, SEC started being very agressive about finding entities making above average returns; my understanding is that if over a set amount of transactions you're making over 30% that you will get "knocked" and the auditors have zero reason not to leak the information. Best example I know is the Walmart parking lot satellite imagery analysis; happy to dig up a link.

That sounds like weeding out insider trading, not finding people with legitimate market beating strategies.

If you're trading on confidential information, your profile will look very interesting indeed. You'll be trading near announcements, and you'll be right all the time. Your turnover vs profit and number of trades will be through the roof. By contrast quant shops with real models will be using the law of large numbers.

Re: Machine learning for financial prediction

#48

Post by lordnacho »

Wonnk13 wrote: post #15
lordnacho wrote: post #6

Interesting article. I do something related, and here's my take: Data mining is useful because it gives you things that are predictive that you might not have considered at first, but make sense after. This is mainly due to combinatorial explosion in the potential number of formulas. You generally have a vague idea of what might be predictive, eg cheapness vs earnings and cash flow, but there's a huge number of ways…

very interesting. Can you recommend any resources for someone with a fairly strong stats / programming background but no real substantive finance experience?

Igor Tulchinsky has a fund that does this. He also writes books and papers about how he does it, with everything you need to do it yourself.

Re: Machine learning for financial prediction

#49

Post by mathgenius »

mcbrown wrote: post #35

Former professional investment manager here... The biggest problem with things like this, which almost nobody talks about in the context of investing, is publication bias. 100 people try to develop a profitable trading algorithm. 1 comes up with one that looks great on back-tests at a 1% confidence (in other words, exactly what you'd expect from random chance alone over 100 trials). That person writes an article/pitc…

> 100 people try to develop a profitable trading algorithm....

It's much worse than this with machine learning approaches. Imagine a million people trying to find a profitable algo, all on your laptop, and you are choosing the best one out of all of those.

If you are used to pen-and-paper trading strategies, or even excel spreadsheets, machine learning is just a completely different level to this. And probably how it works will be unintelligible to anyone. I don't even see how someone can write a business plan based on this.

Re: Machine learning for financial prediction

#50

Post by bladecatcher »

xivzgrev wrote: post #23

Anyone know where he got all the raw data to feed his algo? Clearly he used a lot of data and the two main sources of free info i know of are google finance and yahoo finance. At least with google finance i run into issues with their api if you execute too many calls simultaneously, a bunch end up not returning any data

In my experience, Yahoo finance data is not reliable. In one case, I noticed that the stock price is incorrectly adjusted for dividends for all shares trading on a particular exchange. Free correct data is hard to obtain.
Post reply on HN