Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine learning for financial prediction

robotwealth.com

Post reply on HN
Print view View original

31–40 of 84 posts

Re: Machine learning for financial prediction

#31

Post by eggy »

Bromskloss wrote: post #25
partisan wrote: post #21

Earlier quoted context omitted.

Googling "walmart parking lot analysis" yielded the following as the first result. http://www.cnbc.com/id/38722872

Thanks. It has no information about the technique being leaked by the SEC, though.

Not sure about SEC leaking anything, but satellite data is a tool for investors to use satellite imagery, and image processing to see things like how many cars are in several big branches, make assumptions and correlations to spending, and then act on the information before Walmart releases a quarterly statement, sort of.

Number of container ships docked or leaving port around China can forecast trends in China's exports, again before any official numbers are made available. I don't see this as insider trading. You pay for the satellite time, you gamble on your data analysis, and you either win or lose. If it were certain, others do the same, and the edge is lost quickly by market adjustments.

Re: Machine learning for financial prediction

#32

Post by jzwinck »

melling wrote: post #28
dpweb wrote: post #20

There are a few problems with turning your laptop into a money machine using data analysis. Remember the maxim, past performance is not a guarantee of future results. You can develop strategies based on past data that will beat the market, but, the nature of markets is to adapt to kill your edge. Markets adapt constantly and your edge stops working at an unknown point in time. It's unknowable when that WILL happen be…

"nature of markets is to adapt to kill your edge" If you are a low volume, small time trader, the market isn't going to move as quickly to adapt to you. If you have $100,000, for example, and return 30% a year, you aren't on anyone's radar.

30% on 100k is 30k. You'd be better off getting a regular job unless you can sustain that for more than 10 years. Which you can't predict.

Re: Machine learning for financial prediction

#33

Post by chvid »

xivzgrev wrote: post #23

Anyone know where he got all the raw data to feed his algo? Clearly he used a lot of data and the two main sources of free info i know of are google finance and yahoo finance. At least with google finance i run into issues with their api if you execute too many calls simultaneously, a bunch end up not returning any data

You can get data from Interactive Brokers; I assume most of the other brokers that provide an API will give you data too.

Re: Machine learning for financial prediction

#34

Post by melling »

jzwinck wrote: post #32
melling wrote: post #28

Earlier quoted context omitted.

"nature of markets is to adapt to kill your edge" If you are a low volume, small time trader, the market isn't going to move as quickly to adapt to you. If you have $100,000, for example, and return 30% a year, you aren't on anyone's radar.

30% on 100k is 30k. You'd be better off getting a regular job unless you can sustain that for more than 10 years. Which you can't predict.

No one said that this was your full-time job. No one said that it has to be HFT either. Algorithms identify good trades at 4:01pm and you buy the next day. And no one is saying that you have to trade everyday.

Re: Machine learning for financial prediction

#35

Post by mcbrown »

Former professional investment manager here...

The biggest problem with things like this, which almost nobody talks about in the context of investing, is publication bias.

100 people try to develop a profitable trading algorithm. 1 comes up with one that looks great on back-tests at a 1% confidence (in other words, exactly what you'd expect from random chance alone over 100 trials).

That person writes an article/pitch/business plan based on their algorithm. You never see results from the 99 who failed.

Going forward, the successful algorithm is no more likely to work than the failed 99, but from the perspective of the general public it sure looks like a winner!

Re: Machine learning for financial prediction

#36

Post by SixSigma »

xivzgrev wrote: post #23

Anyone know where he got all the raw data to feed his algo? Clearly he used a lot of data and the two main sources of free info i know of are google finance and yahoo finance. At least with google finance i run into issues with their api if you execute too many calls simultaneously, a bunch end up not returning any data

You can get constant updates from IG via scraping. Market prices are fractal I.e. Self similar at any scale

Re: Machine learning for financial prediction

#37

Post by branchless »

jzwinck wrote: post #32
melling wrote: post #28

Earlier quoted context omitted.

"nature of markets is to adapt to kill your edge" If you are a low volume, small time trader, the market isn't going to move as quickly to adapt to you. If you have $100,000, for example, and return 30% a year, you aren't on anyone's radar.

30% on 100k is 30k. You'd be better off getting a regular job unless you can sustain that for more than 10 years. Which you can't predict.

Exactly. Plus let's remember you are adding nothing to life here. All you are doing is collecting 30k off other people with a slightly less optimal "strategy" than you.

And no I don't believe these people are "adding liquidity and assisting price discovery".

to the reply as I can't post as HN censors detractors of big finance:

I don't believe the benefits of liquidity added by HFT are worth the enormous costs firms sink into it.

Re: Machine learning for financial prediction

#38

Post by branchless »

mcbrown wrote: post #35

Former professional investment manager here... The biggest problem with things like this, which almost nobody talks about in the context of investing, is publication bias. 100 people try to develop a profitable trading algorithm. 1 comes up with one that looks great on back-tests at a 1% confidence (in other words, exactly what you'd expect from random chance alone over 100 trials). That person writes an article/pitc…

So much this. Also even if you have won it means very little going forward. If you put 100 guys in a room and asked them to try to flip N consecutive tails one guy will come out thinking he is the king of flipping, with a rock-solid "system". He's just someone who doesn't understand probability. And as you say you don't hear from the other 99 including the math guy who flipped N/2 heads and is muttering about it.

Re: Machine learning for financial prediction

#39

Post by JoeAltmaier »

mcbrown wrote: post #35

Former professional investment manager here... The biggest problem with things like this, which almost nobody talks about in the context of investing, is publication bias. 100 people try to develop a profitable trading algorithm. 1 comes up with one that looks great on back-tests at a 1% confidence (in other words, exactly what you'd expect from random chance alone over 100 trials). That person writes an article/pitc…

There's an old con game - you send 500 letters to gamblers, predicting the next Dodgers game. 250 predict they'll win; 250 say lose. Game happens, 250 people think Hey lucky guess. To those you send 250 letters, 125 predict they'll win the next game; 125 lose. After 6 games you have 8 people who have seen you guess right 6 times in a row. Get them to pay you for another (worthless) letter.

Re: Machine learning for financial prediction

#40

Post by dsacco »

branchless wrote: post #37
jzwinck wrote: post #32

Earlier quoted context omitted.

30% on 100k is 30k. You'd be better off getting a regular job unless you can sustain that for more than 10 years. Which you can't predict.

Exactly. Plus let's remember you are adding nothing to life here. All you are doing is collecting 30k off other people with a slightly less optimal "strategy" than you. And no I don't believe these people are "adding liquidity and assisting price discovery". to the reply as I can't post as HN censors detractors of big finance: I don't believe the benefits of liquidity added by HFT are worth the enormous costs firms s…

>> And no I don't believe these people are "adding liquidity and assisting price discovery".

The nice thing about reality is that it remains even when your belief persists against it.

In large-cap stocks during rising markets, high frequency trading does improve liquidity[1]. While the effect may not be as prevalent during a downturn and it may not impact smaller stocks as much, I'd like to see your evidence that it actively harms the market or that the practice is vapid and produces nothing of value.

Or is that just a statement you made because it nicely aligns with your political conception of Wall St?

EDIT: It helps to point out that "algorithmic trading" and "high frequency trading" are not at all the same thing, especially as these terms are usually conflated on HN. An algorithmic trading system does not necessarily need to trade at high frequency. Some algorithmic trading systems make trades in intervals of days or weeks, not seconds or milliseconds. The paper cited here describes the market-making activities of what is traditionally called high frequency trading and the benefits it has over human brokers of the past, but it uses the umbrella term "algorithmic trading."

EDIT 2: The parent comment responded to this one by editing his original one, because "HN censors detractors of big finance." You also claimed you don't believe that the liquidity provided by HFT is worth the capital that large firms dump into it.

In 2013 the entire HFT industry made about $1B, down from $5B in 2009[2]. HFT is not a large industry. It is eating much of Wall Street's traditional market-making inefficiencies, which is why it is widely disliked, but it is not "big finance." Big Finance is generally opposed to HFT.

You still haven't provided evidence or numbers to prove or even quantify what you're claiming. Are you saying HFT is not worth the investment to firms, or are you saying it isn't providing some vague "value to society" relative to alternative uses of investor capital?

The first case is obviously nonsensical, as many firms generate profit using high frequency trading strategies. The second case is like saying we shouldn't do anything if it doesn't save impoverished children in Africa. The added liquidity has a material and beneficial impact on trading outcomes for buy-and-hold retail investors, which is shown in my first citation here. You have yet to satisfactorily refute this.

[1]: http://faculty.haas.berkeley.edu/hender/Algo.pdf

[2]: http://www.bloomberg.com/news/articles/2013-06-06/how-the-ro...

Post reply on HN