Great job. From 2007-12-01 to 2008-12-31 you only managed to lose 215.81%. The benchmark only lost 37.04%.
Show HN: How I Used Machine Learning to Optimize My Trading Algorithm
31–40 of 47 posts
Re: Show HN: How I Used Machine Learning to Optimize My Trading Algorithm
#32The site is pretty slick, but the backtest I ran when I cloned the algo is pretty slow. It's been running for about 10+ mins now and it's only 40% done. As well, in the logs, when I see stuff like: 2012-05-31handle_data:35INFO -63.520880 shares of Security(6109) sold. it doesn't really inspire a lot of confidence. What does it mean that -63.520880 shares were sold? Does that mean they were bought? And the fact that y…
(disclaimer: I work for Quantopian)
I can't speak for the author of the algo, but from the algo, it looks like the relevant lines for your question are 34 (order(stock,indicator * context.bet_amount)) and 35 (log.info("%f shares of %s sold." %(context.bet_amount * indicator,stock)).
Our backtester (Zipline) will only order whole number of shares, obviously. If you pass it a fractional number, we take the floor: https://github.com/quantopian/zipline/blob/master/zipline/ge...
The log line you're seeing should probably flip the sign of the number of shares before logging. order(-63, sid(6109)) means sell 63 shares of security 6109. The log line is simply logging the negative value instead of the positive one. Users can log anything they want in their backtest.
As for the slow performance, apologies - being on HN has resulted in a lot of people running this algo and while we're scaling up new servers, it's taking a bit of time to distribute load.
thanks for using Quantopian!
[Edit - added source link to Zipline's order method]
Re: Show HN: How I Used Machine Learning to Optimize My Trading Algorithm
#33What data did you use to train this? Because it looks like it might just be overfitting the training data.
How can one tell the difference?
Re: Show HN: How I Used Machine Learning to Optimize My Trading Algorithm
#34What data did you use to train this? Because it looks like it might just be overfitting the training data.
How can one tell the difference?
Typically, you would train the algorithm on one set of data and then test it on another. So you might train it on data from FY 2010 and then test it out in a simulation of FY 2011.
The fear is that if you train it on FY 2010 and then it does well in a simulation of FY 2010, it might only be because it has stored some representation of a record of FY 2010 which is extremely predictive of FY 2010 but doesn't generalize well to any other year. Testing the algorithm against a simulation of FY 2011 would reveal this flaw.
Re: Show HN: How I Used Machine Learning to Optimize My Trading Algorithm
#35I actually checked out your algorithm today during work (I work at a big bank, aka dead end for a "technologist"). It's an intriguing concept, but I still feel as though classifiers are a poor technique for P&L optimization for a given portfolio. Out of curiosity, what type of data set are you using for backtesting, and what time frame? It's not entirely clear that asset (i.e. stock) prices follow Brownian motion/Weiner process, rather they may be discontinuous.
Re: Show HN: How I Used Machine Learning to Optimize My Trading Algorithm
#36If someone really came up with some fool proof method of beating the market, wouldn't they keep it secret? Meanwhile, most folks should stick to asset class allocation and indexed funds and ETFs.
That's the whole point of Quantopian; to open-source a closed-source and highly secretive profession. See: http://blog.quantopian.com/open-source-and-trading-algorithm...
Re: Show HN: How I Used Machine Learning to Optimize My Trading Algorithm
#37If someone really came up with some fool proof method of beating the market, wouldn't they keep it secret? Meanwhile, most folks should stick to asset class allocation and indexed funds and ETFs.
Traders try to keep their successful strategies a secret, but most strategies "work until they don't" meaning that the algos making money today are not necessarily the same ones that were making money in 2012. Furthermore, as with technology companies in general, traders and developers move around between trading firms, and the ideas move with them.
Re: Show HN: How I Used Machine Learning to Optimize My Trading Algorithm
#38"Past performance does not guarantee future results" is still the operative principle here. Data-mining discovers patterns, but it doesn't lead to deep insight into causes, and markets are perturbed by many events that you don't put into your training algorithm. "The market can remain irrational longer than you can remain solvent" is still important investment advice.
The Keynes quote reminds me of LTCM (just finished reading When Genius Failed).
Re: Show HN: How I Used Machine Learning to Optimize My Trading Algorithm
#39Great job. From 2007-12-01 to 2008-12-31 you only managed to lose 215.81%. The benchmark only lost 37.04%.
Reverse the sell/buy and you could get something.
Re: Show HN: How I Used Machine Learning to Optimize My Trading Algorithm
#40Good try. Some observations:
- The sharp ratio is very bad. Focus on improving. Instead of looking for spectacular gains, focus on solid growth.
- Looking at the daily tick backtest, at many points, the alpha is so negative that the losses your algo occurs would make you delinquent. Again, focusing on better sharp ratio should help here. (RETURNS -72.64%)
- Practical consideration: You look at every stock in data for each tick and its historical prices. This could work perfectly well at a low frequency trades e.g. daily, but not at per-second tick because computing time > transaction time i.e. you would be acting on stale inference.
- Transaction costs?
PS: Pet peeve. Gradient descent is a heuristic at best and not true machine learning :)