Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I made $500k with machine learning and high frequency trading

jspauld.com

Post reply on HN
Print view View original

161–170 of 320 posts

Re: How I made $500k with machine learning and high frequency trading

#161

Post by mcarvin »

Very important to understand that making $500k speculatively is not evidence of an 'edge', nor is trading frequency evidence of the absence of luck. From March 2009 through much of 2010, the market was strongly bullish - if his algorithm showed a positive market bias then his returns would primarily be a function of timing (read luck: and there are a million variants on the nature of the bias that could be unwittingly responsible for his returns, despite the frequency of trades).

We cannot even tell if $500k is a good risk adjusted return - we have no information on volatility, nature of the exposure or most importantly how much money he started with?

Not exactly shocked Jim Simons didn't return his email. But completely shocking that he walked away from a successful automated trading strategy... the only thing rarer than a free lunch is a man willing to walk away from one. suspect.

Re: How I made $500k with machine learning and high frequency trading

#162

Post by pbharrin »

rhplus wrote: post #37

The charts show he was trading between Jun 2009 and Oct 2010. How much of his gains could be attributed to the market recovery in general? The Dow went from about 7000 to 11000, the Russell from about 600 to 800.

Great point, here is a chart of the Russel 2000, incredible growth during that period. http://finance.yahoo.com/q/bc?s=%5ERUT&t=5y&l=on&#38...

Re: How I made $500k with machine learning and high frequency trading

#163

Post by jspaulding »

tarr11 wrote: post #137
jspaulding wrote: post #39

Earlier quoted context omitted.

I tried to address this concern at the start of my post. If you have some idea of how I manipulated the statistics I'd be happy to respond. Having said that I can agree that my case is pretty unusual and that everyone should beware of attempting to do something like this. Even for myself I couldn't do it now. (There is a reason I turned my program off.)

Ie., if you made 500k in one period, but then lost 50k a year for the next 10 years, you've made a net profit of 0. All I know is that you had one good run, similar to how some mutual funds have a good run for a while. How much did you spend before you "tuned" it? How much did you spend afterwards? What were the tax consequences of your trades? Did you make exactly 500k? Have you traded at all since then? You mention…

In my final month I lost $600. I didn't include this on the chart to keep things simpler visually. Since then I have not traded and the reason is that it was abundantly clear that my program was no longer working. That's why I shut it off.

With regard to tuning I may have lost $1000 or so but as I wrote in the article I was able to build a backtesting model that accurately simulated live trading. So once I had that I could basically use it to verify I had sufficient edge to make a profit after covering my commissions.

My risk exposure was very low. When I said large losing positions this meant like $600. But the bottom line is I had a daily stop loss of $3000 enforced at my broker. The most I ever lost was around $2000.

Anyway, there is not really some hidden thing that I am not telling people. It does bug me a bit that your comment is at the top given that it says I'm manipulating statistics and was actually one of the guys that the quants gleefully picked off. I think it's unlikely I traded much with other HFT systems but if I did they certainly lost money.

Re: How I made $500k with machine learning and high frequency trading

#164

Post by juiceandjuice »

jcromartie wrote: post #6
jspaulding wrote: post #2

If anyone has questions for me happy to answer as best I can.

I have one question: Why doesn't every hacker do this to make extra money? Is it within the grasp of anybody who can program to automate trading like this? EDIT: Sounds like it's not really for everybody. You have to own or rent a server with access to direct lines to the exchanges, or else your lag will be such that profiting from HFT is impossible. How much do these cost?

To really be able to do this with a lot of success, you need to be able to do intraday trades, which would make you a day trader under SEC rules, which also requires $25,000 in your brokerage account.

Re: How I made $500k with machine learning and high frequency trading

#165

Post by fr0sty »

mcarvin wrote: post #161

Very important to understand that making $500k speculatively is not evidence of an 'edge', nor is trading frequency evidence of the absence of luck. From March 2009 through much of 2010, the market was strongly bullish - if his algorithm showed a positive market bias then his returns would primarily be a function of timing (read luck: and there are a million variants on the nature of the bias that could be unwittingl…

1. Author graphed his daily returns which should give you a handle on his volatility. He also stated his max drawdown was $2k.

2. Author has stated elsewhere than he began with $10k in seed capital.

3. The "bullish" market you cite increased only about %70 in that timeframe while the author's returns were multiples of that number.

4. Author walked away from a _previously successful_ strategy that no longer produced profits. His montlhy returns went to nearly zero so he stopped.

Edited to fix numbers.

Re: How I made $500k with machine learning and high frequency trading

#166

Post by crucialfelix »

nesu wrote: post #152

Wait. This is not HFT. There's a huge difference between automated and high frequency trading. What he does is only automated scalping at best (or at the fastest). Automated trading is more on strategy, while HFT has more to do with volume and speed. With automated trading, you predict price movements. HFT involves being a liquidity provider. You don't use market technical indicators in HFT, you wait for some really…

quite true. but HFT has become a buzzword like "the cloud" and even many financial industry specialists claim to do HFT but in fact they are just market making. auto-scalping or even hedge funds that day trade with robots are not HFT.

this: http://books.google.de/books?id=dobO95EBcqsC&redir_esc=y

even though its over 10 years old, is real HFT. moving correlations, windowed FFT (of bid and transaction events), microscopic operators, negative first-order autocorrelation of returns.

it is NOT about supplying liquidity or being a market maker. that's just market makers trying to say they are in on the latest trend.

Re: How I made $500k with machine learning and high frequency trading

#167

Post by dschiptsov »

oijaf888 wrote: post #142
dschiptsov wrote: post #101

Earlier quoted context omitted.

He used his own money to test his algos? Come on. Taking a Machine Learning and Statistics course does not make you a trader. You need something else - access to the system. This is why my guess is that he was an employee.

An employee of what? He states above that he was paying roughly $200/month for a server and $1800/month for the software/data connections to his broker.

Prior to setting up my automated trading program I’d had 2 years experience as a “manual” day trader. - mom's and pap's 401k?

Re: How I made $500k with machine learning and high frequency trading

#168

Post by crntaylor »

DanBC wrote: post #66
marshallp wrote: post #31

Earlier quoted context omitted.

I don't get that. It would be true if he just made a few trades, but the author claimed to be making 2000 trades a day. Over a period of months winning that wouldn't qualify as blind luck.

>Over a period of months winning that wouldn't qualify as blind luck. Why not?

It's simple statistics. The author was up 4k/day over 120 days. He doesn't say what his daily volatility was, but let's assume 2k (which squares pretty well with his claim that his worst day was a 2k loss).

With a quick bit of R code, we can simulate his PnL over 120 days multiple times, assuming he has no skill, and see what the probability of him being up 4k/day is. I'll use a t-distribution with 3 degrees of freedom, which allows big up and down swings (again, accentuating the effect of luck).

    > pnl  for (i in 1:1000) pnl[i]  mean(pnl > 4000)
    0.0
That is, there's a zero percent chance that he would have made those returns if he had no skill. And remember that this simulation is overestimating the effect of luck.

Re: How I made $500k with machine learning and high frequency trading

#169

Post by OldSchool »

mattyppants wrote: post #157
unreal37 wrote: post #123

Earlier quoted context omitted.

I think there is a healthy respect for computer code that can let loose upon the world and make money.

Your missing the point that owning part of a company for 10 seconds brings zero social value to the economy. Which I also tend to agree with. Slightly off-topic from the point of the article, which was that he achieved it without backing of a fairly large institution.

True it adds no social value, but it is arguably the ultimate hacker's game :) You can actually add a little social value by placing only orders that increase liquidity; those where someone else "takes out" your standing offer or bid. That and the fact that your profits are taxed as ordinary income in the US for equities at least.

Re: How I made $500k with machine learning and high frequency trading

#170

Post by dschiptsov »

mattyppants wrote: post #155
dschiptsov wrote: post #101

Earlier quoted context omitted.

He used his own money to test his algos? Come on. Taking a Machine Learning and Statistics course does not make you a trader. You need something else - access to the system. This is why my guess is that he was an employee.

No, he said he backtested them against data he collected from the feeds. He paid for the feeds just like everyone else does with his own money.

For me it all looks like a bad-written fiction, and the source is just a blog post.
Post reply on HN