Live data from Hacker News

How I made $500k with machine learning and high frequency trading

jspauld.com

211–220 of 320 posts

Re: How I made $500k with machine learning and high frequency trading

#211
post #48
post #23

This is just glorified gambling. I am not sure what special insight or advantage he had, other than his own model. Every trader has a model. It could have easily been called "how i lost 500k with machine learning". Like gambling, it's easy to manipulate statistics to show that you did well in some period of time. I worked for a large investment bank about 10 years ago, writing trading programs for quant traders who w…

It is only gambling in the sense that any business is gambling: Your customers might stop coming tomorrow because the fad wore off, or a competitor provides a better/cheaper/hipper alternative. (And indeed, living is gambling. It's all just a matter of the risk/reward portfolie). But jspauld has apparently made $2/trade after fees on 250,000 trading, with a very small standard deviation (I would guess less than $2/tr…

[deleted]

Re: How I made $500k with machine learning and high frequency trading

#212
post #43

Earlier quoted context omitted.

This. I love crazy projects and Show HN's until the cows come home, but this one is dangerous that I must repeat the warning to others. I cannot emphasize how important it is to understand that people who trade using price action ( http://en.wikipedia.org/wiki/Price_action_trading ) are just speculating based on where they expect the price to move. It's no different than people who play Texas Hold'em online and specu…

Hey, I didn't actually intend this to be a course. I do not make any money in the market right now so am certainly not qualified to teach a course on it. And of course, if I was making money in the market I wouldn't have posted this at all. So please everyone remember that. These comments have made me realize it's probably for the best if I do not post the source code. Basically you are competing against armies of PH…

I didn't see it in the article and I'm sorry if I missed it... but did you mention what the initial bankroll was?

Re: How I made $500k with machine learning and high frequency trading

#213

Being pedantic, 4000 trades a day isn't HFT. This is stil algo trading, of which HFT is a subset. I consider HFT to be any strategy where speed itself is the what gives the edge. Colocation is usually a prerequisite, though not sufficient. It's a shame HFT gets all the attention, when it's really a tiny portion of trading activity. Algo-trading in general is 70%+ of market activity in the US. Also limiting trades isn…

I don't know the exact definition of HFT but I did run my algorithm from a server collocated with my broker close to the exchange. I modelled lag time in simulation and not having it collocated certainly would have hurt.

Re: How I made $500k with machine learning and high frequency trading

#214
post #23

This is just glorified gambling. I am not sure what special insight or advantage he had, other than his own model. Every trader has a model. It could have easily been called "how i lost 500k with machine learning". Like gambling, it's easy to manipulate statistics to show that you did well in some period of time. I worked for a large investment bank about 10 years ago, writing trading programs for quant traders who w…

I'm a pretty risk averse guy and my typical reaction is to figure out why something won't work. One of my pet excuses is that assuming markets are efficient, someone must have already figured this out/ come up with a better exploit, etc. Most of the time I'm right. What troubles me is encapsulated in the following parable:

A UChicago economist and graduate student are walking across campus. The student says ... hey ... there is a hundred dollar bill on the ground! The economist scoffs and says no there isn't ... if there was one, someone must have picked it up already.

Sometimes I catch myself thinking this way. I have to remind myself that (a) markets aren't perfect, and (b) the real world has huge asymmetries in information, ideas, and perhaps willpower (by this, I mean while 100 people might think of a great idea, not all will attempt to implement it; even then, people will differ in execution).

That said, you're likely right. This trading strategy will likely lose money today :-p

Re: How I made $500k with machine learning and high frequency trading

#215
post #48

Earlier quoted context omitted.

It is only gambling in the sense that any business is gambling: Your customers might stop coming tomorrow because the fad wore off, or a competitor provides a better/cheaper/hipper alternative. (And indeed, living is gambling. It's all just a matter of the risk/reward portfolie). But jspauld has apparently made $2/trade after fees on 250,000 trading, with a very small standard deviation (I would guess less than $2/tr…

No. Every business has a risk element, but what makes this gambling is that there is no good or service being produced. It's a game of trying to outguess the other players, with one trader's gain being another trader's loss (relative to market returns). Because there's a commission on trades, and because you pay taxes on net gains but your minimum tax is zero, high frequency trading by its very nature must a loss for…

> No. Every business has a risk element, but what makes this gambling is that there is no good or service being produced.

I was not aware that this is what defines gambling. And "no service produced" is certainly wrong by accepted economic theory - arbitrageurs provide a price discovery service for everyone; they get rewarded for exposing the inefficient prices, even though it is done through market mechanics rather than a specific customer.

OP appears to be a statistical arbitrageur - which is the same concept, except that it includes a shift in time or space (and incurs risk). You might not be interested in this price discovery service, but other people are paying for it with their wallet. (And it's mostly the market makers who pay for this with reduced profits)

> one trader's gain being another trader's loss (relative to market returns).

That's not true in investing in general - when shares have time to appreciate or depreciate, it is definitely not a zero sum game. Everyone can win, or everyone can lose, or anything in between (it all depends on your time range, and your measure of loss or profit. The "non-zero-sum" element arrives partly from companies using operating profit to buy back their own shares).

> Because there's a commission on trades, and because you pay taxes on net gains but your minimum tax is zero, high frequency trading by its very nature must a loss for most players.

That's only true if all players are hf players. If there is sufficient non-HF activity, then the zero-sum argument does not hold.

(I'm not saying that it's not a good approximation - in most time scales, in most scenarios, it is - but it is not the mathematical truth you imply it is)

Re: How I made $500k with machine learning and high frequency trading

#216

Isn't profit meaningless without knowing initial investment? I read the first few paragraphs and got bored. Why not say upfront what the bankroll was to start? edit: I found what I was looking for in the comments.

Just to save someone else from having to look for it like we both did, his initial investment was

  $10k loan from dad. Built it up to 30k trading manually before my automated program went live.

Re: How I made $500k with machine learning and high frequency trading

#217
post #160
post #23

This is just glorified gambling. I am not sure what special insight or advantage he had, other than his own model. Every trader has a model. It could have easily been called "how i lost 500k with machine learning". Like gambling, it's easy to manipulate statistics to show that you did well in some period of time. I worked for a large investment bank about 10 years ago, writing trading programs for quant traders who w…

This is a very mean and unconstructive comment to someone who made the impressive achievement of building his own automated trading system and actually making money from it. I've started calling out comments like this one, because they cause a bad environment for useful discussion. The only argument in your comment that isn't your own unfounded opinion is that market makers make money from people who execute trades.…

While writing his own trading system is a decent accomplishment, due to things such as an overall rising market in the time period involved and survivorship bias, the original author is likely to be completely mistaken about the reason for his winnings.

Given that he might convince other people to engage in high tech gambling in a less-favorable market than the one he operated in, strong words are called for in this case.

Re: How I made $500k with machine learning and high frequency trading

#218
post #203

Earlier quoted context omitted.

FWIW, I couldn't see your pnl chart. I develop algorithmic strategies for a living, and my first reaction to reading your post was skepticism. I'm skeptical for two reasons. (1) because your methods are so unconventional in an industry where convention rules, and (2) because of the time frame of your success, which happened to be one of the more impressive market recoveries in history. I can't tell you how many peopl…

Thank you. You are right - I should clarify things by saying my program had no directional bias. It was a 50/50 split of longs/shorts. Are there other stats I'm missing?

vol

Re: How I made $500k with machine learning and high frequency trading

#219

Being pedantic, 4000 trades a day isn't HFT. This is stil algo trading, of which HFT is a subset. I consider HFT to be any strategy where speed itself is the what gives the edge. Colocation is usually a prerequisite, though not sufficient. It's a shame HFT gets all the attention, when it's really a tiny portion of trading activity. Algo-trading in general is 70%+ of market activity in the US. Also limiting trades isn…

In the US, HFT is mostly synonymous with "all out tech war, flooding the order queue so your less-equipped peers get lags". (Nanex publishes analysis on these events, which are not occuring several times a month and keep accelerating).

In Europe, HFT is mostly what OP describes, because they have reasonable control (e.g., you have to have one execution per 10 orders or pay a fine; in US exchanges, you can sometime finds 10,000 orders submitted in 3 seconds, hundreds of thousands per hour, with 10-20 executions).

> Also limiting trades isn't really adequate risk management. The tech exists to very accurately model your exposures.

That's basically what AIG did with copulas. Unfortunately, the assumptions in these models tend to break during crisis, when correlations go to one. And AIG went bankrupt.

Limiting trades, done correctly (mathematically AND legally) is the ONLY way to do risk management properly. With more assumptions, you can have "more efficient" risk management in terms of leverage (e.g., you can net S&P and RUSSELL exposure by assuming their correlation structure) - but as AIG has shown, that does not mean you are doing a better job of managing your risk.

Re: How I made $500k with machine learning and high frequency trading

#220
post #203

Earlier quoted context omitted.

Hey, I didn't actually intend this to be a course. I do not make any money in the market right now so am certainly not qualified to teach a course on it. And of course, if I was making money in the market I wouldn't have posted this at all. So please everyone remember that. These comments have made me realize it's probably for the best if I do not post the source code. Basically you are competing against armies of PH…

FWIW, I couldn't see your pnl chart. I develop algorithmic strategies for a living, and my first reaction to reading your post was skepticism. I'm skeptical for two reasons. (1) because your methods are so unconventional in an industry where convention rules, and (2) because of the time frame of your success, which happened to be one of the more impressive market recoveries in history. I can't tell you how many peopl…

> I'm skeptical for two reasons. (1) because your methods are so unconventional in an industry where convention rules, and (2) because of the time frame of your success,

I was also in this business, and there's nothing unconventional about his methods. It would, in fact, closely describe the methods of more than one shop I'm familiar with. (Except they WERE able to overcome the declines). And the 3-6 month indicator lifetime looks eerily familiar.

And these places are anything but "convention rules" - it's "creativity rules, before our competitors get creative enough".

> When someone shows me strategies that worked in 2009 and 2010, I immediately make them prove their strategy was not the equivalent of being long equities.

Assuming the OP is telling the truth, there is no equivalent "long equities" strategy that would make 1500% profit over 6 months (%3000 annualized), with a max drawdown of 20% ($2000 on $10000 - but his max drawdown was probably closer to 5% than to 20%). You are welcome to demonstrate that there is.

Sounds to me like you are doing low frequency strategies; it's a completely different ballgame than HFT. He's done 400,000 trades, half of them long, half of them short. It might have been luck, and he might have been riding something underlying the equities, but this is NOT equivalent to being long equities. He might have found a way to get non-linear leverage (rather than prediction). But that's also worth a lot of money in the right hands.

> buy 10 RUT futures at the beginning of the day, sell 10 at the end, and just scratch 1 lots for the other 998 trades. In a bull market like 09-10, that would have made 400k, and would have nothing to do with Machine Learning or its applications to HFT.

That may be (I wasn't trading in 2009-2010, and don't remember the movements or the required margins), but that would have had much higher volatility (and days with much more than $2000 loss) than the OP had. (Assuming, of course, he is telling the truth)

Post reply on HN