Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I made $500k with machine learning and high frequency trading

jspauld.com

Post reply on HN
Print view View original

251–260 of 320 posts

Re: How I made $500k with machine learning and high frequency trading

#251

Post by sejje »

jspaulding wrote: post #134
martinced wrote: post #60

Earlier quoted context omitted.

It could have easily been called "how i lost 500k with machine learning". If you've really worked in that field than it's very surprising you've never heard about what professional poker players call bankroll management (and they "stole" the concept from professional traders). The whole point is that you can --either if you gain an edge or get lucky-- win big. Very big. But you're never exposing a large part of your…

I have some friends that have made a lot of money playing poker. The analogy is very good. But, in poker you have to be willing to deal with much bigger ups and downs than what I had to deal with. You can hit a bad streak and it can hurt - even if you are in fact a really skilled player and do in fact have an edge. With my program I didn't really have bad streaks because my P&L was averaged out over thousands of trad…

Not always true. There are plenty of ways to minimize risk.

I played HUSNGs for a living for several years, and I could play three tables at a time (about 9 games per hour) with a 60% winrate. That's incredibly low variance--my graph over the long-term was better than a 45 degree incline.

Re: How I made $500k with machine learning and high frequency trading

#252

Post by sejje »

I don't understand trading enough to even understand many of the terms in the article, however I'm curious to one thing: is it possible that a program could be written specifically to exploit yours? And/or is that a potential reason it became unprofitable?

Re: How I made $500k with machine learning and high frequency trading

#253

Post by jspaulding »

throwaway1979 wrote: post #214
tarr11 wrote: post #23

This is just glorified gambling. I am not sure what special insight or advantage he had, other than his own model. Every trader has a model. It could have easily been called "how i lost 500k with machine learning". Like gambling, it's easy to manipulate statistics to show that you did well in some period of time. I worked for a large investment bank about 10 years ago, writing trading programs for quant traders who w…

I'm a pretty risk averse guy and my typical reaction is to figure out why something won't work. One of my pet excuses is that assuming markets are efficient, someone must have already figured this out/ come up with a better exploit, etc. Most of the time I'm right. What troubles me is encapsulated in the following parable: A UChicago economist and graduate student are walking across campus. The student says ... hey .…

It's easy to fall into that mindset. And in fact, that mindset is right back where I am now. The only reason I had the gall to attempt this in the first place was the the simple fact that I was making money at the time (in 2008) 'manually' day trading the Russell 2000. I thought this 'should not be possible' so I figured there's no reason not to try an automated program.

Re: How I made $500k with machine learning and high frequency trading

#254

Post by shrughes »

femto wrote: post #230
highfreq wrote: post #207

Earlier quoted context omitted.

By luck and skill you found a temporary systematic bias that other players missed. It was even luckier that you found it without a lot of upfront losses. But you could have made many attempts and not found any bias and overall lost money and gave up. If lots of people are losing small sums to find these biases, then it may be that the expected return of trying to find biases is zero or negative. At best HFT is a near…

An example of your last point. I know a very good engineer, who used to design innovative chips for 4G/LTE mobile telephony. These chips contributed to the market position of one of today's leading mobile phone manufacturers. Today, this engineer is designing ASICs for high frequency trading (basically a specialised Ethernet switch, with all extra logic stripped out, so packets go through a few nanoseconds faster). H…

Why are you ignoring HFT's positive contribution?

Re: How I made $500k with machine learning and high frequency trading

#255

Post by oelewapperke »

unreal37 wrote: post #114
randlet wrote: post #99

Earlier quoted context omitted.

Thanks for posting this so I didn't have too. Poker is "a game of skill with an element of luck" and should not be confused with say, gambling on roulette or the outcome of a coin toss.

The same could be said of HFT. Traders are "skilled" at having nanosecond access to the orderbook, having their servers co-located in the same rack space as the exchange itself. They are also "skilled" at recognizing a price movement nanoseconds before it actually happens and getting their order in just in time. But the HFT game changes and you have to keep up. Just as a poker player from 10 years ago would not survi…

Most HFT firms simply do arbitrage. They don't do actual price predictions.

Re: How I made $500k with machine learning and high frequency trading

#256

Post by kamaal »

beagle3 wrote: post #92
toomuchtodo wrote: post #59

Earlier quoted context omitted.

I think with the automated trading example, it makes it seem much easier for anyone to dip their cup in the stream. When you think Facebook/Instagram, you think "Damn, those guys got lucky as hell". When you think automated trading, you think, "Hey, it can't be that hard", and start firing up your IDE and rolling out code to talk to an easily provisioned API. Sure, it may take months to lose your shirt selling a phot…

> You can lose everything overnight with automated trading. I'll take it ad absurdum: You can lose everything in a second by not looking left and right while crossing the road. Or even by looking left and right while crossing the road, when someone else is driving recklessly. It is possible to attempt HFT with not much more risk than stating a new InstaFaceGoogApple service. Put $10,000 in your margin account, and us…

>>You can lose everything in a second by not looking left and right while crossing the road. Or even by looking left and right while crossing the road, when someone else is driving recklessly.

The point is you are lured into crossing the road, when you absolutely didn't have to.

Re: How I made $500k with machine learning and high frequency trading

#257

Post by damncabbage »

shrughes wrote: post #254
femto wrote: post #230

Earlier quoted context omitted.

An example of your last point. I know a very good engineer, who used to design innovative chips for 4G/LTE mobile telephony. These chips contributed to the market position of one of today's leading mobile phone manufacturers. Today, this engineer is designing ASICs for high frequency trading (basically a specialised Ethernet switch, with all extra logic stripped out, so packets go through a few nanoseconds faster). H…

Why are you ignoring HFT's positive contribution?

Could you please elaborate what that contribution is?

Re: How I made $500k with machine learning and high frequency trading

#258

Post by genwin »

jspaulding wrote: post #253
throwaway1979 wrote: post #214

Earlier quoted context omitted.

I'm a pretty risk averse guy and my typical reaction is to figure out why something won't work. One of my pet excuses is that assuming markets are efficient, someone must have already figured this out/ come up with a better exploit, etc. Most of the time I'm right. What troubles me is encapsulated in the following parable: A UChicago economist and graduate student are walking across campus. The student says ... hey .…

It's easy to fall into that mindset. And in fact, that mindset is right back where I am now. The only reason I had the gall to attempt this in the first place was the the simple fact that I was making money at the time (in 2008) 'manually' day trading the Russell 2000. I thought this 'should not be possible' so I figured there's no reason not to try an automated program.

People will tell you that you were just a lucky monkey. But you could have run your algorithm on past data, for hundreds or thousands of fake portfolios, to tell, statistically, what the odds of your algorithm being simply lucky are.

In early 2000s I wrote a machine learning algorithm that beat the S&P 100 with over 1 trillion to 1 odds against it being luck. It predicted a full trading day in advance. But that was all on paper at trading firms' puny costs; unlike you I couldn't beat retail costs. It's amazing that you could do that. For that reason alone I think it's highly likely that you were a skilled monkey.

Also like you, nobody in the industry was interested in my code, even after an industry magazine watched it for 3 months and found it gave "stellar" performance. The few people I was able to discuss it with told me point blank that it was impossible to do it skillfully (efficient market theory), so they assumed it was a hoax or the algorithm was just lucky.

Re: How I made $500k with machine learning and high frequency trading

#259

Post by rorrr »

beagle3 wrote: post #223
rorrr wrote: post #197

Earlier quoted context omitted.

Considering he was making 1000-4000 trades per day, I assume he put more than $1M. Trade fees alone would eat a huge chunk of money. The cheapest trade I can find is $3.95 (optionshouse.com). So the fees will be $3950 - $15,800 per day. $15,800 - $79,000 per week.

He was trading futures. You need $10K in margin per future. He states somewhere that he started with $30K or so, and I estimate his "margin at risk" at any point was $100K or less. And he surely wasn't paying retail prices - you can go down to around $0.50/future if you know what you are doing. (In general, futures have 1:5 to 1:20 leverage)

Ok, at $0.5/future, the fees are

$500 - 2,000 per day

$2,500 - 10,000 per week

$130,000 - $520,000 per year

That's just the fees. He must make that much just to break even. If you can make that much starting with $30K, you're a supermotherfucking genius.

Re: How I made $500k with machine learning and high frequency trading

#260

Post by deathflute »

jspaulding wrote: post #194
HockeyPlayer wrote: post #156

I run a 12 person HFT group in Denver. This is an excellent description of how an individual can bootstrap themselves into success. Great story, and nicely presented. The one interesting point that he glossed over is what his indicators were. He wrote, "The indicators that were most useful were all relatively simple and were based on recent events in the market I was trading as well as the markets of correlated secur…

Thank you. I'm looking back through my code and there are really a lot of indicators. It would get pretty technical to explain them. They are all explainable it's just that each one corresponds to slightly different market conditions and I just didn't want to get into it.

Thanks for an excellent post. I would like to see any one indicator explained in detail as well. I am sure it would be of interest to a lot of people.
Post reply on HN