Out of interest, how much capital did you start off with?
How I made $500k with machine learning and high frequency trading
51–60 of 320 posts
Re: How I made $500k with machine learning and high frequency trading
#52Earlier quoted context omitted.
I tried to address this concern at the start of my post. If you have some idea of how I manipulated the statistics I'd be happy to respond. Having said that I can agree that my case is pretty unusual and that everyone should beware of attempting to do something like this. Even for myself I couldn't do it now. (There is a reason I turned my program off.)
Why couldn't you do it now?
Well I could try. But it's not going to be any easier now than it was in 2010. For four months I tried everything I could think of to keep it profitable but in the end nothing worked so I had to shut it off.
Re: How I made $500k with machine learning and high frequency trading
#53I don't have much experience with finance or working experience with machine learning, but I've always wondered how much room there was for a clever amateur to profit in this space, even as it's crowded with much more sophisticated professionals with much more sophisticated algorithms and machines.
I'm thinking back to Garry Kasparov's piece in the NY Book Review a couple years back: http://www.nybooks.com/articles/archives/2010/feb/11/the-che...
He talks about a chess tournament in which it was "anything goes"...competitors could be human, computers, or humans with computers. The expected outcome was that a grandmaster using a Deep Blue-like computer would win, but the winners ended up being a couple of amateurs with three computers:
> The surprise came at the conclusion of the event. The winner was revealed to be not a grandmaster with a state-of-the-art PC but a pair of amateur American chess players using three computers at the same time. Their skill at manipulating and “coaching” their computers to look very deeply into positions effectively counteracted the superior chess understanding of their grandmaster opponents and the greater computational power of other participants. Weak human + machine + better process was superior to a strong computer alone and, more remarkably, superior to a strong human + machine + inferior process.
So in HFT, how much room is there for an amateur to profit over professionals by having a sophisticated process?
Re: How I made $500k with machine learning and high frequency trading
#54Earlier quoted context omitted.
I certainly could open source it. I may just wait a bit on the off chance that somebody wants to purchase it.
that would require some proof that it works today.
Not really.
A lot of people in the business would pay e.g. $5,000 for exclusive rights for something that worked this well in 2009 (with proof that it worked in 2009, e.g. verifiable broker statements), and a smaller amount (say, $5,00) for non exclusive rights.
If he claimed it still works but he wants to sell it, it is a completely different game -- because when these things work, they are cash cows.
Re: How I made $500k with machine learning and high frequency trading
#55The charts show he was trading between Jun 2009 and Oct 2010. How much of his gains could be attributed to the market recovery in general? The Dow went from about 7000 to 11000, the Russell from about 600 to 800.
Well, in the article he said tat he did not care about direction, he would simple buy when his expected price was up, and sell when down. However, there could have easily been a bias in his model that "preferred" and performed better during upward movements. If so, he got lucky.
Longs and shorts were 50/50 and my program showed no preference for up or down days. High volatility and high volume was what it liked.
Re: How I made $500k with machine learning and high frequency trading
#56This is just glorified gambling. I am not sure what special insight or advantage he had, other than his own model. Every trader has a model. It could have easily been called "how i lost 500k with machine learning". Like gambling, it's easy to manipulate statistics to show that you did well in some period of time. I worked for a large investment bank about 10 years ago, writing trading programs for quant traders who w…
Correct. The far more common story is "how I lost $Xk on the stock market". Probably most people with programming/AI skills have tried their luck with the stock picking problem at one time or another. I have a few times, but only in simulation. Even using very elaborate machine learning methods and a lot of training data, making money from automated trades is a difficult problem, and my impression is that it's very much like betting on horses or football games.
Re: How I made $500k with machine learning and high frequency trading
#57The charts show he was trading between Jun 2009 and Oct 2010. How much of his gains could be attributed to the market recovery in general? The Dow went from about 7000 to 11000, the Russell from about 600 to 800.
Well, in the article he said tat he did not care about direction, he would simple buy when his expected price was up, and sell when down. However, there could have easily been a bias in his model that "preferred" and performed better during upward movements. If so, he got lucky.
You don't need a bias to accidentally make money when the market is overall moving up, do you? Picking stocks by throwing darts while blindfolded will, on average, make you money in a market that's moving up.
Re: How I made $500k with machine learning and high frequency trading
#58If anyone has questions for me happy to answer as best I can.
I have one question: Why doesn't every hacker do this to make extra money? Is it within the grasp of anybody who can program to automate trading like this? EDIT: Sounds like it's not really for everybody. You have to own or rent a server with access to direct lines to the exchanges, or else your lag will be such that profiting from HFT is impossible. How much do these cost?
> You have to own or rent a server with access to direct lines to the exchanges, or else your lag will be such that profiting from HFT is impossible.
A profitable predictor is a much, much harder problem.
At a place like Goldman Sachs, a quant with a working predictor gets paid 5 times as much as the IT guy who makes that predictor talk to the market quickly enough.
Because, as your question implies, it is (relatively) easy to do the IT work or hire someone to do it. Not so for the predictors.
Re: How I made $500k with machine learning and high frequency trading
#59Earlier quoted context omitted.
This. I love crazy projects and Show HN's until the cows come home, but this one is dangerous that I must repeat the warning to others. I cannot emphasize how important it is to understand that people who trade using price action ( http://en.wikipedia.org/wiki/Price_action_trading ) are just speculating based on where they expect the price to move. It's no different than people who play Texas Hold'em online and specu…
The Instagram guys found an edge. It won them 730m. Fantastic. More than any edge ever won by me. But the market has changed so much since then, please be careful before you follow this course. You are not wrong, but what you wrote here is applicable to any success story posted on HN. Caveat lector. Always.
I think with the automated trading example, it makes it seem much easier for anyone to dip their cup in the stream.
When you think Facebook/Instagram, you think "Damn, those guys got lucky as hell". When you think automated trading, you think, "Hey, it can't be that hard", and start firing up your IDE and rolling out code to talk to an easily provisioned API.
Sure, it may take months to lose your shirt selling a photo service to Face/Goog/Apple. You can lose everything overnight with automated trading.
My father used to trade commodities for a living in the pit at the CME many moons ago, and when I was growing up I would be his technical side when he was trading out of our suburban Chicago home (setting up FM receiver/satellite dish/etc for real time quote data, staying up late nights with him running through trading scenarios in Tradestation on Win3.1 with data downloaded in bulk from Knight Ridder, and so forth).
Something very important I learned from him was: "The market can stay irrational longer than you can stay solvent." With a startup, you can hit bottom. In the right market, bottom is much further down than you can ever see.
Re: How I made $500k with machine learning and high frequency trading
#60This is just glorified gambling. I am not sure what special insight or advantage he had, other than his own model. Every trader has a model. It could have easily been called "how i lost 500k with machine learning". Like gambling, it's easy to manipulate statistics to show that you did well in some period of time. I worked for a large investment bank about 10 years ago, writing trading programs for quant traders who w…
It could have easily been called "how i lost 500k with machine learning".
If you've really worked in that field than it's very surprising you've never heard about what professional poker players call bankroll management (and they "stole" the concept from professional traders).
The whole point is that you can --either if you gain an edge or get lucky-- win big. Very big. But you're never exposing a large part of your funds in the process.
Maybe OP had an overall "stop loss" at, say, $10K. Had he had five minus $2K days in a row at the start, he'd be out. He wouldn't be broke. He wouldn't be without a car and without a bank account. He just would have lost $10K.
But there's no upper limit as to how much you can win.
All you need is discipline and sound bankroll management.
And, yes, I've won a five-digits figure (hence not anywhere near what OP did) real $$$ at online poker. Starting from $0.01/$0.02 small blind/big blind tables and then working my way up using bankroll management.
It's assymetric. You're foolish if you think that succesful traders who won $x were as likely to lose $x. This is simply not how it works and it's very well explained in OP's article. He's detailing what his maximal daily exposure was and it was tiny compared to what he made.