How I made $500k with machine learning and high frequency trading

A strange position to take, I'm guessing you've absorbed it from the media somehow. His machine was in the market enough to trade 4000 times a day (I would suggest passively, or he would have been eaten by the cost of crossing the spread), so he was basically continuously offering a service to the market - an offer to sell and a bid to buy at the price he thought fair. Do you demand to know what value your local 7-11 provides by selling you milk? The value is that you can buy milk 24 hours a day, so you don't begrudge them the 15c a carton they are making. And they probably charge more than your local supermarket for the convenience too. The dirty thieves.

I refine my position here: http://news.ycombinator.com/item?id=4752742

I do get the value of trading, investing, and some form of arbitrage. Heck, I don't have to search for the farm to get my milk. That's a service, and it does deserve a reward. What the author did is a bit different:

> Most of the market volume was other bots that would only execute a trade with me if they thought they had some statistical edge.

I understood it meant "they would trade with me only if they think I was the sucker". And of course, he would trade with them only if he thinks they were the suckers. It's not providing a service. It's fighting in a zero-sum economy.

Now he did say "most of the market volume". So there's a fraction that's not bots, and probably also a smaller fraction that does not even play the zero sum game, but instead does some positive-sum trading with the mostly-zero-sum players. But this interface boundary seems incredibly thin, compared to the internal zero-sum behemoth. That looks like a highly inefficient use of time, energy, and brains.

And even then, I'm not sure the zero-sum game provides any service to the rest of the world: zero-sum players base their models on the behaviour of each other, not on the actual performance of companies. That would add no new information to the system. At best, that only amplifies the effect of the few that actually predict company performance. And I doubt it does it well.

From what I understood, this contribution is not about making stuff nanoseconds faster, but about how this pushes spreads down. Anyone doing any trading will be happier to see the spreads smaller, wouldn't he?

Note: by spreads I mean the difference between buy and sell prices. I don't know if there is a special word for it in this context.

Okay, thanks. A few minor points.

I agree that "why" doesn't matter, if you know "what". When you don't however, "why" is a useful predictor.

I understand the value of arbitrage, to some extent. In its current form, it's quite heavy, but I can imagine we're better off allowing it.

We should compensate for value, not for risk. The two are not always correlated. I know trading is risky, but that's not the point. If it doesn't create value, it shouldn't be compensated. And I suspect some forms of trading create little to no value at all.

By itself, closing a deal doesn't mean you provided value. It means the other party thinks you provided value to them. When both parties think that, either it's a win-win situation, or someone got tricked. (Delayed bids and offerings complicate this, but it's the same principle.)

If I got it right, the OP did what we could call "short term speculation". If you predict something will rise, buy from a sucker who didn't. If you predict something will fall, sell it to a sucker who didn't. And of course, pray you are not the sucker. Locally, it's totally zero-sum.

Now maybe the whole system facilitates real transactions (with non-speculators) at the boundaries? But even then, for a given volume your reward doesn't seem to be proportional to your facilitation power (which I have no idea how to compute), but to how well you manage to trick your fellow traders. I very much doubt that such twisted incentives can foster a useful, let alone efficient, system.

You're still missing it. Every single additional limit order in the market creates real value. If I want to buy 500 shares of company A, and one market participant is offering 250 at $10. Another is offering 250 at $11. It is going to cost me 250 x 10 + 250 x 11 = $5250. Now, the "short term speculator" you are complaining about shows up and puts his tiny little order in, he is offering 1 share at $10. Now the equation if you want to execute is 251 x 10 + 249 x 11 = $5249. He/she just saved you $1. One real dollar. As someone who wants to get a transaction done on a market, more people posting bids and offers cannot possibly hurt you. Only help you. You have the complete freedom to decide to meet their offered price, to cross the spread, to pick up that carton of milk at the 7-11 and walk to the till and make the transaction.

Now maybe that little speculator who sold you the share at $10, he goes and works the bid, he advertises that he wants to buy a share at $9. Someone fills him. It take 2 minutes of trading to work to the front of the queue (easily possible with an equity). During that time, he is exposed to the risk that the stock might spike up in price, a risk you are no longer exposed to since you got your desired trade done $1 cheaper and 2 minutes ago. Then if he succeeds he makes his $1 profit. It doesn't always work - maybe 60% of the time the price ticks up and he just breaks even. He pays exchange fees and clearing on both the in and out trades too.

In theory, you could get involved in all of this with your 500 share purchase. Try to work the bid, get a better price, etc... or if you cross you are effectively paying for a service. Every order posted in the market is a service.

How wide are bid/offer spreads these days? How can that possibly be a bad thing?