Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I made $500k with machine learning and high frequency trading

jspauld.com

Post reply on HN
Print view View original

281–290 of 320 posts

Re: How I made $500k with machine learning and high frequency trading

#281

Post by loup-vaillant »

gd1 wrote: post #279
loup-vaillant wrote: post #277

Earlier quoted context omitted.

Okay, thanks. A few minor points. I agree that "why" doesn't matter, if you know "what". When you don't however, "why" is a useful predictor. I understand the value of arbitrage, to some extent. In its current form, it's quite heavy, but I can imagine we're better off allowing it. We should compensate for value, not for risk. The two are not always correlated. I know trading is risky, but that's not the point. If it…

You're still missing it. Every single additional limit order in the market creates real value. If I want to buy 500 shares of company A, and one market participant is offering 250 at $10. Another is offering 250 at $11. It is going to cost me 250 x 10 + 250 x 11 = $5250. Now, the "short term speculator" you are complaining about shows up and puts his tiny little order in, he is offering 1 share at $10. Now the equati…

(Edit: "limit order" looks like technical jargon. If so, I don't know what it means. Maybe you didn't say what I thought you said)

In your example, it looked like you wanted your 500 share for something else than selling them. So you're talking about "real transactions (with non-speculators) at the boundaries". So yes, those transactions are a useful service provided to you by the speculators. I'm not denying that.

On the other hand, your wording seems to imply that most transactions are of this type (non-speculator with speculator). As far as I understand the system, they are not. If the OP is to be trusted, the vast majority of transactions are between 2 speculators trying to outsmart each other. Do we at least agree on that narrow point?

My second point is that a such a transaction (between 2 speculators) is zero sum. Locally, because whatever the first speculators won, the other lost. And globally, because wherever the share is, it could still be sold to a non-speculator. Making the transaction between speculators doesn't make it any easier. I'd say it could make it harder, for the non-speculators now have to buy the share faster than the speculators (or pay extra to have traders do it for them). Seriously, where is the value in that?

Now maybe a good speculator tends to provide a good service to non-speculators. However, they are not selected by the quality of the service they provide. They are selected by their ability to rip each other off. How quality of service arises from such a cut-throat competition is mysterious to me. (Usually, we compete for quality of service directly, so the connection is obvious. Speculating is not the same thing.)

Re: How I made $500k with machine learning and high frequency trading

#282

Post by AnIrishDuck »

lrem wrote: post #275
white_devil wrote: post #268

Earlier quoted context omitted.

It's always the same bullshit excuse: "providing liquidity". It's just that you pretty much need to be another HFT bot to partake in that liquidity.

From what I understood, this contribution is not about making stuff nanoseconds faster, but about how this pushes spreads down. Anyone doing any trading will be happier to see the spreads smaller, wouldn't he? Note: by spreads I mean the difference between buy and sell prices. I don't know if there is a special word for it in this context.

Exactly. HFT reduces counterparty risk for market makers (because with HFT, it's much more likely that there will be a counterparty for any given trade). This enables the market makers to reduce their bid-ask spreads; the profit from the bid-ask spread is what covers the risk a market maker faces from their market clearing obligations.

Re: How I made $500k with machine learning and high frequency trading

#283

Post by beagle3 »

rorrr wrote: post #259
beagle3 wrote: post #223

Earlier quoted context omitted.

He was trading futures. You need $10K in margin per future. He states somewhere that he started with $30K or so, and I estimate his "margin at risk" at any point was $100K or less. And he surely wasn't paying retail prices - you can go down to around $0.50/future if you know what you are doing. (In general, futures have 1:5 to 1:20 leverage)

Ok, at $0.5/future, the fees are $500 - 2,000 per day $2,500 - 10,000 per week $130,000 - $520,000 per year That's just the fees. He must make that much just to break even. If you can make that much starting with $30K, you're a supermotherfucking genius.

You should look more deeply into how these things work. Let's take the DAX Futures for example (which he was trading). All numbers are in EUR.

One "tick" (minimal movement) is worth 12.5 EUR. At volume you pay 0.5 EUR, IIRC, but let's assume you pay 1 EUR in fees, everything included.

If you bought and sold at the same price, you lost 1 EUR/trade. This is the cost of business.

If you bought, and sold after a favorable 1 tick movement, (e.g. bought at 4013.0 and sold at 4013.5), you're 10.5 EUR richer - 12.5 on the difference, minus 1 for each trade (one buy, one sell).

If you bought and sold after an unfavorable one tick movement (e.g. bought at 4013.5 and sold at 4013.0), you're 14.5 EUR poorer - 12.5 on the difference, and 1 for each trade (one buy one sell).

OP averaged $2/trade over 200,000 trades; that means he had 2/3 right calls, and 1/3 wrong calls or so if he only traded dax and only had 1 tick moves.

He was very smart, but you're looking at it wrong - the fees are the cost of doing business, much like salaries are the cost of producing software. In finance, you rarely care about revenue or "notional" (which can easily run into the trillions per year for a small trader - for ~1 eur, you get 75,000 eur in notional value on the dax).

You just roll the fees up-front into your choices when thinking about it, and it all makes much more sense.

(Not trying to take away from OPs very commendable achievement - just trying to give the common perspective on how to view this)

Re: How I made $500k with machine learning and high frequency trading

#284

Post by beagle3 »

kamaal wrote: post #256
beagle3 wrote: post #92

Earlier quoted context omitted.

> You can lose everything overnight with automated trading. I'll take it ad absurdum: You can lose everything in a second by not looking left and right while crossing the road. Or even by looking left and right while crossing the road, when someone else is driving recklessly. It is possible to attempt HFT with not much more risk than stating a new InstaFaceGoogApple service. Put $10,000 in your margin account, and us…

>>You can lose everything in a second by not looking left and right while crossing the road. Or even by looking left and right while crossing the road, when someone else is driving recklessly. The point is you are lured into crossing the road, when you absolutely didn't have to.

How are you "lured" by reading an article about someone who successfully crossed the road, any more than you are "lured" into a singing career by reading about Adele or "lured" into building an instagram clone?

Re: How I made $500k with machine learning and high frequency trading

#285

Post by kolbe »

damncabbage wrote: post #257
shrughes wrote: post #254

Earlier quoted context omitted.

Why are you ignoring HFT's positive contribution?

Could you please elaborate what that contribution is?

How about efficiency? People call the liquidity providing aspects of HFT 'bullshit', but computers have vastly reduced the manpower necessary to manage a market.

Each futures pit used to have hundreds of traders, who required several assistants/support and commanded a huge salary. Many firms needed multiple traders in a pit, just to be able to make sure they could provide liquidity to all possible market participants. Today, a couple strategists with a small team of programmers can cover dozens of futures markets at once.

The same principle holds across bond, FX, equity and options markets alike. HFT has supplanted a terribly inefficient market with a better one. Is it perfect or even good? Probably not, but it's magnitudes better than the traditional method.

Re: How I made $500k with machine learning and high frequency trading

#286

Post by beagle3 »

dmmalam wrote: post #238
beagle3 wrote: post #219

Earlier quoted context omitted.

In the US, HFT is mostly synonymous with "all out tech war, flooding the order queue so your less-equipped peers get lags". (Nanex publishes analysis on these events, which are not occuring several times a month and keep accelerating). In Europe, HFT is mostly what OP describes, because they have reasonable control (e.g., you have to have one execution per 10 orders or pay a fine; in US exchanges, you can sometime fi…

When I say limiting trades, I mean naively saying 'I will have at most x positions outstanding'. Each trade has an associated risk (variance), that interacts in complicated ways in a portfolio, which I'm sure you know. silly example, 100 small positions could be less risky than 1 large position or, 1 long, and 1 short trade will cancel each other out and create a riskless portfolio (with 0 return). You need to have a…

> interacts in complicated ways in a portfolio, which I'm sure you know.

Yes. And assumptions about this are bound to break at the most inopportune moment, see e.g. AIG, which I already referred to. Read about the "copula model" disaster, as your statement indicates you are unaware of its details. https://en.wikipedia.org/wiki/David_X._Li#CDOs_and_Gaussian_...

> 1 long, and 1 short trade will cancel each other out and create a riskless portfolio (with 0 return)

This is true if and only if the long and short are in the same exchange, AND exchange rules allow netting long vs. short deterministically. Otherwise, you have counterparty risk. E.g., you can be long SPY and short SP contract (in equal underlying), which would theoretically mean your only exposure is interest rate changes (and sometimes not even that!)

However, since this is in different exchanges, it might happen that during a flash crash, your SPY position will be liquidated for insufficient margin at a low price, but then the price bounces back, and you've lost money on a perfectly hedged position.

OP's model (limiting exposure and assuming the worst, if I understand correctly) is not statistically efficient use of margin, but it's way better at actually managing risk than any statistical model.

Re: How I made $500k with machine learning and high frequency trading

#287

Post by beagle3 »

raverbashing wrote: post #272
toomuchtodo wrote: post #59

Earlier quoted context omitted.

I think with the automated trading example, it makes it seem much easier for anyone to dip their cup in the stream. When you think Facebook/Instagram, you think "Damn, those guys got lucky as hell". When you think automated trading, you think, "Hey, it can't be that hard", and start firing up your IDE and rolling out code to talk to an easily provisioned API. Sure, it may take months to lose your shirt selling a phot…

"You can lose everything overnight with automated trading." Didn't your father teach you about "Stop orders" You can't have your algorithm cranking away without supervision. And to be extra sure, lots of testing and LIMITS. Limit the amount and value of orders. With stocks, worst case: you lose the face value of stocks in your portfolio Derivatives: you can lose more, even 'infinite liability' (still, it's constraine…

Stop orders don't guarantee execution or any specific limit to the loss; During a flash crash, you'll realize that a "10% stop loss" order CAN become a 50% loss.

Re: How I made $500k with machine learning and high frequency trading

#288

Post by mcguire »

lrem wrote: post #275
white_devil wrote: post #268

Earlier quoted context omitted.

It's always the same bullshit excuse: "providing liquidity". It's just that you pretty much need to be another HFT bot to partake in that liquidity.

From what I understood, this contribution is not about making stuff nanoseconds faster, but about how this pushes spreads down. Anyone doing any trading will be happier to see the spreads smaller, wouldn't he? Note: by spreads I mean the difference between buy and sell prices. I don't know if there is a special word for it in this context.

Do you know of any data on the size of the spreads over time?

Re: How I made $500k with machine learning and high frequency trading

#289

Post by unreal37 »

mattyppants wrote: post #157
unreal37 wrote: post #123

Earlier quoted context omitted.

I think there is a healthy respect for computer code that can let loose upon the world and make money.

Your missing the point that owning part of a company for 10 seconds brings zero social value to the economy. Which I also tend to agree with. Slightly off-topic from the point of the article, which was that he achieved it without backing of a fairly large institution.

I didn't miss that point - I just chose not to address it. Lots of things add zero (or negative) social value to the economy including Farmville or the gazillionth Instagram clone. That doesn't mean they're not worth doing.

And HFT or day-trading does bring some value to the world. It enables companies to go to the public markets and raise capital. And investors to sell their shares without having to wait too long for a buyer. But those positives come along with negatives as well.

Re: How I made $500k with machine learning and high frequency trading

#290

Post by smrtinsert »

martinced wrote: post #60
tarr11 wrote: post #23

This is just glorified gambling. I am not sure what special insight or advantage he had, other than his own model. Every trader has a model. It could have easily been called "how i lost 500k with machine learning". Like gambling, it's easy to manipulate statistics to show that you did well in some period of time. I worked for a large investment bank about 10 years ago, writing trading programs for quant traders who w…

It could have easily been called "how i lost 500k with machine learning". If you've really worked in that field than it's very surprising you've never heard about what professional poker players call bankroll management (and they "stole" the concept from professional traders). The whole point is that you can --either if you gain an edge or get lucky-- win big. Very big. But you're never exposing a large part of your…

Stop-losses are not as effective or nearly as simple as they are described in typical financial media.

http://falkenblog.blogspot.com/2011/02/stop-loss-myth.html (with additional citations in the blog)

Post reply on HN