Ask HN: Any high frequency trading hackers • Hacker News Board

Re: Ask HN: Any high frequency trading hackers

#41

Post by _grrr » Sun, Nov 28, 2010, 9:14 AM UTC

Challenges:

* The need for speed, at every level of the architecture (network, tcp/ip, hardware, app)

* Reducing order (send/ack) round-trip times, this generally means putting your servers in a data cetre as close to the exchange as possible (co-located, if the exchange offers it). If your trading across multiple exchanges simultaneously it gets trickier.

* Sourcing market data - can you source direct from the exchange, rather than through a 3rd party like Reuters? Again, it comes down to how fast you can re-act to the market.

* Back-testing - you need historical data to test a model, then you need a way of testing the model - what are you going to test against? How are you going to simulate the exchange?

* Expense - it's expensive - market data, co-location etc etc all costs, as others suggested. HFT is generally short term positions, with some arbitrage strategies holding positions for less than a few milliseconds. A medium term (intra-day) type strategy requires less intensive (expensive) technology as your not trading to capture market prices that might only be extant for a few milliseconds.

Re: Ask HN: Any high frequency trading hackers

#42

Post by izendejas » Sun, Nov 28, 2010, 10:29 AM UTC

pkghost wrote: post #33

What value does HFT create?

That's a very important question to ask and I myself would like to get a response from some of the readers who're hacking away at it, instead of having you get downvoted.

How does one prevent a catastrophy that may result from "rogue algorithms" that are far worse than Infinium's? And is such risk truly worth it?

Re: Ask HN: Any high frequency trading hackers

#43

Post by retube » Sun, Nov 28, 2010, 10:35 AM UTC

HFT is, as the name suggests, all about speed - sub-millisecond latency in some markets. It requires a lot of physical and expensive resources, not to mention an extremely deep knowledge of the mechanics of whatever contracts you wish to trade and the exchanges they are traded on. Basically there's a reason this field is played only by the big banks and hedge funds. (I bank I know spent at least $10m setting up an HFT desk)

Algo trading is probably a much better option: basically trading off the back of quant/stat analysis you have done with respect to prices (or relative prices). You'll learn lots about whatever contracts/instruments/markets you're interested in, plus get to flex your geeky skills. And you can do it from a laptop at home over a regular intenet connection with some cheap, if not free, trading platform or api.

Re: Ask HN: Any high frequency trading hackers

#44

Post by _grrr » Sun, Nov 28, 2010, 12:03 PM UTC

thinkingeric wrote: post #17

mdwrigh2 wrote: post #14

Earlier quoted context omitted.

Do you have any suggestions on where to start reading on Algorithmic Trading? Perhaps also a way that I could test algorithms against the market, without actually risking any money?

You can buy historical 'tick data'. Be forewarned, however, that simulations run on this kind of data are not the same as real trading since it doesn't reflect the bid/ask. Also in my opinion and experience, price alone is insufficient data for analysis. Also don't forget to figure in total execution cost as it makes a huge difference in the evaluation of algorithms, not to mention the 'bank roll' necessary to allow…

For serious HFT development you need to have order book data too. This costs.

Re: Ask HN: Any high frequency trading hackers

#45

Post by ggruschow » Sun, Nov 28, 2010, 12:45 PM UTC

pkghost wrote: post #33

What value does HFT create?

HFT provides liquidity which reduces everyone else's cost and/or exposure to short term risk.

Low-latency trading though costs everyone. Nobody actually needs anything faster than a fill in a blink of an eye. The only people who think they do really need smarter match engines or to stop taking advantage of people

Re: Ask HN: Any high frequency trading hackers

#46

Post by ig1 » Sun, Nov 28, 2010, 12:46 PM UTC

Dn_Ab wrote: post #25

ig1 wrote: post #11

I'd suggest maybe look at algo trading rather than HF, it's much more accesible to outsiders, plus you can use the algorithms on places like betfair and stand a decent chance of actually making money.

what parts of betfair are you suggesting? I am skeptical but intrigued. while profitable gambling is possible I expect the potential returns are much less for respective successful strategies with advantaged gambling also less amenable to extracting any sort of substansively edge giving pattern. edit: i'm refering to algorithmic trading v gambling not HFT.

Betfair gives you access to the order book and you can get historical data relatively cheaply. Pretty much any liquid market on Betfair will do (in-play sports events tend to be good). Betfair markets to tend to show a lot of the same patterns as "real markets', although obviously there are differences. But it's a decent place to learn the fundamentals.

You're right that potential returns are much higher on real markets, purely because there's far more money traded on real markets. On the other hand entry to "real markets" is much more expensive, Betfair is fairly cheap to enter and don't charge a per-trade commission. Plus there are more market inefficiency on Betfair that don't exist in more developed markets.

Re: Ask HN: Any high frequency trading hackers

#47

Post by regularfry » Sun, Nov 28, 2010, 1:38 PM UTC

Dn_Ab wrote: post #25

ig1 wrote: post #11

I'd suggest maybe look at algo trading rather than HF, it's much more accesible to outsiders, plus you can use the algorithms on places like betfair and stand a decent chance of actually making money.

what parts of betfair are you suggesting? I am skeptical but intrigued. while profitable gambling is possible I expect the potential returns are much less for respective successful strategies with advantaged gambling also less amenable to extracting any sort of substansively edge giving pattern. edit: i'm refering to algorithmic trading v gambling not HFT.

Betfair isn't just for gambling. You can trade betting positions. In that sense, it's a marketplace amenable to algorithmic trading like any other, entirely without relying on having a gambling advantage.

Re: Ask HN: Any high frequency trading hackers

#48

Post by Dn_Ab » Sun, Nov 28, 2010, 1:48 PM UTC

ig1 wrote: post #46

Dn_Ab wrote: post #25

Earlier quoted context omitted.

what parts of betfair are you suggesting? I am skeptical but intrigued. while profitable gambling is possible I expect the potential returns are much less for respective successful strategies with advantaged gambling also less amenable to extracting any sort of substansively edge giving pattern. edit: i'm refering to algorithmic trading v gambling not HFT.

Betfair gives you access to the order book and you can get historical data relatively cheaply. Pretty much any liquid market on Betfair will do (in-play sports events tend to be good). Betfair markets to tend to show a lot of the same patterns as "real markets', although obviously there are differences. But it's a decent place to learn the fundamentals. You're right that potential returns are much higher on real mark…

If it wouldn't trouble you to do so, could you go into more detail please? Do you have experience with this?

You are saying that any sport event, in particular those that are in play actually have some statistical mineable property with profitably exploitable strategies? If this is the case, with such a low barrier should this not be a game with zero expectation in the long run (maybe negative if there are minimum bet rules)? Whatever distribution that can be learned would have much higher variance than your typical intra day trades no? And while these techniques may be similar in spirit, do the fundamentals map sufficiently? As they say, the devil is in the details. but I speak from ignorance.

Re: Ask HN: Any high frequency trading hackers

#49

Post by a904guy » Sun, Nov 28, 2010, 2:42 PM UTC

Dn_Ab wrote: post #36

a904guy wrote: post #29

Earlier quoted context omitted.

Absolutely So the system is built around the ~50 different indicators and oscillators that I've found the formulas for generating. These "methods" are variable driven, meaning each method could have 1 to 4 variables ranging between 0.0001 to 260. The current testing platform that it self manages has the end goal to find the highest gain after commissions using a combination of any of these methods for opens and close…

Awesome. It sounds like you are running some kind of stochastic optimization algorithm on the data points and also doing so across a cluster! Are you eventually gonna leverage GPUs on the machines as well? Also, do you have a blog or something? >I tried to keep it very basic explanation, there is a lot of other things that go on, to make this all work flawlessly. nope, was more than enough. Don't want you to give awa…

I have been tinkering with pyCUDA and openCL. I will have to convert all my algorithms to kernel code for CUDA but it shows VERY promising results. I can see the application being run across GPUs to keep costs low. I have a handful of nice nVidia cards with around ~200 cores each.

I tried to write up a little about Edward and his backend process a while back on my personal website, but I find that it changes too much currently to keep anything on paper. I code more than I can write about the code ;]

The system is being ran against other virtual systems now. Everything has been trending almost exactly along my estimates so I'm very excite to get it started. The main testing grounds is with my Optionshouse.com, which their interface is nice. All interactions are XML requests, so I've been able to easily write an API that is VERY solid, to do everything I need.

I'm moving the system this week so that I can get it into more high availability setup. After that I would begin working it against my personal IRA account to see how it goes on the long side at least.

Re: Ask HN: Any high frequency trading hackers

#50

Post by a904guy » Sun, Nov 28, 2010, 2:43 PM UTC

logicwins wrote: post #35

a904guy wrote: post #29

Earlier quoted context omitted.

Absolutely So the system is built around the ~50 different indicators and oscillators that I've found the formulas for generating. These "methods" are variable driven, meaning each method could have 1 to 4 variables ranging between 0.0001 to 260. The current testing platform that it self manages has the end goal to find the highest gain after commissions using a combination of any of these methods for opens and close…

Also interested. I have wanted to try GAs with this same general approach for awhile but I have a trading background not a coding one. logicwins atthe gmail dot com.

I'll shoot you an email today.