Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I'm building an open-source, high-frequency trading system

scarcecapital.com

Post reply on HN
Print view View original

171–180 of 198 posts

Re: Show HN: I'm building an open-source, high-frequency trading system

#171

Post by ycombobreaker »

acjohnson55 wrote: post #64
zygomega wrote: post #43

Earlier quoted context omitted.

Good points. To be more precise, I think that an open source approach to system research and design can be hyper-competitive versus closed-door secret-squirrel development run by a committee looking for short-term wins. This project benefits me firstly because I get feedback on my initial scratchings and maybe even collaboration on areas outside my comfort zone. And it could well enable me and others to avoid having…

Others are two lazy? It's my understanding that the average Joe has no access to the hardware and low latency network connections to be able to do this on his own.

> It's my understanding that the average Joe has no access to the hardware and low latency network connections to be able to do this on his own.

Yes, but for US equities this is purely economical: _anyone_ can buy colocated rackspace, direct market data feeds, and direct connectivity. The average Joe does not want to do this any more than he wants to build his own tennis-shoe factory.

Re: Show HN: I'm building an open-source, high-frequency trading system

#172

Post by ycombobreaker »

zygomega wrote: post #18
cmdkeen wrote: post #3

Earlier quoted context omitted.

HFT isn't necessarily a scourge - if anything it discourages people from short term investing, which is a Good Thing. Long term investing isn't affected by HFT, except in providing more liquidity at the point you want to exit any position. If you're planning on holding for 5+ years you're competing against relatively few people and can do good research. Any shorter timeframe and you're competing against millions, aga…

Long-term investors trade every day. You wake up and the market has gone down a few percent, you're going to rebalance at least. You decide to buy 30 year treasuries you're not going to hold them for thirty years - you're going to sell 29.5 year bonds and buy 30 year bonds in a few months time. And managers turn over long-term investor portfolios quickly. The average holding period for an SP500 company shareholder is…

> And every time a trade happens you run the risk of getting clipped by the faster guys who see you coming.

I think it is natural, but fallacious, to apply line-of-sight properties to trading. If you plan to trade, then there are two ways that another participant can "see you coming": 1. If you don't have direct market access, your order gets routed through a broker. The broker sees your order before it hits the market, and if he jumps in line ahead of you then that is front-running, and a Bad Thing. Your broker can get in a lot of trouble for this sort of thing. 2. If you are trying to move a large position by sending multiple orders to the market (one after another), then all market participants have the chance to react to the first order. It's really a game to try to move a lot of inventory at once, without tipping your hand to anyone else in the room. Thems the breaks. Nobody else gets to see anyone's order before the matching engine has already processed it, so there's no way to jump "ahead" of it.

OTOH, maybe your long-term investor is trying to time the market: wait for a signal intraday, and pick that moment to send an order. In that case, if it is a good intraday signal then it is likely that someone else will compete. It is unlikely for a long-term trader to have spent as much on infrastructure as a HF trader, so the juicy signals will result in missed executions that _look_ like front-running.

Re: Show HN: I'm building an open-source, high-frequency trading system

#173

Post by ycombobreaker »

fnordfnordfnord wrote: post #131
white_devil wrote: post #2

Are you trying to get hired by the financial industry, or is there some other reason for doing this? As you may well be aware, HFT is a scourge on the world's economy, and it's a game only the biggest and best-connected players benefit from.

>HFT is a scourge on the world's economy I'd point out that a lot of the things that are going on in HFT, are rehashes of old trading scams, and either are or would be illegal if there was any adult supervision. I was tempted to say something about the SEC being left behind the technology, and doesn't understand it. But for that to be true, and have this crap go on for so long, they either need to be complete fools,…

Nanex is wonderful to read, but IMO they frequently misinterpret the data. Off the top of my head:

- They only use consolidated feeds for US equities, never direct market-data feeds. The consolidated feed necessarily contains less information that direct feeds (to satisfy more stringent bandwidth requirements), which masks some "interesting" effects of how the exchanges publish their data.

- They disregard that the CME feed publishes a fixed depth-of-book, and whenever they look at total liquidity in the book it can appear to flicker when deep levels fall "out" of the back of the book, even if liquidity is actually improving with the presence of a new inside level.

- They make a big deal about wholesaler matching only occurring when the consolidated book is not locked. Their rationale is that subpenny prices are always wholesalers, and (erronously) therefore a lack of subpenny-priced trades must mean a lack of wholesale matching.

These mistakes sound believable, but they do not hold up to any of scrutiny. Use their site to find interesting events, but be very careful about taking their conclusions at face value.

Edit: bullet-list formatting

Re: Show HN: I'm building an open-source, high-frequency trading system

#174

Post by dyno12345 »

zygomega wrote: post #9
white_devil wrote: post #2

Are you trying to get hired by the financial industry, or is there some other reason for doing this? As you may well be aware, HFT is a scourge on the world's economy, and it's a game only the biggest and best-connected players benefit from.

No, I'm already in the finance industry. There are two main reasons I'm doing this: - I think the finance industry is very closed when it comes to intellectual property development, and an open source approach can be seriously competitive. An open approach may well be the future when it comes to being 'connected' - HFT is an interesting multi-disciplinary problem and the shear breadth of expertise required - modern c…

If you're already in the finance industry, how do you have enough free time to work on this?

Re: Show HN: I'm building an open-source, high-frequency trading system

#175

Post by ycombobreaker »

kasey_junk wrote: post #147
fnordfnordfnord wrote: post #145

Earlier quoted context omitted.

Latency arbing is a scourge, and makes limit orders practically useless. Quote stuffing, would be illegal if trading were still done on little slips of paper. Imagine dumping 10,000 slips of paper on the trading desk, and then shouting "just kidding!" In the words of Lawrence from Office Space, "You'd get your ass kicked" Or even worse, 10,000 empty bids. That's why people hate HFT.

I'm not trying to be argumentative, but I am curious why you find latency arb so problematic, and what you think it's impact is on limit orders? I for one think latency arb is one of the bigger net wins for hft. As a market participant, each venue I have to maintain a presence at is a cost to me. I'm willing to pay the latency arb shops their cut to provide me price consistency because for my models it is much cheape…

> I'm not trying to be argumentative, but I am curious why you find latency arb so problematic

That is a loaded term. "Latency arb" as you described it is HFT keeping all protected exchanges synchronized, and it is a good thing. It means that everyone else can ignore the 13 exchanges, and send their orders to the market with the most competitive pricing for connectivity. "Latency" arb" as described in most literature critical of HFT is the specific practice of submitting and canceling non-bona-fide quotes to an exchange with the intent of slowing down the matching engine. If you can slow down the matching engine that most other participants are using, you can effectively delay the public response your actions on the other 12 exchanges. A trading strategy that operates on the basis of a DoS attack on one exchange is definitely problematic.

Re: Show HN: I'm building an open-source, high-frequency trading system

#176

Post by ycombobreaker »

fmstephe wrote: post #29

I'm building an open source matching engine. I would love to pair up the two systems for a stress test. I'll keep tracking your project and ping you again when I have a system up a running if you're interested. https://github.com/fmstephe/matching_engine

I see a comment about ITCH in there... Are you modeling your interface after NASDAQ's ITCH/OUCH interfaces? If so, I think it would be interesting to look at the protocol specifications for all of the other major US equity exchanges. I think they are all freely-downloadable.

Re: Show HN: I'm building an open-source, high-frequency trading system

#177

Post by ycombobreaker »

relaxitup wrote: post #150

On a possibly related note, Josh Levine, creator of the Island trading engine, released the Foxpro source a few years ago.. Interesting stuff.. http://josh.com/notes/island-ecn-10th-birthday/default.htm http://josh.com/notes/island-ecn-10th-birthday/ISLAND.PRG.TX...

Jerk boy!

Re: Show HN: I'm building an open-source, high-frequency trading system

#179

Post by nazka »

jackpirate wrote: post #167
carterschonwald wrote: post #159

Earlier quoted context omitted.

It's not out yet. If you really want to hear about things as they happen, signup for the announce list linked to from www.wellposed.com , I've yet to fire off any emails to that list, but I anticipate 2-3 emails over the next 1-2 months (after a year of hard work and focused thinking). What's also kinda awesome is I think the alpha release with all the Lin alg functionality should be under 2k loc. a lot of the work h…

Hey, I'm the author of the [HLearn library]( http://hackage.haskell.org/package/HLearn-algebra ). I just got two papers accepted into TIFP and ICML about the algebraic nature of machine learning, and how we can make machine learning algorithms both fast and user friendly in Haskell. I plan for a major update of my library in about a month to make it up-to-date with these research contributions, and I'd love to chat a…

Cool! Thx guys. Did you look at this library: Cloud Haskell/distributed-process? [1]

[1] https://github.com/haskell-distributed/distributed-process

Re: Show HN: I'm building an open-source, high-frequency trading system

#180

Post by kasey_junk »

ycombobreaker wrote: post #175
kasey_junk wrote: post #147

Earlier quoted context omitted.

I'm not trying to be argumentative, but I am curious why you find latency arb so problematic, and what you think it's impact is on limit orders? I for one think latency arb is one of the bigger net wins for hft. As a market participant, each venue I have to maintain a presence at is a cost to me. I'm willing to pay the latency arb shops their cut to provide me price consistency because for my models it is much cheape…

> I'm not trying to be argumentative, but I am curious why you find latency arb so problematic That is a loaded term. "Latency arb" as you described it is HFT keeping all protected exchanges synchronized, and it is a good thing. It means that everyone else can ignore the 13 exchanges, and send their orders to the market with the most competitive pricing for connectivity. "Latency" arb" as described in most literature…

What you are describing is quote stuffing. Nearly every participant in HFT/algo trading agrees that it is a problematic practice. Nearly every venue has either enacted or is enacting policies and procedures to either prevent it or severely penalize it, because that is what all of their customers want.
Post reply on HN