Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I'm building an open-source, high-frequency trading system

scarcecapital.com

Post reply on HN
Print view View original

191–198 of 198 posts

Re: Show HN: I'm building an open-source, high-frequency trading system

#191

Post by zygomega »

StefanKarpinski wrote: post #187
carterschonwald wrote: post #152

Earlier quoted context omitted.

i've looked at how it compiles to llvm, and it basically punts any optimization to the llvm side, aside from the most basic tracing jit method monomorphization/specialization. (as in, I spent part of an afternoon end of last week skimming the entire code base). This is troubling, because when you don't design your language to make it "manageable" to do static analysis / optimization on, it be becomes DIFFICULT to add…

Thanks for checking Julia out. I fear there is a fair amount of misinformation in your comment, however, so it seems that a few things were lost in your reading of Julia's code (some of which is admittedly quite tricky). I hope you don't mind if I address some of it. The first point is that Julia already has excellent performance on a par with most compiled languages, including, e.g. Haskell, whether they are using L…

Extremely helpful - this really clarifies the differentiation in the Julia approach and I'm excited your team is taking this direction. There's a lot of people cheering Julia on, even if we're wimping ut remaining on the sidelines.

It would help if this explanation was a bit more prominent.

Re: Show HN: I'm building an open-source, high-frequency trading system

#192

Post by zygomega »

ycombobreaker wrote: post #173
fnordfnordfnord wrote: post #131

Earlier quoted context omitted.

>HFT is a scourge on the world's economy I'd point out that a lot of the things that are going on in HFT, are rehashes of old trading scams, and either are or would be illegal if there was any adult supervision. I was tempted to say something about the SEC being left behind the technology, and doesn't understand it. But for that to be true, and have this crap go on for so long, they either need to be complete fools,…

Nanex is wonderful to read, but IMO they frequently misinterpret the data. Off the top of my head: - They only use consolidated feeds for US equities, never direct market-data feeds. The consolidated feed necessarily contains less information that direct feeds (to satisfy more stringent bandwidth requirements), which masks some "interesting" effects of how the exchanges publish their data. - They disregard that the C…

That's why I thought the DNQ iqfeed was a suitable choice - no consolidation at all. Would love to know what the "interesting" effects are (beyond the fixed book CME effect) - will be looking a bit harder at the stream to find them!

Re: Show HN: I'm building an open-source, high-frequency trading system

#193

Post by zygomega »

ycombobreaker wrote: post #172
zygomega wrote: post #18

Earlier quoted context omitted.

Long-term investors trade every day. You wake up and the market has gone down a few percent, you're going to rebalance at least. You decide to buy 30 year treasuries you're not going to hold them for thirty years - you're going to sell 29.5 year bonds and buy 30 year bonds in a few months time. And managers turn over long-term investor portfolios quickly. The average holding period for an SP500 company shareholder is…

> And every time a trade happens you run the risk of getting clipped by the faster guys who see you coming. I think it is natural, but fallacious, to apply line-of-sight properties to trading. If you plan to trade, then there are two ways that another participant can "see you coming": 1. If you don't have direct market access, your order gets routed through a broker. The broker sees your order before it hits the mark…

Yes, very loose terminology. I was thinking of a combination of two events:

1. VWAP based trading of large positions, which creates assymetric momentum effects in volume and price (and which is then somewhat forecastable)

2. Not then recognizing that you are forecastable (as a result of playing the VWAP game). The extra information that someone who looks at intraday price relationships has over someone who doesn't. If you wade into the middle of a market that is short-run forecastable (eg it's trending downwards to a new level and the market maker/HFT guys are battling their battles), and you don't check as to whether it's short-run forecastable, then you're probably the patzy at the poker table.

Re: Show HN: I'm building an open-source, high-frequency trading system

#194

Post by zygomega »

ttty wrote: post #178

Which software you used to build the diagram on the main page?

The graphviz dot language: http://www.graphviz.org/Documentation.php

It has good support in org-mode and a there's a nice haskell package that takes the dot code and turns it into an internal graph representation.

Re: Show HN: I'm building an open-source, high-frequency trading system

#195

Post by skebanga »

fawce wrote: post #75
zygomega wrote: post #46

Earlier quoted context omitted.

Yes, I am aware of that trend and considering a switch. I'm personally more used to R is all, but if I found collaborators I'd jump to where the code base goes. The only other factor is I find R pretty aligned with haskell having a somewhat functional pedigree, so that code translates pretty nicely between a rapid hack at the problem to the more robust approach.

zygomega, I'm one of the zipline maintainers and we'd love to collaborate with you. There are a few people doing HFT research with zipline, and there's a lot of work to do. At quantopian (my day job), we focus on longer hold periods, so there is room in the zipline ecosystem for you to do HFT. The main benefit we've found with python as the algo language is that it allows for stat programming with pandas, but also OO…

I too would be interested in collaborating if you switched to python. In my ideal world it would be C++ backend, python frontend

Re: Show HN: I'm building an open-source, high-frequency trading system

#196

Post by mangrish »

zygomega wrote: post #144
mangrish wrote: post #139

Cool! Very interesting. I think given the languages you have selected you are coming more from a quant background? These languages are great for heuristics and analysis but you would really want all 'static' components such as connectivity built in assembly/C/C++. For 'algo' components I like Java as you can still pull microsecond order latencies when crunching numbers but more importantly it gives you a huge time to…

thanks mangrish, quant. How could you tell? I'm conecting with an aggregator cause the direct market feeds are monopolistic price gougers. It's an easy switch if we make it up that curve. I'm not sure if I even understand what middleware is (I'm a quant), but I think the answer is the disruptor!

From the languages.. I've worked with lots of quants and they all rave about R!

Yeah..the aggregator is where you get done in both cost and latency. The prop shops and hedge funds pay through the nose for that stuff so unless you come packing a little capital, true HFT is an issue.

On the middleware, not really.. so you would have a market data component that will be pushing stuff to various components (real time risk, the pricer, and the trading engine). The disruptor sits on the 'in' queue to those components.. the middleware is what pushes messages between instances running on the same/different machines.

Hope that helps :)

Re: Show HN: I'm building an open-source, high-frequency trading system

#197

Post by fmstephe »

ycombobreaker wrote: post #176
fmstephe wrote: post #29

I'm building an open source matching engine. I would love to pair up the two systems for a stress test. I'll keep tracking your project and ping you again when I have a system up a running if you're interested. https://github.com/fmstephe/matching_engine

I see a comment about ITCH in there... Are you modeling your interface after NASDAQ's ITCH/OUCH interfaces? If so, I think it would be interesting to look at the protocol specifications for all of the other major US equity exchanges. I think they are all freely-downloadable.

I was using ITCH data to try to test the matching algorithm. However, I found that the number of hidden trades made it fairly poor for validation, which was what I was most interested in.

I _was_ using the ITCH format for messages and switched away from it. I may revisit that decision in the future.

I agree that it would be really interesting to look at the messaging protocols of real exchanges, and I have made a quick stab at it. But I am focused on getting something very simple working right now.

If you wanted to play around with ITCH files the executable generated in the itch directory will run any ITCH file through the matching engine sort of like a debugger, allowing you to step forwards and backwards in time and see how the internal state of the matching engine changes.

Re: Show HN: I'm building an open-source, high-frequency trading system

#198

Post by carterschonwald »

StefanKarpinski wrote: post #190
carterschonwald wrote: post #188

Earlier quoted context omitted.

you make interesting points. I'll have to read the links and think about it.

I'm also in New York if you want to chat about it over a beer some time. I doubt I can convert a hardcore Haskell programmer on my "scruffy" unchecked, dynamic point of view, but we might have a good conversation about it. We've occasionally quipped that Julia drags Matlab programmers halfway to Haskell, so maybe there's some common ground.

Sure, that'd be fun!

I think we've had 1-2 interactions where we've not quite gotten along, but I might have just been misinterpreting (or mixing up julia devs)

Post reply on HN