i don't think you know what high frequency in hft means. haskell, r, emacs org-mode (lmao wtf?)... yeah good luck with that.
Show HN: I'm building an open-source, high-frequency trading system
51–60 of 198 posts
Re: Show HN: I'm building an open-source, high-frequency trading system
#52Have you actually used this in a production setting?
No, and I am making no claims. I have a market event feed coming in, a good idea of what the event processing looks like and a rough idea of how to send an order to a broker. I think the project needs to get to production fast though.
Re: Show HN: I'm building an open-source, high-frequency trading system
#53Isn't the barrier to HFT the fact that you need enough capital so that your profits cover the cost of co-located servers and FPGAs in the exchange datacenter, without which you have a latency handicap? (in addition to, of course, coming up with good algorithms)
Yes, if the only edge you have is speed. But a good algorithm may well be the next battleground as the benefits of ultra-low latency reduce.
Re: Show HN: I'm building an open-source, high-frequency trading system
#54I'm building an open source matching engine. I would love to pair up the two systems for a stress test. I'll keep tracking your project and ping you again when I have a system up a running if you're interested. https://github.com/fmstephe/matching_engine
We have some plans on the drawing board for market maker simulators that could do with some fast matching so I'll keep an ear open for that ping.
Re: Show HN: I'm building an open-source, high-frequency trading system
#55Have you considered python for the research/algo side of things? I work in a hedge fund in London at the moment and there's a massive shift away from R towards python in the algorithmic shops that I know about here. There's also the great work that quantopian are doing on their backtesting framework zipline [1] [1] https://github.com/quantopian/zipline
Hi boothead, just curious what you do when you run into the inevitable performance issues that can't be ameliorated with libraries like NumPy? Is Cython or PyPy useful, or do you use Python more for prototyping, and rewrite some portions in C?
I haven't worked in anything as high performance as this (though I've been doing lots of number crunching in Python recently) but there are a couple of great libraries that you can use before you jump into Cython. Check out Numba [1] and numexpr [2]
[1] https://github.com/numba/numba [2] https://code.google.com/p/numexpr/
Re: Show HN: I'm building an open-source, high-frequency trading system
#56Have you considered python for the research/algo side of things? I work in a hedge fund in London at the moment and there's a massive shift away from R towards python in the algorithmic shops that I know about here. There's also the great work that quantopian are doing on their backtesting framework zipline [1] [1] https://github.com/quantopian/zipline
Hi boothead, just curious what you do when you run into the inevitable performance issues that can't be ameliorated with libraries like NumPy? Is Cython or PyPy useful, or do you use Python more for prototyping, and rewrite some portions in C?
Python is definitely preferred on the research rather than the production side generally. Some organisations do use it on the infrastructure side too, but these guys are algorithmic traders, and not HFTs so latency isn't that big a deal.
WRT to the problem of deploying prototyping/research code, there are the following unsolved problems that I'm aware of:
* Going from matrix operations over the whole timeseries (taking care to avoid problems with your algo looking ahead) for speed in a research setting to deploying to an environment that streams updates to the timeseries one at a time. I think that this is an area that haskell has the potential to excel at, given it's strong guarantees on structure.
* Concurrency. The options in python all suck to some degree - especially if you have to interact with C libraries or extensions. I don't think it will ever make sense to build your real time market date in python. Again here haskell has an advantage.
However the python ecosystem seems to be almost perfect for researchers:
* Excellent and flexible data slurping/munging/transforming.
* numpy, scipy, pandas, theano, scikits... 'nuff said
* ipython
* Cross platform
HTH
Re: Show HN: I'm building an open-source, high-frequency trading system
#57Earlier quoted context omitted.
> an open source approach can be seriously competitive Do you really think any Joe Schmoe off the street could just grab your open source HFT and start making money with it? If not, how does your project benefit anyone? Are you working in the industry, but not in HFT? Maybe this is project is just practise for getting a job in HFT? I imagine that's where programmers get the fattest paychecks in the world. Only a frac…
Good points. To be more precise, I think that an open source approach to system research and design can be hyper-competitive versus closed-door secret-squirrel development run by a committee looking for short-term wins. This project benefits me firstly because I get feedback on my initial scratchings and maybe even collaboration on areas outside my comfort zone. And it could well enable me and others to avoid having…
Sorry, you're criticising the mindset of "short-term wins" and you're in HFT? What do you think HFT is? Wake up and smell the coffee.
Re: Show HN: I'm building an open-source, high-frequency trading system
#581. Timestamps in your seed data might benefit from nanoseconds if you really talking about "high" frequency.
2. I agree with you comment that it is easier to think about concurrency in Haskell than in something like C++, however you can't really compete with C/C++ in Haskell. Not even with cgo (Go packages that call C code), not with OCaml or any other higher level beasts that promise the speed. Fortran would be the only one faster for the "algo" part of your initiative. But again, if this is just an exercise, Haskell and others (I prefer Clojure for example :) will do just fine.
3. Would make sense to split the "platform" in two (very different) parts: "Quantitative Analysis" (a collection of tools and rules) and "Technical Glue to Read and Stream". Each can/should be divided further of course, but the two above are essential yet very different for a true "HTF Platform".
Re: Show HN: I'm building an open-source, high-frequency trading system
#59Have you considered python for the research/algo side of things? I work in a hedge fund in London at the moment and there's a massive shift away from R towards python in the algorithmic shops that I know about here. There's also the great work that quantopian are doing on their backtesting framework zipline [1] [1] https://github.com/quantopian/zipline
Yes, I am aware of that trend and considering a switch. I'm personally more used to R is all, but if I found collaborators I'd jump to where the code base goes. The only other factor is I find R pretty aligned with haskell having a somewhat functional pedigree, so that code translates pretty nicely between a rapid hack at the problem to the more robust approach.
Are you planning to run R in production, or is it purely for research that then gets translated to haskell?
Re: Show HN: I'm building an open-source, high-frequency trading system
#60I'm building an open source matching engine. I would love to pair up the two systems for a stress test. I'll keep tracking your project and ping you again when I have a system up a running if you're interested. https://github.com/fmstephe/matching_engine
We have some plans on the drawing board for market maker simulators that could do with some fast matching so I'll keep an ear open for that ping.
What sort of performance baselines would you require?