Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I'm building an open-source, high-frequency trading system

scarcecapital.com

Post reply on HN
Print view View original

151–160 of 198 posts

Re: Show HN: I'm building an open-source, high-frequency trading system

#151

Post by carterschonwald »

carterschonwald wrote: post #148
spitfire wrote: post #134

Earlier quoted context omitted.

From my talks with Carter previously he's working on a haskell based platform for analytic tooling in Haskell. Basically the core primitives for making large scale data analysis apps in Haskell. Or that's what he was up to in august. Hopefully he can chime in here and update everyone on his progress. I honestly hope he succeeds in his plans.

Still working on it! Been taking a bit longer to get the core worked out that I'd have liked, but life happens (eg my mom had cancer for a month this winter, though she's fine now, which is awesome. She didn't even need chemo or rad!!). Also, I was original planning to NOT write my own linear algebra substrate, but I quickly realized all the current tools suck, and that I needed to come up with a better numerical sub…

I really really really hope the biz+tech combo validates... because then I could occasionally stop and think "holy fuck, I'm bootstrapping my fantasy job / company, the likes of which I imagined / dreamed of as way back as middle school and high school!"

Realistically theres 3 different outcomes:

the tech doesnt validate (and thus the biz doesnt either) --- then i'm looking for a day job ... (and I'm pretty darn egomaniacal and loud, finding a good fitting dayjob would take a bit of work!)

the tech works yet the business doesnt --- Not sure how that would happen esp since no investors means enough income to support myself would still be a successful business, though I guess i'd have some compelling portfolio work if I went job hunting

the tech and biz both validate, and earning enough to move out of my parents --- magic pony fantasy land of awesome. what more could anyone want? MORE AWESOME PROBLEM DOMAINS THAT NEED BETTER TOOLS (i mean, that would really be sort of the ideal, but remains to be seen if that can happen.)

Re: Show HN: I'm building an open-source, high-frequency trading system

#152

Post by carterschonwald »

pygy_ wrote: post #87
zygomega wrote: post #46

Earlier quoted context omitted.

Yes, I am aware of that trend and considering a switch. I'm personally more used to R is all, but if I found collaborators I'd jump to where the code base goes. The only other factor is I find R pretty aligned with haskell having a somewhat functional pedigree, so that code translates pretty nicely between a rapid hack at the problem to the more robust approach.

Did you look at Julia[0]? It is a Lisp, semantically, with a MATLAB-like syntax (including all the linear algebra sugar), and a LLVM back end. The stated goal of the language is to give to scienctific programmers the convenience of high level languages for prototyping, yet the speed of low level ones. It has multiple dispatch, based on a powerful type system, homoiconicity and macros (optionally hygienic). The packag…

i've looked at how it compiles to llvm, and it basically punts any optimization to the llvm side, aside from the most basic tracing jit method monomorphization/specialization.

(as in, I spent part of an afternoon end of last week skimming the entire code base).

This is troubling, because when you don't design your language to make it "manageable" to do static analysis / optimization on, it be becomes DIFFICULT to add that capability later. Julia is about as dynamic as JS or Scheme or Lua, plus it has all those runtime type tag cases in code... but whereas JS has relatively unbounded manpower (relatively!) to make a fast JIT, whereas lua is a small enough language that we have the amazing impressive LuaJIT, whereas Scheme is designed with very thoughtful semantics / specifications in mind, Julia lacks.

more succintly, Julia lacks a clear enough thoughtful choice in static/dynamic semantics for the pre LLVM side to have an easy optimization story given a small sized core team, such optimization engineering will either take a long time to develop, or will require some commercial entity to sink serious capital into writing a good JIT / optimizer. LLVM is a good backend, but it can't leverage the semantics of the input language, just the semantics of the LLVM bitcode you provide. You really really can't punt on optimization on the before LLVM side.

also: Julia doesn't have a type system, it has a dynamic check system. (Show me the code for static type check in the julia code base as it exists on github and I'll owe you a beer :) )

Let me repeat: Julia doesn't have clear static semantics / phase distinction, and doesn't really seem to do anything beyond method specialization before passing code on to LLVM. This means it can't get compelling performance compared with any other LLVM backed language.

LLVM is amazing, and i use ghc -fllvm exclusively, and while theres some really really awesome optimizations that LLVM does for haskell code (for example), a HUGE amount of high level code performance is really depending on the language (any language!) doing optimizations before passing things onto the LLVM.

Re: Show HN: I'm building an open-source, high-frequency trading system

#153

Post by freyrs3 »

boothead wrote: post #109
zygomega wrote: post #70

Earlier quoted context omitted.

> Going from matrix operations over the whole timeseries ... to deploying to an environment that streams updates to the timeseries one at a time. I think that this is an area that haskell has the potential to excel at, given it's strong guarantees on structure. exactly my thought. Algo guys get stuck in matrix land because that's where their tools take them. Whereas this came out in R last week: http://cran.r-project…

Some of the work that continuum have done on blaze [1] look to be tackling the problem of streaming in python too. In fact some of the ideas in this library come directly from haskell and one of the main developers is Stephen Diehl, who posted the very popular "what I wish I knew about haskell" slides recently. Looks like a match made in heaven :-) [1] http://blaze.pydata.org/

Stephen here, yes boothead is right. Using a combination of Haskell and Python you can make a really powerful trading system. Python has a lot of user-facing algorithm tools and Haskell has the robustness and parallelism for the backend that Python doesn't.

If you're interested in advice on how to bridge the two worlds let me know, there's a lot of of upcoming technology ( LLVM, Blaze, pipes, zeromq, cloud-haskell ) that could be very useful.

Re: Show HN: I'm building an open-source, high-frequency trading system

#154

Post by tmzt »

jmandzik wrote: post #83
boothead wrote: post #69

Earlier quoted context omitted.

Replying here as I can't add on another reply below The way we do our monitoring/UI is with zeromq/protocol buffers as an external surface to the trading system (basically a PUB socket) and a very thin bridging and translation layer to WebSockets and JSON in python. That way you get to use d3 or anything else on the front end. We use backbone, knockback and d3 with coffeescript.

Thatsvery similar to what I/we use to build our realtime network monitoring tools, swapping out python for PHP (regrettably). It's validating to see other shops in different industries zeroing in on the similar stacks.

Are you using ReactPHP for the ZMQ/webSocket parts or something else?

Re: Show HN: I'm building an open-source, high-frequency trading system

#155

Post by relaxitup »

mikevm wrote: post #102

How can someone who has absolutely no idea about the financial industry and HFT learn about it? Can anyone recommend any resources (books, tutorials, etc..)?

mikevm, check out Dark Pools by Scott Patterson. Great book about the evolution of HFT:

http://www.amazon.com/Dark-Pools-High-Speed-I-Financial/dp/0...

Re: Show HN: I'm building an open-source, high-frequency trading system

#157

Post by dchichkov »

kasey_junk wrote: post #147
fnordfnordfnord wrote: post #145

Earlier quoted context omitted.

Latency arbing is a scourge, and makes limit orders practically useless. Quote stuffing, would be illegal if trading were still done on little slips of paper. Imagine dumping 10,000 slips of paper on the trading desk, and then shouting "just kidding!" In the words of Lawrence from Office Space, "You'd get your ass kicked" Or even worse, 10,000 empty bids. That's why people hate HFT.

I'm not trying to be argumentative, but I am curious why you find latency arb so problematic, and what you think it's impact is on limit orders? I for one think latency arb is one of the bigger net wins for hft. As a market participant, each venue I have to maintain a presence at is a cost to me. I'm willing to pay the latency arb shops their cut to provide me price consistency because for my models it is much cheape…

'Quote stuffing' is there, because of absolutely ridiculous SEC rules. If you'd put in fractional prices and remove NBBO rules, there wouldn't be any 'quote stuffing'. And latency arb. What's wrong with latency arb? How else can you move information from one exchange to another fairly?

Think of HFT's as of part of the financial network infrastructure. Where exchanges play role of nodes and HFT companies role of links/queues/buffers.

Re: Show HN: I'm building an open-source, high-frequency trading system

#158

Post by nazka »

carterschonwald wrote: post #149
zygomega wrote: post #135

Earlier quoted context omitted.

Classical landing page had me hooked anyways :-)

thankee good sir. I'm glad some people appreciate my vague semblance of prioritization skills. Hopefully that pans out to being able to have any early customers be of the sophisticated sort that I'm excited to work with / help!

What is the name of your library? Or where do you post news?

Re: Show HN: I'm building an open-source, high-frequency trading system

#159

Post by carterschonwald »

nazka wrote: post #158
carterschonwald wrote: post #149

Earlier quoted context omitted.

thankee good sir. I'm glad some people appreciate my vague semblance of prioritization skills. Hopefully that pans out to being able to have any early customers be of the sophisticated sort that I'm excited to work with / help!

What is the name of your library? Or where do you post news?

It's not out yet. If you really want to hear about things as they happen, signup for the announce list linked to from www.wellposed.com , I've yet to fire off any emails to that list, but I anticipate 2-3 emails over the next 1-2 months (after a year of hard work and focused thinking).

What's also kinda awesome is I think the alpha release with all the Lin alg functionality should be under 2k loc. a lot of the work has been figuring out how to make the design composable and extensible enough that I can write a first working version with good performance just on my own. Ironically, that's also a compelling way to Validate that my tech delivers.

Re: Show HN: I'm building an open-source, high-frequency trading system

#160

Post by fbru02 »

fnordfnordfnord wrote: post #133
zygomega wrote: post #72

Earlier quoted context omitted.

There is a rather large economic rent attached to HFT at the moment. If my project or others like it can eat into that rent a touch, it might not stay that way.

Exactly, if you take the position that gains from HFT are basically "stolen" profits from otherwise more productive market participants, then any that can be "stolen back" is a win.

There have always been "market-makers" why do you care if they are algorithmic or human ?
Post reply on HN