Live data from Hacker News

We chose Java for our high-frequency trading application

medium.com

221–230 of 287 posts

Re: We chose Java for our high-frequency trading application

#221
post #63

I have a question: When i finished my theoretical Physics PhD, HFT was THE way out of academia for making money in my field. I didn't follow this path as it felt at the time a pretty evil thing to do. Does HFT provide any benefits for our society?

Yes, it absolutely does. Rapid efficient capital allocation allows our economy to react nimbly to anything that gets thrown at it (like covid).

The sharp v recovery in the markets was surprising, but not so much when you realize what's backing it. People gave credit to the wrong group. They said it was the fed, it wasn't, it was the HFT. All the fed did was add some zeros to accounts and relax some interest rates. HFT had to figure out where the money goes.

Without HFT, it likely could have taken a year for the market to recover. It would have taken forever to unravel all the changes (some ephemeral, some not) and figure out how to efficiently allocate all that money.

If I was the democrats, I'd find a way to connect the market to main street rather than try to create ftt. I believe the first country that does this will win the 21st century.

Imagine being able to come up with a business idea, partner with some friends, do a bit of proving out and publicizing your team and money just flows into it based purely on the merit of your real potential. No politics, no long lunches and dinners with VC, no irrational idiots. Just pure, unfettered meritocracy.

Or imagine having a long standing mainstreet business that gets hit with a one time, out of its control, disaster. No need to go the bank with hat in hand, the money just shows up in your account, with hyper competitive interest rates. The same could be said if you have an idea for expansion or efficiencies. Just send out a press release with a few specifics, and next day you are underwritten.

SPAC is sort of like that. Sadly, democrats have their own brand of xenophobia. They tax what they don't understand. It's very unfortunate that the GOP didn't vote with the democrats to impeach Trump. Historically, I think it will go down as the most colossal political mistake since Nixon.

One way to think about all this, is DeepMind / starcraft. AI could already out-micro any competitor, but now it's getting to a point where it can out-strategy as well.

Having tools like that to manage our markets is a very powerful tool and makes the US not only hyper efficient but a far more competitive than any other nation.

That said, the whipsaws of computers figuring out things too fast can be tough on people. pure, unfettered meritocracy can be hard for those without merit to offer.

Yang and his UBI isn't a bad idea.

Where it provides less value are traders who intentionally inject volatility. Some guy went to jail over that (flash boys)

Re: We chose Java for our high-frequency trading application

#222
post #163

Earlier quoted context omitted.

Why would you think that? Rust's major selling point for safety guarantees is the borrow checker. It does not provide most of OCaml's type and functional semantics. That being said: Jane Street is pretty avant garde in this respect. I expect there already is, or soon will be, a successful trading firm which likewise builds its tech brand on Rust and alternatives to Rust standard library primitives. But overall adopti…

It has immutability, no null pointers, and error types. It has escape hatches available for everything, but unnecessary use of these should be caught in code review, so I don't think it's a big deal. Not sure what correctness features OCaml could add beyond those, but then again I've never programmed in OCaml, only Scala, so I might be wrong.

This is perhaps the most "blub" comment ever written. There are whole classes of bugs that can be caught with more advanced type systems that the Rust type system + borrow checker will never catch. No null pointers and immutability is the bare minimum.

If you're interested, check out https://www.youtube.com/watch?v=10gSoVZ5yXY for an example of the types of compile time guarantees can be had.

Re: We chose Java for our high-frequency trading application

#224

I think I actually saw these folks present at JavaOne a couple years ago? Either that or there's more than one shop branding itself as "HFT" that uses Java. I worked in the industry and it's always a little funny to see who calls themselves HFTs vs quants. Basically, there's a bit of a spectrum of fast vs smart. In general it's hard to do incredibly smart stuff fast enough to compete in the "speed-critical" bucket of…

I discovered something amazing when working with some people who were writing HFT software. Why do you need 1TB of RAM in these machines? Because when you're Java based, you want to avoid stop-the-world GC pauses. These trading systems only have to be up from 9:30AM-4:30PM EST, so they simply disable GC altogether! At the end of a trading day, restart the app or reboot the system.

How was these achieved before Epsilon? http://openjdk.java.net/jeps/318

Also ZGC / Shenandoah can help a lot otherwise

Re: We chose Java for our high-frequency trading application

#225
post #63

I have a question: When i finished my theoretical Physics PhD, HFT was THE way out of academia for making money in my field. I didn't follow this path as it felt at the time a pretty evil thing to do. Does HFT provide any benefits for our society?

You can go on Robinhood and buy stocks for free. Courtesy of some HFT in return for getting first dibs on your trades. If we compare that to the fees of yesteryear it's brought real value to long-term retail investors.

Unsophisticated investors only using market orders will also pay less spreads than they used to.

Going out on a limb, I think the losers are mutual funds and similar who are less able to monetize their fundamental analysis by buying or sell big chunks of stock without moving the market.

Re: We chose Java for our high-frequency trading application

#226

Earlier quoted context omitted.

What the GP is referring to (Misc.Unsafe) is basically going back to manual pointer reads/writes (IE writing more or less C code in Java), the benefit is you get C speed/memory layout/cache-locality but with the downside of writing it in a less suited language (Java). HOWEVER If you DO _allocate_ much then 1TB seems like a great choice since GC pauses are killers w.r.t. latency in comparison to cache issues. A cache…

Can you explain why you would code java this way instead of dropping down to C?

Probably because they started out in Java and don't want a full rewrite once they've gotten this far (might be that they don't have many C++ devs anyhow). I remember working on similar codebases 10 years back when J2ME games were still a thing (J2ME runtimes usually had horrible GC's and some people really went overboard in trying to avoid them) and it wasn't that fun at all (Even if the questDB codebase that is linked elsewhere in this thread seems to be saner).

Also JNI for calling C/C++ code is fairly bad and error prone (if you want to combine code), JNA is better but i think this is one of the bigger reasons that Oracle is funding GraalVM is that it promises "seamless" interoperability and breaking up things into Java and C/C++ parts might be a good option in the future.

Re: We chose Java for our high-frequency trading application

#227
post #190

Earlier quoted context omitted.

isn't true HFT purely latency driven? ie: if you see a sell order for $0.10 and a buy order for $0.11 before anyone else, it is like free money.

I'm a HFT quant trader and HFT is not purely latency driven. You don't want to be slow, but most firms are increasingly perusing longer and longer term strategies that aren't as latency sensitive. Pure risk free latency arb isn't really a profit center anymore. The speed wars are pretty much over. The industry has consolidated and any player that didn't adapt to the new, more cerebral game is dead.

> The speed wars are pretty much over. The industry has consolidated and any player that didn't adapt to the new, more cerebral game is dead.

Except for the one that won the speed war, and keeps investing massively to remain 10ns ahead.

Re: We chose Java for our high-frequency trading application

#228
post #63

I have a question: When i finished my theoretical Physics PhD, HFT was THE way out of academia for making money in my field. I didn't follow this path as it felt at the time a pretty evil thing to do. Does HFT provide any benefits for our society?

The market is a machine that routes society’s resources into making nearly everything. Making better routing decisions has immense value. When someone can profit from the same decision everyone else would make by doing it 0.000000012 days more quickly, IMHO he’s exploiting a design flaw in the way the market clears. That’s amplifying noise rather than signal.

ok, that was a bit of my original question (not well asked).

i understand any civilisation needs funding to grow (do we need growth is another core question), and possibly market to do so.

yet any meaningful decision won't be made at the millisecond scale, sure you can train a model to learn short term rules and generate long term profit. but does that profit society as a whole, i doubt it.

Re: We chose Java for our high-frequency trading application

#229
post #63

I have a question: When i finished my theoretical Physics PhD, HFT was THE way out of academia for making money in my field. I didn't follow this path as it felt at the time a pretty evil thing to do. Does HFT provide any benefits for our society?

what aspect of theoretical physics did you get your phd in?

strings and dualities

https://arxiv.org/abs/1109.4280

Re: We chose Java for our high-frequency trading application

#230

Earlier quoted context omitted.

why not just use C++ or something and never deallocate memory then?

You generally choose the language which has the libraries and ecosystem which solves your problem. For instance, you'd be silly to use anything but Java to work with Hadoop. This is the pragmatic choice.

But as soon as you make use of almost any library from the ecosystem, you get allocations. Doesn't that defeat the purpose? Why not isolate the Java-using part in a separate process from the non-allocating fast part?
Post reply on HN