Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We chose Java for our high-frequency trading application

medium.com

Post reply on HN
Print view View original

151–160 of 287 posts

Re: We chose Java for our high-frequency trading application

#151

Post by aasasd »

deepGem wrote: post #105
oppositelock wrote: post #74

Earlier quoted context omitted.

I discovered something amazing when working with some people who were writing HFT software. Why do you need 1TB of RAM in these machines? Because when you're Java based, you want to avoid stop-the-world GC pauses. These trading systems only have to be up from 9:30AM-4:30PM EST, so they simply disable GC altogether! At the end of a trading day, restart the app or reboot the system.

I guess what you mean is the no-op garbage collector which is available in Java 11 http://openjdk.java.net/jeps/318 Even this isn't fail proof right ? Last-drop latency improvements. For ultra-latency-sensitive applications, where developers are conscious about memory allocations and know the application memory footprint exactly, or even have (almost) completely garbage-free applications, accepting the GC cycle might…

> * I guess what you mean is the no-op garbage collector which is available in Java 11*

I think I heard of the same pattern before 2018, so I guess there are other ways to do it. Perhaps Java has some option like memory usage value at which the GC is run.

Re: We chose Java for our high-frequency trading application

#152

Post by nicoffeine »

_threads wrote: post #101

Why are we still building high frequency trading apps ? Wasn’t it part of the old world we meant to change ?

Money corrupts. Wall Street money corrupts absolutely.

In a sane market, there would be no advantage to making an investment and then selling it a few milliseconds later. It introduces unnecessary volatility and incentivizes insider trading and other illegal activity[1]. It defeats the original purpose of the market: to make long term investments. It also leads to some unbelievable waste:

Traders’ need for speed has grown so voracious that two companies are currently building underwater cables (price tag: around $300 million each) across the Atlantic, in an attempt to join Wall Street and the London Stock Exchange by the shortest, fastest route possible. When completed in 2014, one of the cables is expected to shave five to six milliseconds off trans-Atlantic trades.[2]

The problem is that our 20% of our economy is "financial services". It's doubled in the last hundred years or so[3]. Technology isn't shrinking this sector because the people who own it also own lobbyists who own Congress. They write the rules so they can play stupid financial games, and then force the government to bail them out when the latest ponzi scheme hits maturity.

[1] https://www.zerohedge.com/article/first-hft-casualty-finra-f...

[2] https://www.motherjones.com/politics/2013/02/high-frequency-...

[3] https://www.washingtonpost.com/news/monkey-cage/wp/2016/03/2...

Re: We chose Java for our high-frequency trading application

#153

Post by pjmlp »

scaladev wrote: post #56
pjmlp wrote: post #47

Earlier quoted context omitted.

A language that brings the Gentoo installation experience to external libraries and requires special purpose data structures to compile properly?

Do you prefer the Linux from Scratch installation experience which C and C++ provide?

Only for clueless developers that don't know their trade.

Re: We chose Java for our high-frequency trading application

#154

Post by mroche »

Androider wrote: post #131
benjaminjackman wrote: post #121

Earlier quoted context omitted.

Speaking from experience with JVM HFT applications (we used Scala). There are a lot of tricks though to not require 1TB. And allocation in general is a bad idea even if you don't collect because it scatters stuff all over memory and messes up cache locality. You really, really don't want to allocate in a performance sensitive jvm application if you can avoid it. It's the opposite of a lot of what I was told and taugh…

1TB of RAM is cheap though, versus spending engineering hours. Reminds me of the classic WTF "That would've been an option too" https://thedailywtf.com/articles/That-Wouldve-Been-an-Option...

This was a great read, thanks for that! Never heard of The Daily WTF before.

Re: We chose Java for our high-frequency trading application

#155

Post by fractionalhare »

nv-vn wrote: post #143
fractionalhare wrote: post #28

Earlier quoted context omitted.

The person you're replying to is almost certainly not using the word "safe" in the same sense you are. In particular: safety in trading has to do with risk management, strategy correlation, side effects, and correctness. Software security matters, in the abstract, but trading firms don't care about it nearly as much as tech companies do. The other thing is that being as fast as C++ is not compelling enough a reason t…

Jane Street famously uses OCaml for these types of guarantees, so I would think Rust would offer similar benefits.

Why would you think that? Rust's major selling point for safety guarantees is the borrow checker. It does not provide most of OCaml's type and functional semantics.

That being said: Jane Street is pretty avant garde in this respect. I expect there already is, or soon will be, a successful trading firm which likewise builds its tech brand on Rust and alternatives to Rust standard library primitives. But overall adoption will probably continue to be anemic.

Re: We chose Java for our high-frequency trading application

#156

Post by proverbialbunny »

immmmmm wrote: post #63

I have a question: When i finished my theoretical Physics PhD, HFT was THE way out of academia for making money in my field. I didn't follow this path as it felt at the time a pretty evil thing to do. Does HFT provide any benefits for our society?

When bitcoin was young there wasn't much in the way of HFT between exchanges, which sucked because if you wanted to transfer coins to someone across the world you would on average lose 7-12% on price differences. Western Union, in comparison, was a flat 10%, making it bitcoin at the time almost pointless.

When there isn't enough volume HFT strategies tend to not run, and so to give an example, my first time trading options was an iron condor on /GC (gold). I bought at a reasonable price, but when I went to sell, where I would have netted a cool 50%, because of low volume and I didn't know to set a limit order, I lost about 50%, so even if my trade was "successful" I lost. If there was a bit more volume HFT could step in and we could have normalized prices where limit orders are not required. On the other end, without HFT, it's easy to play that game and setup orders of opportunity where someone else might mess up and you can make a pretty penny, not that I've personally tried this.

Re: We chose Java for our high-frequency trading application

#157

Post by nv-vn »

gpsx wrote: post #129
traK6Dcm wrote: post #82

Earlier quoted context omitted.

Some people argue that HFT provides liquidity for retail investors, but it's debatable whether that liquidity is real or not since it'll be gone during black swan events. At the same time HFT profit from uninformed/retail flow. So it's debatable wether the actual activity provides values. I'd say probably not. But there can be indirect value in working in HFT, just like there is with other demanding jobs. There's int…

I'm not an economics or finance expert, so I may not have a very extensive view of this, but I think there is a meaning to the expression of providing value. If a product has more value to a person than the price of the product, assuming they have correct knowledge, that is the creation of value. In high frequency trading, or stock trading in general, people may be willing to pay more than the listed price, but I don…

I disagree. Trading may seem zero sum on a short timescale, but over the long term markets (empirically) trend upwards. Thus, simply being invested in diverse indices should yield some positive return in the long run. This makes sense as long as we assume GDP growth over time, which is an assumption that has kinda been baked into government/economics to my understanding.

At a short time scale, yes there tends to be a winner and a loser, but even then it's not so clear cut. Most of the trades happening at a short time scale are institutional rather than retail (i.e. people who have the risk tolerance for this type of trading & who have profited enough from it to keep doing this type of behavior). The activity of most HFT firms is some combination of market making & arbitrage, where the profit can be thought of as a fee for trading for other participants (e.g. Robinhood doesn't charge commission but sells order flow, which tends to be cheaper than commission per trade). In general, price improvement is also possible since these make markets more efficient, so as an example it would not be unusual to see firm A sell to firm B who sells to firm C, where both A & B see a profit from their sales. This is possible because prices across markets are not automatically synced, so A has access to a better price than B who has access to a better price than C. And because institutional trading dwarfs any kind of retail trading, the trades can appear high frequency from the perspective of the HFT firm while only occurring, say, once every six months from the perspective of the retail trader.

Re: We chose Java for our high-frequency trading application

#158

Post by jacques_chester »

Androider wrote: post #131
benjaminjackman wrote: post #121

Earlier quoted context omitted.

Speaking from experience with JVM HFT applications (we used Scala). There are a lot of tricks though to not require 1TB. And allocation in general is a bad idea even if you don't collect because it scatters stuff all over memory and messes up cache locality. You really, really don't want to allocate in a performance sensitive jvm application if you can avoid it. It's the opposite of a lot of what I was told and taugh…

1TB of RAM is cheap though, versus spending engineering hours. Reminds me of the classic WTF "That would've been an option too" https://thedailywtf.com/articles/That-Wouldve-Been-an-Option...

> 1TB of RAM is cheap though, versus spending engineering hours.

It's not about the amount of RAM, it's how fast and predictable the overall system is. Note in particular the remark about cache locality; that can be the difference between nanoseconds and microseconds.

Re: We chose Java for our high-frequency trading application

#159

Post by aasasd »

oppositelock wrote: post #74
cashewchoo wrote: post #5

I think I actually saw these folks present at JavaOne a couple years ago? Either that or there's more than one shop branding itself as "HFT" that uses Java. I worked in the industry and it's always a little funny to see who calls themselves HFTs vs quants. Basically, there's a bit of a spectrum of fast vs smart. In general it's hard to do incredibly smart stuff fast enough to compete in the "speed-critical" bucket of…

I discovered something amazing when working with some people who were writing HFT software. Why do you need 1TB of RAM in these machines? Because when you're Java based, you want to avoid stop-the-world GC pauses. These trading systems only have to be up from 9:30AM-4:30PM EST, so they simply disable GC altogether! At the end of a trading day, restart the app or reboot the system.

Game devs face similar challenges, especially on phones. Though they have to be less radical in their solutions—which afaik usually boil down to ‘preload the level into memory, use object pools, and don't do any new allocations’.

Re: We chose Java for our high-frequency trading application

#160

Post by [deleted] »

Androider wrote: post #131
benjaminjackman wrote: post #121

Earlier quoted context omitted.

Speaking from experience with JVM HFT applications (we used Scala). There are a lot of tricks though to not require 1TB. And allocation in general is a bad idea even if you don't collect because it scatters stuff all over memory and messes up cache locality. You really, really don't want to allocate in a performance sensitive jvm application if you can avoid it. It's the opposite of a lot of what I was told and taugh…

1TB of RAM is cheap though, versus spending engineering hours. Reminds me of the classic WTF "That would've been an option too" https://thedailywtf.com/articles/That-Wouldve-Been-an-Option...

[deleted]
Post reply on HN