Live data from Hacker News

Java is better than C++ for high speed trading systems

news.efinancialcareers.com

71–80 of 483 posts

Re: Java is better than C++ for high speed trading systems

#71
post #27

Earlier quoted context omitted.

This is very true. I've seen colleagues port Python to C++ and wonder why it's slower. Of course Python is slow, but it will call into efficient C routines for a lot of things that aren't just available out of the box in C++. So what does the average programmer do? They implement their own, inefficient version of this in C++. Now you've saved the overhead of copying your data from the Python to the C-world and back,…

When I moved from C/C++ to Java on the 90s, I was amazed how casually things like hash tables would be used. A single function might create a couple hash tables, do some data juggling and sorting, and solve a problem in like 20 lines of readable, performant code. In my experience, C coders almost invariably employ less performant algorithms and data structures due to the incidental complexity involved in memory alloc…

This is where I always hope a JVM will learn that, use escape analysis to keep things off the heap (they can do this), swap out data structures for more efficient ones for small amounts of data, or tune data structure initialization params (they can't do this).

Re: Java is better than C++ for high speed trading systems

#72
post #65

I wonder if this will change in the future when rust may become a more mature language. It would offer the speed and the safety needed for this kind of task.

In what regard do you think it's not mature enough yet?

The thing about Rust is that so many features are considered unstable and require switching to the nightly toolchain. That's just not suitable for real-world production use - yes, stuff does get stabilized eventually, but it takes time to really nail down the best possible design. Still, it's way better than the huge mess that is C++.

Re: Java is better than C++ for high speed trading systems

#73

I’ve seen this sentiment before and worked on both a “low latency Java” team and low latency C++ teams. I have some sympathy for the idea that the JVM is better since it means you won’t spend all your time chasing crash reports. The thing is, like another comment hinted at, is that this issue is generally more reflective of the environment you build in than the technology choice. Here’s a good talk on the reasons for…

I thought the actual approach taken these days is to disable GC for any JVM based trading systems and use arena based architectures or similar for application where you need super low latency, like a LOB mirror or execution engine, no?

Yeah, basically everything has to be that way.

At the super fast speeds you start running into things like:

* why won’t the devirtualizer trigger?

* these object headers are sure wasting a lot of cache

* there’s a lot of forced pointer indirection

And you just end up spending vast amounts of time and effort trying to shave off those few microseconds you’re wasting in the JVM. Once you add in all the effort trying to get around the GC, it’s bleak picture for competing with quality C++ without spending far more effort.

Re: Java is better than C++ for high speed trading systems

#74
post #66

I used to work on a high-speed messaging system built in Java, and garbage collection was a killer. No matter how much we tweaked the code and the GC parameters, and this eventually got escalated pretty high up in Sun itself, we'd get these occasional multi- minute pauses where the whole thing would just freeze up, buffers would fill and stuff would start dropping on the floor. This was over 10 years ago, so things m…

You should not have any full GC (pause) events, ever, in properly written production Java code. Source: I worked on several extremely high performance Java systems at Sun and also worked optimizing various Java servers at subsequent jobs based on my Sun experience. At one company (after Sun) I took on a backend service which was doing GC pauses every 3-5 minutes, the code was a dumpster fire mess. After cleaning thin…

Extremely high performance properly written Java code has been mentioned so many times in Java performance discussions that it has acquired an almost legendary status, right there with the sufficiently smart compiler.

Re: Java is better than C++ for high speed trading systems

#75

I’ve seen this sentiment before and worked on both a “low latency Java” team and low latency C++ teams. I have some sympathy for the idea that the JVM is better since it means you won’t spend all your time chasing crash reports. The thing is, like another comment hinted at, is that this issue is generally more reflective of the environment you build in than the technology choice. Here’s a good talk on the reasons for…

The worst (messiest, laziest and most chaotic) engineer I know works at a brokerage, coding obscenely large and unreliable edifices (and in a number of languages, from C++ to Excel+VB)... He does what he is asked, and doesn't worry about telling them what they need. They never demand code that doesn't crash, they see that as a fact of life, and developers are an overhead. Basically someone else owns the risk, the brokers don't, so why should he?

People who don't like this style (or the code) leave.

The direction for quality has to come from the top and be supported by systematic mechanisms. In fact that would be a question to ask in interviews.

Re: Java is better than C++ for high speed trading systems

#76
post #27

Earlier quoted context omitted.

This is very true. I've seen colleagues port Python to C++ and wonder why it's slower. Of course Python is slow, but it will call into efficient C routines for a lot of things that aren't just available out of the box in C++. So what does the average programmer do? They implement their own, inefficient version of this in C++. Now you've saved the overhead of copying your data from the Python to the C-world and back,…

When I moved from C/C++ to Java on the 90s, I was amazed how casually things like hash tables would be used. A single function might create a couple hash tables, do some data juggling and sorting, and solve a problem in like 20 lines of readable, performant code. In my experience, C coders almost invariably employ less performant algorithms and data structures due to the incidental complexity involved in memory alloc…

absolutely. Few years ago i got back deep into C++ after 18 years of mostly Java, and i'm amazed, it is like travel back in time - we (a large C++ platform) are looping over vectors instead of using hashtables/maps, copying strings all the time (we have 4 major types of strings plus some minor ones), and don't get me started on multithreading and locking/synchronization - such an easy and natural thing in Java - even these days the people in C++ world is still seem to be hesitant to use locking and as result of that hesitance push themselves into a myriad of multithreading issues when they risk and/or are forced to use threads. It a sight to behold when a customer waits for that one core/thread to finish the operation on a 400 core 16TB RAM machine - if it were Java all the 400 cores would be burning, the performance would still feel kind of so-so, and the customer would be building up the nerve to upgrade to 800 core 32 TB :)

>pointer indirection

30+ years ago, a student at University, i loved how pointers allowed to code algorithms efficiently and expressively compare to the non-pointer languages of the time. In the industry today the situation is completely opposite as correctly noted by the parent.

Re: Java is better than C++ for high speed trading systems

#77
post #66

I used to work on a high-speed messaging system built in Java, and garbage collection was a killer. No matter how much we tweaked the code and the GC parameters, and this eventually got escalated pretty high up in Sun itself, we'd get these occasional multi- minute pauses where the whole thing would just freeze up, buffers would fill and stuff would start dropping on the floor. This was over 10 years ago, so things m…

You should not have any full GC (pause) events, ever, in properly written production Java code. Source: I worked on several extremely high performance Java systems at Sun and also worked optimizing various Java servers at subsequent jobs based on my Sun experience. At one company (after Sun) I took on a backend service which was doing GC pauses every 3-5 minutes, the code was a dumpster fire mess. After cleaning thin…

Maybe, but that just raises the question...If you're coding your system under the constraint that you should not be having any GC-event, what's the point of picking a GC language in the first place?

Re: Java is better than C++ for high speed trading systems

#78
post #26
post #15

Earlier quoted context omitted.

Avoiding allocations is the key part: "...we avoid all the garbage". I found the same is true when trying to write high performance Java UI code for Android. If you have to do something complex each frame, make sure to pre allocate or pool your objects. If the hot parts of the code are written in a C-like style, then the JIT and other optimizations can give you C-like performance. You'll still need to write C code if…

I've found that coding in a "C-like style" offers great performance too. Are there any languages that target the JVM and are designed for higher speed? If it enforced the "C-like style" at the language level, perhaps it would be easier to follow? Of course at this point the usual answer is just use Rust, but is there a language that meets in the middle? Sometimes I just want the GC to do the work and I'm okay with th…

> Are there any languages that target the JVM and are designed for higher speed?

Yes: Java

Performant Java looks mostly like C, just with no manual malloc() calls.

Sure you could write Java in a J2EE way but.. that not a good choice.

Re: Java is better than C++ for high speed trading systems

#79
These discussions tend to zoom in on just one quality attribute, latency in this case. The point this article tries to make is that there are multiple quality attributes and that you trade them off against each other. There is no right or wrong here other than getting sucked into having these choices imposed on you by irrational arguments made by engineers about things they care about vs. things that are actually business relevant.

In the case of high speed trading systems, any second they are late to market, they are losing money. It might be the fastest thing ever once it's finished but while it's not running, its slower competitors have the market to themselves. Worse, any month you are late is time your competitors can use against you to learn from your mistakes and beat you at your own came. High speed trading is of course extremely competitive. So things change all the time. So, now we are talking speed of development and maintainability in addition to raw performance and throughput. The impact of most improvements you make are in any case typically temporary in the sense that they only provide you an advantage as long as your competitors don't catch up. So, getting bogged down into lengthy maintenance and bug fixing cycles while your competitors copy everything that you got right means you lose some or all of the market opportunity.

This article mentions what happens when C++ projects fail. Basically, you are late to market with something that doesn't work as advertised (slow, unstable). That's not an inherent risk of course but it's a risk none the less. Performance often comes at the price of complexity. Complexity has its own penalties as well in the form of e.g. poor maintainability, bugs, or stability. C++ is frankly notorious for this. You compensate with having skilled engineers acting in a disciplined way. And sometimes that actually works as advertised. The project mentioned in the article sounds like it had issues across the board. Probably, at least one of the root causes for that was that they were trying to take some short cuts to get it to market.

The JVM ecosystem has a lot going for when making such trade-offs. That's why it's so entrenched in the industry. And it's been around long enough that there are lots of solutions to all sorts of issues that you might need solutions for. Also, you always have the option to go native in Java; it's not an either or kind of thing.

For the same reason you see people opting for a not particularly fast language like python in a domain where you get to use fast GPUs to get things done quickly (i.e. machine learning). Python gets that job done, everything on the critical path is basically delegated to lower level native stuff running on a GPU (or custom hardware in some cases). I wouldn't be surprised to learn that some high frequency traders have lots of python code around in addition to their precious native code. It would be a pragmatic thing to do.

Understanding your domain and understanding the trade offs is what differentiates successful companies and engineers here. There are lots of valid reasons for preferring C++ for some things. Personally, I rarely encounter such requirements and I'd probably prefer Rust over C++ if I did. And since I don't, I actually have limited experience with Rust as well because other than the intellectual challenge I don't see how I'd be getting much real world advantage out of it. I'm pretty sure that holds true across large parts of our industry. It's why you see people actively not caring about performance by choosing to run on slow (virtual hardware) on slow cloud networks using slow run time environments that include slow interpreters and slow application frameworks because they can get stuff done quickly with it and can trivially scale by throwing dollars at the problem. Slow is fast enough for most.

Re: Java is better than C++ for high speed trading systems

#80
post #66

I used to work on a high-speed messaging system built in Java, and garbage collection was a killer. No matter how much we tweaked the code and the GC parameters, and this eventually got escalated pretty high up in Sun itself, we'd get these occasional multi- minute pauses where the whole thing would just freeze up, buffers would fill and stuff would start dropping on the floor. This was over 10 years ago, so things m…

You should not have any full GC (pause) events, ever, in properly written production Java code. Source: I worked on several extremely high performance Java systems at Sun and also worked optimizing various Java servers at subsequent jobs based on my Sun experience. At one company (after Sun) I took on a backend service which was doing GC pauses every 3-5 minutes, the code was a dumpster fire mess. After cleaning thin…

Did you ever detailed on the Internet about those methods to attain no GC pause? It would be interesting to learn from your experience
Post reply on HN