Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We chose Java for our high-frequency trading application

medium.com

Post reply on HN
Print view View original

271–280 of 287 posts

Re: We chose Java for our high-frequency trading application

#271

Post by pjmlp »

pak9rabid wrote: post #51

Speaking about JVM garbage collection, why isn't automatic reference counting being used instead of traditional garbage collection? It seems kind of like the best of both words in a automatic memory-managed environment of providing safety as well as performance. Languages like Objective-C and Swift have proven that this is very viable. I wonder what's keeping the JVM from adopting it?

Because it is way slower despite cargo cult about its performance, and any optimization algorithms, just turn reference counting into poor man's tracing GC.

https://github.com/ixy-languages/ixy.swift/blob/master/perfo...

Re: We chose Java for our high-frequency trading application

#272

Post by mschuster91 »

kanox wrote: post #215
mschuster91 wrote: post #201

Earlier quoted context omitted.

The key difference between Java and PHP is the lack of a compilation step. I can change a line of code, press cmd+r and I see the change - whereas in Java more often than not I have to wholly restart the application container as hot-reloading is a complicated mess and tough to set up. Also, in Java it is easy to end up with resource leaks whereas PHP always starts from a clean slate for each request. Way less headach…

> Also, in Java it is easy to end up with resource leaks whereas PHP always starts from a clean slate for each request. Way less headache for operations. That assumes you're running a new separate PHP processes for each request and an application built this way is unlikely to scale terribly well.

> That assumes you're running a new separate PHP processes for each request and an application built this way is unlikely to scale terribly well.

No. PHP always wipes the complete execution stack even in a long lived execution model like FPM or Apache mod_php.

Re: We chose Java for our high-frequency trading application

#273

Post by mschuster91 »

koheripbal wrote: post #204
mschuster91 wrote: post #201

Earlier quoted context omitted.

The key difference between Java and PHP is the lack of a compilation step. I can change a line of code, press cmd+r and I see the change - whereas in Java more often than not I have to wholly restart the application container as hot-reloading is a complicated mess and tough to set up. Also, in Java it is easy to end up with resource leaks whereas PHP always starts from a clean slate for each request. Way less headach…

Right, but in a corporate production environment where everything is probably deployed compiled or compiled shortly after, does that matter? No one is altering prod code.

It certainly makes zero or almost no downtime deployments easier. Assume a Java application with a database that accesses said database during initialization - for a schema change you have to take great care in code that both the old and the new code can concurrently work on the same database to achieve low downtime.

With a PHP application, it's easier - spin up the new Docker container, disconnect the loadbalancer, do the schema change, and connect the loadbalancer to the new Docker container.

Re: We chose Java for our high-frequency trading application

#274

Post by gozzoo »

This article starts very interesting but the rest sounds like infomercial for Zing by Azul Systems.

Both issues related to low latency metioned here (JIT and GC) can be solved with very simple and common methods with the JVM provided by Oracle.

We had similar issues in my old company, where we implemented a graph database in Java. It was very slow and somtimes freezed for alomost a minute (512 GB heap).

Here are the strategies we used for solving these issues.

- performance - we built warm up logic in our app, which called the most time sensative parts on startup. It delays the start up process by 10 sec, but after that the application is very fast.

- GC - we pre-alocated big chunks of memory outside of the heap and put there the 'heavy' objects - those which require most effort from the GC. We also optimized the code so that all other allocations are in the stack (escape analysis). This requires some discipline, but eventually we ended up in situation where there was nothing left for the GC to do.

I'm not sure if it's worth paying hunderds tousands $ in license fees for something that can be achieved with good programming practices and discipline, which most companies need anyways.

Re: We chose Java for our high-frequency trading application

#275

Post by whizzter »

bluestreak wrote: post #264
whizzter wrote: post #211

Earlier quoted context omitted.

Would you do it again? Coming from C/C++ first, Java then and coming to C# it just feels so much more pragmatic with f.ex. struct(s), slices and stackalloc (and unsafe blocks with pointers in a pinch) allowing for GC less programming w/o resorting to turning pointers to integers and using function calls for memory access all around. (Noticed that you do i guess query compilation via the ASM toolkit?)

It is a good question. I feel there is a happy medium where boilerplate is in Java and more intricate data processing routines are in C++. Makes both worlds simpler. I like C++ better. Those things you mentioned - Java truly sucks at indeed, but you don't always need them. When it comes to IDE, testing, compilation speed, cross platform code and finding talent - Java is way easier than C++.

Yeah i see that outlook, my comment was actually mostly about C# (been using it for the past year and these things above are in it as well as all the base being Java-ish)

Re: We chose Java for our high-frequency trading application

#276

Post by otabdeveloper4 »

zarkov99 wrote: post #123
otabdeveloper4 wrote: post #95

Earlier quoted context omitted.

Complaining about advertising is like complaining about money. Business absolutely cannot exist without advertising. The need must be filled one way or another. The same cannot be said about HFT and stock markets in general.

Its not the need for advertising. Its the unreasonable effectivness and devastating side-effects of the attention sequestration techniques developed in the last 10 years. Its not only that years of our of lives are being directly stollen by vapid bullshit, its that the effects of this manipulation extend well beyond the actual time spent engaging with it, with absolutely horrific consequences to our mental health, ou…

Again, business cannot exist without advertising. Even money and finance is negotiable - business can survive on barter alone, even though the overhead would be monstrous.

But advertising is not negotiable, it must exist in one way or another if the economy exists.

If you want to go down this road, argue about the "how", not the "what".

The idea of banning advertising outright is too silly for words.

Re: We chose Java for our high-frequency trading application

#277

Post by CodesInChaos »

pkulak wrote: post #258
mfontani wrote: post #142

Earlier quoted context omitted.

A tangible example: if you've got a script the likes of: #!/bin/bash echo 'Hi!' sleep 20 echo 'Hello!' ... and runt it ... and while the script is executing the sleep line, modify it so that instead of 'echo "Hello!"' it does something else, like 'echo "Oops!"'... ... once it resumes from the sleep, it'll execute the updated code. Don't ask me how I know... I just wish it were more well-known. Don't modify shell scri…

That is amazing. Bash doesn't even load the script into memory? It just keeps a line pointer and chunks through the open file? There's something comforting about that. :D

I think batch files do a linear scan for the target label every time the goto is executed.

Re: We chose Java for our high-frequency trading application

#278

Post by abledon »

whizzter wrote: post #218
nobleach wrote: post #135

Earlier quoted context omitted.

Is the Azul JVM open source? I thought it was just the support which costed money. (I'm just not familiar enough with it)

It is commercial, and sadly their GC algorithms are patented. I'm really anxious for those patents to start running out because the functionality is quite clever. Basically one main reason you need to stop the world in common GC's right now (and thus cause pauses) is because if you want to compact memory a GC thread has no idea if other threads reads/writes to an object while moving leading to worst case corrupt/lost…

why does the hardware provide page-fault for free again?

Re: We chose Java for our high-frequency trading application

#279

Post by ksec »

MangoCoffee wrote: post #149
Androider wrote: post #131

Earlier quoted context omitted.

1TB of RAM is cheap though, versus spending engineering hours. Reminds me of the classic WTF "That would've been an option too" https://thedailywtf.com/articles/That-Wouldve-Been-an-Option...

agree. hardware is getting cheaper and cheaper

Except Memory / DRAM.

Re: We chose Java for our high-frequency trading application

#280

Post by eneveu »

cashewchoo wrote: post #5

I think I actually saw these folks present at JavaOne a couple years ago? Either that or there's more than one shop branding itself as "HFT" that uses Java. I worked in the industry and it's always a little funny to see who calls themselves HFTs vs quants. Basically, there's a bit of a spectrum of fast vs smart. In general it's hard to do incredibly smart stuff fast enough to compete in the "speed-critical" bucket of…

I think the JavaOne presentation was made by the LMAX devs: https://www.youtube.com/watch?v=eTeWxZvlCZ8

They built the Disruptor data structure around 2011 for their high performance financial exchange on the JVM: https://lmax-exchange.github.io/disruptor/files/Disruptor-1....

I used the Disruptor at a smart grid startup in 2012-2014, after LMAX open sourced it.

Martin Thompson has a lot of interesting presentations on the concept of mechanical sympathy:

- https://www.youtube.com/watch?v=929OrIvbW18 - Adventures with concurrent programming in Java: A quest for predictable latency by Martin Thompson - https://www.youtube.com/watch?v=03GsLxVdVzU - Designing for Performance by Martin Thompson

Post reply on HN