why not just use C++ or something and never deallocate memory then?
Malloc is usually slower than GC-based allocation (which basically just increments a pointer). Of course, one can emulate this via custom allocators in C++. My guess is that Java development is just easier and quicker, and the JIT may even result in more effective optimizations than AOT compilation.
In C++ you can link with your own version of malloc (one which just returns the next consecutive block, and can be implemented simply by adding the allocated size to a pointer).
The market is a machine that routes society’s resources into making nearly everything. Making better routing decisions has immense value. When someone can profit from the same decision everyone else would make by doing it 0.000000012 days more quickly, IMHO he’s exploiting a design flaw in the way the market clears. That’s amplifying noise rather than signal.
ok, that was a bit of my original question (not well asked). i understand any civilisation needs funding to grow (do we need growth is another core question), and possibly market to do so. yet any meaningful decision won't be made at the millisecond scale, sure you can train a model to learn short term rules and generate long term profit. but does that profit society as a whole, i doubt it.
Trading facilitates more efficient allocation of resources at both a geographical and temporal scale. HFT market makers facilitate more efficient trading. If anyone tried pitching automated trading to you as some form of altruistic endeavor then obviously that’s bullshit. These people are out here to make a buck. Those milliseconds to nanoseconds moves are just competition between MMs happening in realtime. Is it done for the sake of a better world? No. But it just so happens to benefit society as a byproduct (on average) in the form of tighter spreads and liquidity. It’s up to you to decide if that’s good enough for you.
If we’re going to paint anyone as “real” villains in the world of trading then I’d argue it’s the informed big block traders. ie hedge funds, banks trading exotics with a high barrier of entry, etc. At least MMs are largely market neutral.
here is one: https://github.com/questdb/questdb . Disclaimer, I work on this project. Main reason we use Java is speed of development (which increased with amount of base libraries written) and ease of testing.
Would you do it again? Coming from C/C++ first, Java then and coming to C# it just feels so much more pragmatic with f.ex. struct(s), slices and stackalloc (and unsafe blocks with pointers in a pinch) allowing for GC less programming w/o resorting to turning pointers to integers and using function calls for memory access all around. (Noticed that you do i guess query compilation via the ASM toolkit?)
It is a good question. I feel there is a happy medium where boilerplate is in Java and more intricate data processing routines are in C++. Makes both worlds simpler. I like C++ better. Those things you mentioned - Java truly sucks at indeed, but you don't always need them. When it comes to IDE, testing, compilation speed, cross platform code and finding talent - Java is way easier than C++.
What the GP is referring to (Misc.Unsafe) is basically going back to manual pointer reads/writes (IE writing more or less C code in Java), the benefit is you get C speed/memory layout/cache-locality but with the downside of writing it in a less suited language (Java). HOWEVER If you DO _allocate_ much then 1TB seems like a great choice since GC pauses are killers w.r.t. latency in comparison to cache issues. A cache…
Can you explain why you would code java this way instead of dropping down to C?
Productivity and safety of Java.
Those unsafe bits are a tiny portion of the overall code.
Can you explain why you would code java this way instead of dropping down to C?
Probably because they started out in Java and don't want a full rewrite once they've gotten this far (might be that they don't have many C++ devs anyhow). I remember working on similar codebases 10 years back when J2ME games were still a thing (J2ME runtimes usually had horrible GC's and some people really went overboard in trying to avoid them) and it wasn't that fun at all (Even if the questDB codebase that is link…
JNI replacement is called Panama it doesn't have anything to do with GraalVM.
GraalVM is the new name of MaximeVM one of the JVM meta-circular JVMs. Other well known ones being JikesRVM and SquawkVM.
What Oracle and others in the Java community are doing is reducing the need to drop down to C with support for value types and more fine grained control over native memory.
Java 16 will have the first preview release of the new low level memory APIs.
Which version of Java are you using? And what is Rnd? The class included with JDK is Random. We rely on escape analysis to elide such object creation for us and it works reasonably well. If something this trivial doesn't work for you, file bug with them. We have had success with that as well.
This ran on Java 11. This isn't so much of an issue for us. We are trying to avoid allocations, even as trivial as those. There are other examples that allocated where they should not, for example this lambda will allocate. () -> System.out.println(1) I lost hope in escape analysis quite frankly. Rnd is something I have written because Java's Random is slow and clunky. https://github.com/questdb/questdb/blob/master/c…
Escape analysis is very JVM implementation specific, but inline classes should finally sort it out.
This is perhaps the most "blub" comment ever written. There are whole classes of bugs that can be caught with more advanced type systems that the Rust type system + borrow checker will never catch. No null pointers and immutability is the bare minimum. If you're interested, check out https://www.youtube.com/watch?v=10gSoVZ5yXY for an example of the types of compile time guarantees can be had.
Does OCaml have dependent types? I thought that was a Coq and Idris (and Haskell, to a lesser extent) thing. I am aware that dependent types exist, but quite frankly I've yet to see a popular/ergonomic language implement them. As much as we'd like absence of null pointers, error types, and immutability to be the norm it is not and it will take decades until it is.
You can also do it in Ada/SPARK, Eiffel DBC, F*, Dafny.
Basically all robust HFT shops are using Java, python is great for prototyping but in the end too slow.
All robust HFT shops? What do you mean by robust and how did you come to that (to me) surprising conclusion?
by working there. I would say sure a lot of the platform is written in C, much of the architecture and front end market-making apps are written in C but when it comes to the algo's and tools to spin up new pricing models in a way that is fast, those tools are written in Java.