> I’ve done performance-engineering for decades in Java, C++, and C for both data analytics and supercomputing/HPC. Java performs significantly worse than C++ in all cases without exception.
I've done similar work (not supercomputing/HPC, but yes for soft and hard realtime software, including safety-critical software) and I couldn't disagree more. Of course, we didn't get to write every program in both Java and C++, but the main question was how much effort it took to achieve the required performance. Over multiple projects it was clear that hitting the performance targets was, on the whole, significantly easier in Java.
> This is the result you should expect from first principles; something has gone horribly wrong with your software optimization if Java is faster than C++ or even Rust.
Strong disagreement here, but we need to be specific about what we mean when we say performance.
It is undoubtedly true that for every Java program there exists a C++ program with the same performance, and the proof is simple: every Java program is a C++ program with the classes being input. But that C++ program is close to 2MLOC long. The same could also be said about a C++ program vs. an Assembly program, as every C++ program could be written as an Assembly program.
But when I talk about performance, I refer to what I think most programmers care about when it comes to performance. Not how fast can a program hypothetically be given enough effort and expertise, but how fast can my program be in my budget.
Both speculative compiler optimisations and memory management optimisations are simply not an option for low level languages due to their constraints, and they are very powerful global optimisations. Given a lot of expertise and effort (that must continue throughout the software's lifetime, and often increases as it evolves) you can work around these limitations, but Java was designed so that you can benefit from them, which means more performance per unit of effort.
In large programs more general constructs (e.g. dynamic dispatch) and patterns (concurrency, great variance in object lifetime) grow in prevalence, and low level languages require more effort and discipline to work around their shortcomings in these areas. Optimising JITs that allow aggressive speculative optimisations and moving collectors were invented and adopted to address these shortcomings. You could claim that the advanced mechanisms that were developed to address C++'s performance issues have failed to achieve their goal, although it won't be easy and much of it comes down to empirical questions of which patterns arise more or less frequently in software, but given that this is what these mechanisms were at least intended to achieve, you certainly can't claim that they fail to do so "from first principles". Some compilation optimisations need speculation; some memory management optimisations need moving pointers. Not having these optimisations available in a program you can write without a lot of special effort cannot make it faster "from first principles".
So no, I don't believe at all that something has to go wrong for a Java program to be faster than a C++ program given a certain budget for the program. Indeed, in larger, more complex programs, I believe the very opposite is true. In most situations, if you get the same performance in C++ as you do in Java, then something has gone terribly wrong with your Java program.
As someone who's worked on a pretty famous JVM feature (virtual threads), I can tell you that we and the designers of low-level languages consciously make different performance tradeoffs because we optimise for different programs and people, and have different preferences when it comes to average case vs. worst case, but there is no universal dominance in performance to either one of these approaches over the other.
One obvious example was our decision to remove Unsafe from Java. Some Java developers voiced opposition, citing a program speed competition (the "one-billion-row challenge" [1]) where Unsafe improved the performance of an entry (which was later cloned and tweaked by others) by 25%. But we saw it as further motivation for the decision. Among over a dozen performance experts who submitted entries, only one was able to write a program efficient enough for Unsafe to make a big difference, and the variance in the results even among the top 20 or so entries was larger than Unsafe's improvement. By removing Unsafe, we would harm that one expert's program, but it would allow us to perform more aggressive constant-folding optimisations that would result in much greater performance improvements over the entire ecosystem. Even from a design philosophy perspective alone, this removal of control to the detriment of some programs "for the greater good" of performance over the entire ecosystem is almost unthinkable in low level languages, because control is what they're for. Did that decision make Java a faster or a slower language? That depends on how you look at performance.
[1]: https://github.com/gunnarmorling/1brc