Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Every Byte Matters

fzakaria.com

Post reply on HN
Print view View original

141–150 of 159 posts

Re: Every Byte Matters

#141

Post by SleepyMyroslav »

pron wrote: post #138
SleepyMyroslav wrote: post #137

Earlier quoted context omitted.

I am not GP poster. I find pron points interesting even if I work in the gamedev on game engines. If you don't mind I will try to explain how I see them interesting. Since I have not worked on Rust systems I will stick to C++. Note his example elsewhere in this discussion of 2 projects done at same time in Java and Rust and the complaint that Rust system used too many locks. This can happen in C++ too. But why it doe…

> The performance scales between them are many orders of magnitude. Imho it does not mean that Java anywhere near the best of what C++ can do. I don't think you're aware of where Java is today. Here's a recent talk about some of the issues we're working on now: https://youtu.be/J4O5h3xpIY8 I said that in the past the people who believed Java can't match or exceed C++'s performance were typically those with a lot of l…

as for my experience, yep I do not have Java experience and a long list of C++ projects.

> what is exactly that you'd think makes Java harder to compile in an optimised way than C++?

In games C++ is doing some simulations and data delivery for GPU. Code that does work on GPU is not mixed with rest of C++ code. So invoking Cuda (or the likes) in the middle of computation is a cheat code that Java does not have. Simulations on the CPU need to be efficiently parallel ( think 12 hardware threads for last gen or 4-6 threads for smaller platforms) and most likely specialized for hardware SIMD ( think AVX2 for last gen or SSE2 like for smaller platforms). To wrangle multi GB data efficiently a lot of compression/decompression and data structures are needed. Does Java still has overhead per class instance? It might force designs with arrays of primitive data types that are more verbose.

Add there per platform I/O and everything. It means that games force people to unlearn everything that language ever thought about standard I/O. Even more about being cross platform. In C++ it means something completely different. In C++ you can't trust language implementation vendor with anything. From your comment I assume that Java teams rely on language implementation in lots of ways. In C++ being efficient means do it yourself. How efficient our memory allocation is? Answer can only be per engine/project. There is no 'average' because 'vendor provided' is the bottom of the barrel quality. No one is improving vendor provided exactly because no one is expected to use it.

In short there are hard to compare many different C++. I can't see them compare to each other much less to other programming languages like Java. This might be not the answer you wanted but that's all I have.

Re: Every Byte Matters

#142

Post by jayd16 »

rao-v wrote: post #122
jayd16 wrote: post #57

Earlier quoted context omitted.

It doesn't quite make sense to keep object identity at the language level. Inherently the data in the arrays cannot be the same memory of the data in the objects fields. To get the speed up, you can't just abstract it as an access pattern because it's tied to the specific way the memory is laid out. If you were trying to make some kind of collection type that could be queried by both row and column, you would need to…

I was thinking about just asking the compiler to do the expensive reshuffle when I need it to, but you could go further and expect the compiler to figure out the likely access pattern and spend the transformation budget. Heck memory is cheap (fine was cheap) give me a data structure that amortizes writes cleverly by maintaining both SoA and AoS at the same time

Well the whole point is that memory (cache in this case) is not cheap at all. We have very little and the point is to do as little loading of what you don't care about as possible to keep the cache full of only exactly what is needed for the task.

How do you imagine it's possible to write to every SoA and every AoS and have that as cheap as only the first step?

Re: Every Byte Matters

#143

Post by pron »

SleepyMyroslav wrote: post #141
pron wrote: post #138

Earlier quoted context omitted.

> The performance scales between them are many orders of magnitude. Imho it does not mean that Java anywhere near the best of what C++ can do. I don't think you're aware of where Java is today. Here's a recent talk about some of the issues we're working on now: https://youtu.be/J4O5h3xpIY8 I said that in the past the people who believed Java can't match or exceed C++'s performance were typically those with a lot of l…

as for my experience, yep I do not have Java experience and a long list of C++ projects. > what is exactly that you'd think makes Java harder to compile in an optimised way than C++? In games C++ is doing some simulations and data delivery for GPU. Code that does work on GPU is not mixed with rest of C++ code. So invoking Cuda (or the likes) in the middle of computation is a cheat code that Java does not have. Simula…

> So invoking Cuda (or the likes) in the middle of computation is a cheat code that Java does not have.

It does (and has since JDK 22). But what we're working on now is JIT-compiling Java code to CUDA (not arbitrary code, but certainly code that's suitable for a kernel): https://openjdk.org/projects/babylon/articles/hat-matmul/hat...

> and most likely specialized for hardware SIMD ( think AVX2 for last gen or SSE2 like for smaller platforms)

Yep, we've had good SIMD support for a few years now. (https://javapro.io/2026/04/09/java-vector-api-faster-vector-...)

> Does Java still has overhead per class instance? It might force designs with arrays of primitive data types that are more verbose.

That is the last area where Java is still behind but the work on arrays-of-structs (with no headers) is nearly complete. A first release of that is imminent.

> In C++ being efficient means do it yourself

Right, and that's precisely what I meant about low-level languages being optimised for control and not performance. You could do things at such a low level in Java, but the main problem is not the performance but that it's just less convenient than in C++.

Anyway, aside from some outdated (or soon-to-be-outdated) things, what you pointed out is mostly about lack of convenient direct low-level control rather than general performance, and that is exactly when low-level languages can be a better fit.

Re: Every Byte Matters

#144

Post by Rendello »

gmueckl wrote: post #78
jayd16 wrote: post #31

Earlier quoted context omitted.

Even more so, it shows that SoA data structure means you can add fields to your 1M monsters with little impact.

This is valid for sequential scanning of the data. The CPU will fill whole cache lines at once with the arrays that do get used and the algorithm touches all the field instances in the array. Now think about random access to single struct instances instead: the CPU loads a cache line worth of data for each field and uses only one element out of the whole cache line. This is much worse than a compact structure represe…

> SoA is not universally better.

This is an important part of Data-oriented Design: the representation of the data should be pragmatically tied to its access patterns, not dogma.

Richard Fabian's DoD book gives the example that (x,y,z) is almost always better as a classic array-of-structs rather than a struct-of-arrays, because if you're accessing one dimension, you probably are want to process the other two dimensions at the same time:

https://www.dataorienteddesign.com/dodbook/node9.html#SECTIO...

Re: Every Byte Matters

#146

Post by SJMG »

pron wrote: post #135
imtringued wrote: post #131

Earlier quoted context omitted.

>HotSpot uses much more RAM by design as there are inefficiencies caused by using too little of it. Ah yes, the swapping induced by IntelliJ overflowing my system RAM is supposed to reduce the inefficiencies of using too little memory. Great... Thanks pron, you've fully bought into all the JVM kool-aid talking points without ever trying to question them. One of the reasons I upgraded to 32 GB RAM in 2019 was to run a…

> Ah yes, the swapping induced by IntelliJ overflowing my system RAM is supposed to reduce the inefficiencies of using too little memory. Great... That's like me saying, oh great, so the swapping introduced by MS Word or Outlook shows just how efficient C++ is... > Thanks pron, you've fully bought into all the JVM kool-aid talking points without ever trying to question them. Oh I didn't just "buy" them. As a low-leve…

This looks to be the end of the conversation now. Just wanted to drop in and thank you for your time commenting, pron.

The common discourse is that "XYZ language is close to the metal and therefore Blazing Fast (tm)" people become tribalistic and forgot that this there are engineering considerations and trade-offs all the way down. I appreciate you making the argument for the JVM delivering performant code when a budget matters.

Re: Every Byte Matters

#147

Post by nitwit005 »

kakacik wrote: post #10
noelwelsh wrote: post #5

The JVM is currently pretty bad for memory allocation. Every object (i.e. not a primitive) has a header that IIRC is 12 bytes. But there is good news in JVM land: this will be reduced to 8 bytes in the next JVM release, and Project Valhalla will give the tools to do away with headers entirely in some cases. Project Valhalla also has tools to manage off-heap memory, which is important in many cases. The JVM is an odd…

Most of real world use of Java platform has next to 0 concerns like those. Some more niche use case may benefit, good, but overall success map isn't changing anytime soon. Reasons for its long term success lie elsewhere.

While this is true, it is true because the applications where it might be a concern avoid using Java.

Re: Every Byte Matters

#148

Post by pron »

setr wrote: post #140
pron wrote: post #134

Earlier quoted context omitted.

> Isn’t that only true though specifically at 100% CPU utilization? No. Because any RAM access requires CPU, using up any CPU effectively captures some ability to use RAM. > I don’t see how you can map X% CPU utilization to Y% RAM capture. You're right that there isn't a fixed formula, but the most efficient balance can have a narrow range, because CPU and RAM are typically sold as a package with a rather narrow RAM/…

> I covered that in my talk, which will be eventually published on YouTube. Any idea how I get myself notified once it’s up? Or a YT account to poll

https://www.youtube.com/java

Don't confuse it with the interview about my talk, which is already up, but doesn't cover any of the important details.

Re: Every Byte Matters

#149

Post by mabster »

imtringued wrote: post #129
pron wrote: post #119

Earlier quoted context omitted.

> the performance of Java has to be the best-kept secret in the industry Is it, though? It's the first language of choice for a large number, if not most performance-critical applications. > Because you are the only person I've ever heard making such claims seriously. Your sources must be very limited, then, because in serious compiler and runtime design and memory management circles this is quite common. There is a…

>I wouldn't say HPC and video game engines are "traditional performance critical work". Not because they're not performance critical, but because the range of performance critical programs is far larger - think bank card transaction processing; think mobile phone routing, and there are many more examples (also, AAA video game engines are indeed very traditional in their design and tech choices, but their performance-…

Low level CPU-related optimisation is absolutely still a thing. The GPU is always filled to the brim trying to get as much quality out of a graphics frame so a lot gets offloaded to the CPU. When I was doing this I was doing a lot of low-level CPU optimisation. GPU optimisation was usually more about transform process topology but there was plenty of low-level work to do there too.

Games are both high throughput AND low-latency and C++ is still king there

Re: Every Byte Matters

#150

Post by rao-v »

jayd16 wrote: post #142
rao-v wrote: post #122

Earlier quoted context omitted.

I was thinking about just asking the compiler to do the expensive reshuffle when I need it to, but you could go further and expect the compiler to figure out the likely access pattern and spend the transformation budget. Heck memory is cheap (fine was cheap) give me a data structure that amortizes writes cleverly by maintaining both SoA and AoS at the same time

Well the whole point is that memory (cache in this case) is not cheap at all. We have very little and the point is to do as little loading of what you don't care about as possible to keep the cache full of only exactly what is needed for the task. How do you imagine it's possible to write to every SoA and every AoS and have that as cheap as only the first step?

Many operations are map reduce where writes are low volume compared to reads
Post reply on HN