Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Every Byte Matters

fzakaria.com

Post reply on HN
Print view View original

71–80 of 159 posts

Re: Every Byte Matters

#71

Post by pron »

tealpod wrote: post #64
pron wrote: post #39

Earlier quoted context omitted.

> Do you have concrete examples of large scale Java programs that are significantly more performant than comparable programs in native languages like C++? Yes. I was working in a place that made large sensor-fusion applications, air-traffic control applications, and logistical planning, each in the 2-8MLOC range. Over time, we ported all of them from C++ to Java because C++'s performance overheads were too annoying t…

We compiled one of our Java app to native binary using GraalVM (for encyption and secret managment needs). Side effect is the Java native binary performance is excellent, app startup time also significantly less compared to JVM version. I am not sure how it compares with C++, Rust and Zig, but we made a benchmark with a similar Go binary, Java native version performance (load tests) is similar to Go binary. Only RAM…

The RAM difference is primarily because both Native Image (what you call Graal VM) and Go use much simpler and less efficient memory management techniques. HotSpot uses much more RAM by design as there are inefficiencies caused by using too little of it. Memory management - and especially very sophisticated approaches that are only used by the best resourced teams - is an especially misunderstood aspect.

I gave a talk on the subject that I hope will be published soon, and while I can't reproduce it here, let me give an example that offers some basic intuition. Imagine needing to do some computation in two ways on a machine with 1GB of free RAM. You could run for 10s, taking up 100% CPU and consuming 80MB of RAM, or for 9s, taking up 100% CPU and consuming 800MB of RAM. The second is more efficient, despite taking up 10x more RAM and saving "only" 10% of CPU, regardless of the relative cost of RAM and CPU. This is because taking up 100% of the CPU effectively captures 100% of RAM (as no other program can use it), so both programs capture the entire 1GB only the second one captures it for a second less. This scales to non extreme situations because accessing RAM requires CPU, so using CPU means capturing RAM whether you use it or not. So HotSpot uses it if it can use it to balance the CPU utilisation.

In some situations it may not matter, and I assume that if Native Image and Go work just as well for you, then the workload isn't very high, but under high workloads, this can matter a lot.

Re: Every Byte Matters

#72

Post by pron »

re-thc wrote: post #47
pron wrote: post #45

Earlier quoted context omitted.

I'm saying that most developers aren't writing code where layout is a primary contributor to the program's performance. Even in performance-sensitive applications, only a minority of the team are working on the hot spots. And speaking about costs, knowing what to optimise is the key to software performance. Improving the performance of an operation by 10000x will improve the performance of your program by less than 1…

> I'm saying that most developers aren't writing code where layout is a primary contributor to the program's performance. I've heard this theory before. This isn't just about performance and I don't buy it. I've seen too many examples of this is just a temporary solution so it doesn't matter. >3 years later that "temporary solution" was still there and at the heart of many operations yet it's now to hard and too cost…

This has absolutely nothing to do with what I said. I wasn't referring to people who think that program performance doesn't matter (although I'm sure there are many of those) but to people working on code that either doesn't impact the overall program's performance much or it does but not due to layout. The number of developers working on code where layout is a major contributor to performance is relatively low, and this includes people working on programs where layout does impact performance significantly (because even in such a program, that particular hot path is not touched by every developer).

Re: Every Byte Matters

#73

Post by re-thc »

gf000 wrote: post #62
re-thc wrote: post #47

Earlier quoted context omitted.

> I'm saying that most developers aren't writing code where layout is a primary contributor to the program's performance. I've heard this theory before. This isn't just about performance and I don't buy it. I've seen too many examples of this is just a temporary solution so it doesn't matter. >3 years later that "temporary solution" was still there and at the heart of many operations yet it's now to hard and too cost…

Then what is it that you are saying? That I should use JMH to determine the best layout for my helper class that will be initialized 3 times? Like most of the software (by line of code) is boring plumbing from one service to another with some dumb business logic sprinkled in. Something like a single config option for your database driver matters orderS of magnitude more in many types of applications. It's much more n…

Consider the cost of every field, of every action.

Understand the language, the memory model, etc. Don't do "it works on my machine". Understand the architecture, layout, implications etc.

E.g. if you need an int and not a long you should clearly use an int. Wait until you do this every time and things blow up and it's too "hard" to change.

It's called be aware of your actions. Take responsibility of what you do.

> It's much more niche to work on stuff where such changes actually matter,

Not true and that's why there's so much wastage.

A lot of things matter. I've seen more times than the other way that simple awareness and changes can pay for my salary, e.g. not updating to newer EC2 instances when they get released in AWS. Even in a mid size company that was hundreds to thousands in savings.

I've seen CI/CD pipelines where the developers never considered caching and it takes hours to run. It's not free. When every PR and update (hundreds a day) triggers a run it's a cost and a cost not just on machines but developer time waiting.

I can list a lot more examples and everyone in the chain can contribute.

Re: Every Byte Matters

#74

Post by pron »

re-thc wrote: post #73
gf000 wrote: post #62

Earlier quoted context omitted.

Then what is it that you are saying? That I should use JMH to determine the best layout for my helper class that will be initialized 3 times? Like most of the software (by line of code) is boring plumbing from one service to another with some dumb business logic sprinkled in. Something like a single config option for your database driver matters orderS of magnitude more in many types of applications. It's much more n…

Consider the cost of every field, of every action. Understand the language, the memory model, etc. Don't do "it works on my machine". Understand the architecture, layout, implications etc. E.g. if you need an int and not a long you should clearly use an int. Wait until you do this every time and things blow up and it's too "hard" to change. It's called be aware of your actions. Take responsibility of what you do. > I…

> Consider the cost of every field, of every action.

This runs counter to most modern software performance principles. Thanks to modern hardware optimisations (cache hierarchy, ILP, branch prediction), modern compiler optimisations (aggressive inlining that leads to a much wider view), and increased concurrency, the notion of some action having a cost lost most meaning about 20 years ago, and increasingly since. Because how fast some action is now depends on a much broader context of what else is going on in the program (and the machine), action X can be faster than Y in one program and the same or slower than Y in another.

Because it's nearly impossible to generalise (and so what was true in your previous program may not be true in your current one unless they're nearly identical), the advice is to first profile your program so that you know how fast or slow different parts are in the context of your particular program and then to focus the optimisation efforts on the hot paths in your program. Otherwise, you may end up spending effort where it makes no difference, and this comes at the cost of optimising what matters, overall harming performance.

Taking responsibility means being smart about directing your resources to where they can have the most impact.

Re: Every Byte Matters

#75

Post by readthenotes1 »

"In that time, you get used to huge classes. New functionality? Just add a new method and field to the class"

I guess this is one reason why object-orientation has such a bad reputation.

I once worked at a bank where the OO mentor had taught people that the only object they needed was "Tape" and have them replicate the structure of data on the old spooled tape reels.

The struct of arrays reminds me of this optimization.

Re: Every Byte Matters

#77

Post by gmueckl »

jayd16 wrote: post #70
Altern4tiveAcc wrote: post #67

Earlier quoted context omitted.

> SoA is weak if you are adding/removing monsters more often than accessing a single "hot" field. Why is that? Genuinely curious. Does "weak" mean that it performs worse than AoS, or that the gains aren't as significant versus AoS?

Presumably they're referring to resizing the arrays.

Array resizing is avoidable with an embedded free list if ordering is of no concern.

Re: Every Byte Matters

#78

Post by gmueckl »

jayd16 wrote: post #31
moring wrote: post #30

The article shows nicely how "every byte matters" is false. First, it starts off by talking about the cost of a new field, when the actual topic is array-of-structs vs. struct-of-arrays. Then, this: > How much of an impact can this have? > Reading is:alive (1 byte) Across 1M Monsters You aren't reading one byte here, you are reading 1M bytes! Of course, optimizing the access to 1M bytes is something to consider. Opti…

Even more so, it shows that SoA data structure means you can add fields to your 1M monsters with little impact.

This is valid for sequential scanning of the data. The CPU will fill whole cache lines at once with the arrays that do get used and the algorithm touches all the field instances in the array.

Now think about random access to single struct instances instead: the CPU loads a cache line worth of data for each field and uses only one element out of the whole cache line. This is much worse than a compact structure representation of the same data.

SoA is not universally better.

Re: Every Byte Matters

#79

Post by tsimionescu »

Altern4tiveAcc wrote: post #67
notatyrannosaur wrote: post #46

Earlier quoted context omitted.

> you can add fields to your 1M monsters with little impact. Great for this access pattern, but I wouldn't make a general statement like that. This is the same thing as row-oriented vs column-oriented databases, OLTP vs OLAP. SoA is weak if you are adding/removing monsters more often than accessing a single "hot" field.

> SoA is weak if you are adding/removing monsters more often than accessing a single "hot" field. Why is that? Genuinely curious. Does "weak" mean that it performs worse than AoS, or that the gains aren't as significant versus AoS?

It's because removing a monster with 20 fields from an SoA structure means resizing 20 arrays. Removing the same monster from an AoS array involves resizing a single array, which you're going to process in a very cache friendly way.

Re: Every Byte Matters

#80

Post by ChrisMarshallNY »

Obscurity4340 wrote: post #55
ChrisMarshallNY wrote: post #52

I started off with Machine Code, on a device with 256 bytes (not KB) of RAM. That was 256 bytes, to install the executable, reserve the stack, and set up the heap. We often used bit (not byte) fields, to convey information. Made life challenging. However, being able to be sloppy has its definite advantages. It takes a long time to design highly-optimized stuff. If just declaring a couple of new properties takes thirt…

How do you deal with all the daemons and automatic crap that does this on Mac? Isnt it all reinforced by SIP?

I think all operating systems have these.

In this one case, allocating a MapView via storyboard, caused some kind of cascading strong reference stuff.

Simply allocating it programmatically, fixed it.

Took awhile to get there, though.

Post reply on HN