Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Every Byte Matters

fzakaria.com

Post reply on HN
Print view View original

81–90 of 159 posts

Re: Every Byte Matters

#81

Post by jayd16 »

gmueckl wrote: post #78
jayd16 wrote: post #31

Earlier quoted context omitted.

Even more so, it shows that SoA data structure means you can add fields to your 1M monsters with little impact.

This is valid for sequential scanning of the data. The CPU will fill whole cache lines at once with the arrays that do get used and the algorithm touches all the field instances in the array. Now think about random access to single struct instances instead: the CPU loads a cache line worth of data for each field and uses only one element out of the whole cache line. This is much worse than a compact structure represe…

No it's not always better and I didn't mean to imply it was. I was simply saying that the article argues against its title.

In both cases you want to think about locality of the next read and structure the data accordingly.

Re: Every Byte Matters

#82

Post by re-thc »

pron wrote: post #72
re-thc wrote: post #47

Earlier quoted context omitted.

> I'm saying that most developers aren't writing code where layout is a primary contributor to the program's performance. I've heard this theory before. This isn't just about performance and I don't buy it. I've seen too many examples of this is just a temporary solution so it doesn't matter. >3 years later that "temporary solution" was still there and at the heart of many operations yet it's now to hard and too cost…

This has absolutely nothing to do with what I said. I wasn't referring to people who think that program performance doesn't matter (although I'm sure there are many of those) but to people working on code that either doesn't impact the overall program's performance much or it does but not due to layout. The number of developers working on code where layout is a major contributor to performance is relatively low, and…

> but to people working on code that either doesn't impact the overall program's performance much or it does but not due to layout

And that's the problem. Who decides that? How do you know and that's my problem with it. Things always change. It's always temporary, not in the hot path, doesn't matter etc until it does.

So what is considered "doesn't impact" often comes back to bite.

Re: Every Byte Matters

#83

Post by Altern4tiveAcc »

tsimionescu wrote: post #79
Altern4tiveAcc wrote: post #67

Earlier quoted context omitted.

> SoA is weak if you are adding/removing monsters more often than accessing a single "hot" field. Why is that? Genuinely curious. Does "weak" mean that it performs worse than AoS, or that the gains aren't as significant versus AoS?

It's because removing a monster with 20 fields from an SoA structure means resizing 20 arrays. Removing the same monster from an AoS array involves resizing a single array, which you're going to process in a very cache friendly way.

Assuming ordering isn't a concern, can't you just have a field called "removed" and skip those when iterating?

Or swap it with the last monster, and keeping an index for the last monster alive.

Re: Every Byte Matters

#85

Post by theandrewbailey »

fp64 wrote: post #61
the__alchemist wrote: post #58

Earlier quoted context omitted.

Odin is heavily inspired by the lang he or she is referring to!

A sibling comment also mentioned Jai. Not sure what I am missing that the original post was explicitly referring to Jai, some inside joke maybe? I am sorry, I only know Odin. Jai is this cult on reddit/discord, right? You get access if you socialize enough or something? Not my thing. Not for a language.

(original poster here)

I was just throwing out an idea. I had no idea there were already implementations! Because, to my knowledge, conventional popular languages like C/C++/C#/Java/JS/Python don't do that, and automatically doing that (under certain conditions) feels like an easy performance win.

Re: Every Byte Matters

#86

Post by marcosdumay »

Altern4tiveAcc wrote: post #83
tsimionescu wrote: post #79

Earlier quoted context omitted.

It's because removing a monster with 20 fields from an SoA structure means resizing 20 arrays. Removing the same monster from an AoS array involves resizing a single array, which you're going to process in a very cache friendly way.

Assuming ordering isn't a concern, can't you just have a field called "removed" and skip those when iterating? Or swap it with the last monster, and keeping an index for the last monster alive.

Then you have to read the "removed" field on every field read on every operation.

SoA is only useful when you don't read multiple fields for most operations.

Re: Every Byte Matters

#87

Post by the__alchemist »

fp64 wrote: post #61
the__alchemist wrote: post #58

Earlier quoted context omitted.

Odin is heavily inspired by the lang he or she is referring to!

A sibling comment also mentioned Jai. Not sure what I am missing that the original post was explicitly referring to Jai, some inside joke maybe? I am sorry, I only know Odin. Jai is this cult on reddit/discord, right? You get access if you socialize enough or something? Not my thing. Not for a language.

Ah. So, the context (Which I read too far into evidently): 1: One of Jai's initial primary marketing points was to address exactly this: SoA performance with AoS ergonomics. 2: Odin is (or was initially) inspired by Jai.

Re: Every Byte Matters

#88

Post by pron »

re-thc wrote: post #82
pron wrote: post #72

Earlier quoted context omitted.

This has absolutely nothing to do with what I said. I wasn't referring to people who think that program performance doesn't matter (although I'm sure there are many of those) but to people working on code that either doesn't impact the overall program's performance much or it does but not due to layout. The number of developers working on code where layout is a major contributor to performance is relatively low, and…

> but to people working on code that either doesn't impact the overall program's performance much or it does but not due to layout And that's the problem. Who decides that? How do you know and that's my problem with it. Things always change. It's always temporary, not in the hot path, doesn't matter etc until it does. So what is considered "doesn't impact" often comes back to bite.

That is why profiling is the only way to good performance. It's what lets you know what matters, and it's the only thing that does or can. I've been doing low-level (as well as high level) programming for more than 25 years, and I don't know in advance what is more efficient than what. An operation that was inefficient in the program I wrote yesterday under high contention or bad branch prediciton could be efficient in the program I'll write tomorrow. I can only know that if I profile my specific program (and when writing code for different architectures, I need to profile my program on all of them, because what's efficient on x86-64 may be inefficient on Aarch64 or vice-versa). The days we could tell that something is efficient or not, except for the obvious cases, are gone. Computers, at both the hardware and software infrastructure layers, don't work like that anymore.

If your profile shows you a hot path that's responsible for 90% of the time your program spends, any second optimising anything outside of it harms your performance, as it's a second spent on low ROI instead of high ROI.

Re: Every Byte Matters

#89

Post by WhitneyLand »

pron wrote: post #39
AlotOfReading wrote: post #27

Earlier quoted context omitted.

In general, the tradeoff is between optimisations that help large programs vs optimisations that help small programs. Do you have concrete examples of large scale Java programs that are significantly more performant than comparable programs in native languages like C++? My understanding was that this dynamic hadn't fundamentally changed much since the 2010s, when Java was able to occasionally edge out a win in 1-2 be…

> Do you have concrete examples of large scale Java programs that are significantly more performant than comparable programs in native languages like C++? Yes. I was working in a place that made large sensor-fusion applications, air-traffic control applications, and logistical planning, each in the 2-8MLOC range. Over time, we ported all of them from C++ to Java because C++'s performance overheads were too annoying t…

”they ported a small-to-medium service from Java to Rust. The result was such a huge performance drop that it wouldn't meet their minimum requirements”

That result would say less about performance of languages than it would about competency of developers with a language.

I just don’t buy that a task could be assigned to two teams with comparable expertise and domain knowledge in Rust and Java, and have the Rust result be at a “huge” performance deficit.

No, don’t believe that was an apples to apples comparison.

Re: Every Byte Matters

#90

Post by manoDev »

Tip: to get LN cache sizes on mac, the commmand is

    $ sysctl -a | grep "l.*cachesize" | gnumfmt --field=2 --to=si
    hw.perflevel1.l1icachesize:   132k
    hw.perflevel1.l1dcachesize:   66k
    hw.perflevel1.l2cachesize:    4,2M
    hw.perflevel0.l1icachesize:   197k
    hw.perflevel0.l1dcachesize:   132k
    hw.perflevel0.l2cachesize:      13M
    hw.l1icachesize:   132k
    hw.l1dcachesize:   66k
    hw.l2cachesize:    4,2M
And the equivalent to LEVEL1_DCACHE_LINESIZE is

    $ sysctl -a | grep hw.cachelinesize
    hw.cachelinesize: 128
Post reply on HN