Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mispredicted branches can multiply your running times

lemire.me

Post reply on HN
Print view View original

31–40 of 113 posts

Re: Mispredicted branches can multiply your running times

#31

Post by vsareto »

imtringued wrote: post #28
vsareto wrote: post #23

Earlier quoted context omitted.

>It is important to understand your platform all the way down to the CPU, including things like branch prediction and caches if you want to have performant software. Let's not drastically increase job requirements for no good reason. >It's time to learn your platforms, software people. Many of these platforms have undocumented CPU instructions, so until you get a full accounting of that, what's the point? You can't l…

>what's the point? You can't learn the platform fully if they keep that a secret. By that logic we should never learn anything.

You can still learn it and use it and make performance adjustments at low level, just don't claim to know it fully. Subtle differences, kind of like low level code.

Those ops could be better in some cases, and until you know what they do, you can't answer that.

Re: Mispredicted branches can multiply your running times

#32

Post by vsareto »

emsy wrote: post #27
vsareto wrote: post #23

Earlier quoted context omitted.

>It is important to understand your platform all the way down to the CPU, including things like branch prediction and caches if you want to have performant software. Let's not drastically increase job requirements for no good reason. >It's time to learn your platforms, software people. Many of these platforms have undocumented CPU instructions, so until you get a full accounting of that, what's the point? You can't l…

>Let's not drastically increase job requirements for no good reason. If anything, job requirements for programming are way too low. I wouldn't let a mechanic anywhere near my car if I knew s/he didn't understand the basics of an Otto engine. I'm the first to admit I don't know as much about modern CPUs as I should, but I don't live in a fantasy world where I convince myself that I don't need to know it. >Many of thes…

>I wouldn't let a mechanic anywhere near my car if I knew s/he didn't understand the basics of an Otto engine.

That's hilarious because most of the electronics of cars are utter shit and it has nothing to do with how well those developers knew the CPU.

>So we should get the vendors to publish a proper documentation. This doesn't change anything about software developers responsibilities.

Most responsibilities are not concerned with low-level optimizations when javascript in particular typically deals with network connections

Re: Mispredicted branches can multiply your running times

#33

Post by kasey_junk »

vsareto wrote: post #30
naikrovek wrote: post #25

Earlier quoted context omitted.

> Let's not drastically increase job requirements for no good reason. Well, I would say that it's a very good reason, and that learning about branch prediction and caches is not a "drastic" step by any means. Is there any software that you write whose users would not be made happier if the software performed better? Any at all? > Many of these platforms have undocumented CPU instructions You don't need to know the se…

>Is there any software that you write whose users would not be made happier if the software performed better? Any at all? I'd say security is a bigger issue than performance most of the time. And most gains are going to happen within the code itself by, e.g., not writing n^2 when there's a log(n) solution or something similar. Plus we're talking about javascript, and that's likely to be software with network concerns…

> Good algorithm knowledge and practice is the most cost-effective way of writing performant code

I’d love to see that common thought validated because in practice I’ve seen it to not be true at all.

There are lots of cases where the complexity effects of the algorithm are swamped by cache effects. In fact basic foundational assumptions about complexity analysis are dangerously untrue on modern systems.

In my experience in either high throughput or low latency systems algorithmic complexity is never the issue. It’s always cache coherence, CPU prefectching/prediction, lock contention or over copying of data.

Re: Mispredicted branches can multiply your running times

#34

Post by vsareto »

kasey_junk wrote: post #33
vsareto wrote: post #30

Earlier quoted context omitted.

>Is there any software that you write whose users would not be made happier if the software performed better? Any at all? I'd say security is a bigger issue than performance most of the time. And most gains are going to happen within the code itself by, e.g., not writing n^2 when there's a log(n) solution or something similar. Plus we're talking about javascript, and that's likely to be software with network concerns…

> Good algorithm knowledge and practice is the most cost-effective way of writing performant code I’d love to see that common thought validated because in practice I’ve seen it to not be true at all. There are lots of cases where the complexity effects of the algorithm are swamped by cache effects. In fact basic foundational assumptions about complexity analysis are dangerously untrue on modern systems. In my experie…

>I’d love to see that common thought validated because in practice I’ve seen it to not be true at all.

If you have Javascript devs that came out of some boot camp with no knowledge of either, do you teach algorithms 101 or low-level CPU programming 101 first?

I would argue your codebase would benefit from teaching them algorithms first, then the other one.

>In my experience in either high throughput or low latency systems algorithmic complexity is never the issue.

Do you encounter those workloads written in Javascript often?

Re: Mispredicted branches can multiply your running times

#35

Post by DrScientist »

naikrovek wrote: post #15

This is one of those things that is completely lost on someone who has never written in a low level language. I automatically assume JavaScript developers to be completely oblivious to this entire class of software development knowledge. It is important to understand your platform all the way down to the CPU, including things like branch prediction and caches if you want to have performant software. Software has been…

And if your platform is a virtual machine or interpreter running across a number of chip archs?

Perhaps it would be better to simply have appropriate tests and benchmarks to see what's slow on what platform?

Otherwise it's guess work based on incomplete understanding of the many layers below.

Re: Mispredicted branches can multiply your running times

#36

Post by rdc12 »

saagarjha wrote: post #5

> If you are accessing the content of an array, many languages will add “bound checking”: before accessing the array value, there will be a hidden check to see whether the index is valid. If the index is not valid, then an error is generated, otherwise the code proceeds normally. Bound checks are predictable since all accesses should (normally) be valid. Consequently, most processors should be able to predict the out…

At the software level, you don't get a choice about this thou, the CPU is going to speculate about a bounds check regardless, the optimization is it a different abstraction layer.

Pretty much any thing on the market with the compute power of a cellphone, is going to have a feature like this too

Re: Mispredicted branches can multiply your running times

#37

Post by emsy »

vsareto wrote: post #32
emsy wrote: post #27

Earlier quoted context omitted.

>Let's not drastically increase job requirements for no good reason. If anything, job requirements for programming are way too low. I wouldn't let a mechanic anywhere near my car if I knew s/he didn't understand the basics of an Otto engine. I'm the first to admit I don't know as much about modern CPUs as I should, but I don't live in a fantasy world where I convince myself that I don't need to know it. >Many of thes…

>I wouldn't let a mechanic anywhere near my car if I knew s/he didn't understand the basics of an Otto engine. That's hilarious because most of the electronics of cars are utter shit and it has nothing to do with how well those developers knew the CPU. >So we should get the vendors to publish a proper documentation. This doesn't change anything about software developers responsibilities. Most responsibilities are not…

>That's hilarious because most of the electronics

That was an analogy. I wasn't talking about car electronics. I was talking about the lack of professional education in programming.

>Most responsibilities are not concerned with low-level optimizations when javascript in particular typically deals with network connections

I have yet to encounter a javascript program that doesn't run on a CPU. Also, javascript is one of the most versatile and widely used languages. It runs on clients, servers, high end machines and embedded systems. So of course performance matters. If it didn't we wouldn't need WASM, asm.js and countless ridiculous JS engine optimizations. The time "saved" having to learn proper programming by using JS is dwarfed by the time wasted in CPU and end user time.

Re: Mispredicted branches can multiply your running times

#38

Post by ip26 »

DrScientist wrote: post #35
naikrovek wrote: post #15

This is one of those things that is completely lost on someone who has never written in a low level language. I automatically assume JavaScript developers to be completely oblivious to this entire class of software development knowledge. It is important to understand your platform all the way down to the CPU, including things like branch prediction and caches if you want to have performant software. Software has been…

And if your platform is a virtual machine or interpreter running across a number of chip archs? Perhaps it would be better to simply have appropriate tests and benchmarks to see what's slow on what platform? Otherwise it's guess work based on incomplete understanding of the many layers below.

Generically cache and branch predict friendly code is worth most of the benefit. Micro-tuning is small potatoes compared to accidentally wiping the cache on each instruction (e.g. with badly designed objects) or mispredicting a thousand times in a row (e.g. with a poorly designed loop)

Re: Mispredicted branches can multiply your running times

#39

Post by naikrovek »

vsareto wrote: post #30
naikrovek wrote: post #25

Earlier quoted context omitted.

> Let's not drastically increase job requirements for no good reason. Well, I would say that it's a very good reason, and that learning about branch prediction and caches is not a "drastic" step by any means. Is there any software that you write whose users would not be made happier if the software performed better? Any at all? > Many of these platforms have undocumented CPU instructions You don't need to know the se…

>Is there any software that you write whose users would not be made happier if the software performed better? Any at all? I'd say security is a bigger issue than performance most of the time. And most gains are going to happen within the code itself by, e.g., not writing n^2 when there's a log(n) solution or something similar. Plus we're talking about javascript, and that's likely to be software with network concerns…

> I'd say security is a bigger issue than performance most of the time.

you are fucking SKILLED at dodging questions and changing the topic to suit your own needs. you would fail out of a debate class in the first week, though.

I was asking about performance. You can be secure and perform well. I ask again: Is there any software that you write whose users would not be made happier if the software performed better? Any at all?

> >You don't need to know any hidden instructions or secrets to take advantage of the platform.

> You don't know what they do though. Some of those ops could be more advantageous to performance to use in some cases. You can't fully know the platform if there are secret ops.

You don't need to know every transistor of the platform to improve performance. You appear to be taking an all or nothing approach to this, and that's just now how this works. The goal is to improve performance with a reasonable amount of effort, NOT to produce the exact perfect binary. If you want to spend hours upon hours with sandsifter[0] then feel free, but I would consider that to be wasted time in a day to day development position.

> Good algorithm knowledge and practice is the most cost-effective way of writing performant code and is more than likely going to be the lion's share of issues.

No. No, no no no no nonononononno no no no. Does algorithm knowledge contribute? yes. Is it the single greatest way to get performance? Absolutely not. Absolutely not. This is one of those things that everyone "knows" but is actually not true.

[0]: https://github.com/xoreaxeaxeax/sandsifter

Re: Mispredicted branches can multiply your running times

#40

Post by emsy »

vsareto wrote: post #34
kasey_junk wrote: post #33

Earlier quoted context omitted.

> Good algorithm knowledge and practice is the most cost-effective way of writing performant code I’d love to see that common thought validated because in practice I’ve seen it to not be true at all. There are lots of cases where the complexity effects of the algorithm are swamped by cache effects. In fact basic foundational assumptions about complexity analysis are dangerously untrue on modern systems. In my experie…

>I’d love to see that common thought validated because in practice I’ve seen it to not be true at all. If you have Javascript devs that came out of some boot camp with no knowledge of either, do you teach algorithms 101 or low-level CPU programming 101 first? I would argue your codebase would benefit from teaching them algorithms first, then the other one. >In my experience in either high throughput or low latency sy…

>If you have Javascript devs that came out of some boot camp

Do you also go to a doctor that came out of a bootcamp? Is your house built by a constructor that came out of a bootcamp? Would you fly with an aviator that came out of a bootcamp? Would you run banking software made by a developer that came out of a bootcamp?

Post reply on HN