Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Print(“lol”) doubled the speed of my Go function

medium.com

Post reply on HN
Print view View original

71–80 of 130 posts

Re: Print(“lol”) doubled the speed of my Go function

#71

Post by Galanwe »

mcv wrote: post #51
distcs wrote: post #40

Earlier quoted context omitted.

I don't think I'd call "branch prediction" as "low level esoterica". It is a basic fact about how CPUs are implemented since many decades now. I learnt these things in my university coursework. Any module on CPU or computer system architecture is going to teach you all this stuff. But I'm sure you could learn these things from books on this topic too.

I didn't, and frankly, half of the articles I read about it make me think branch prediction is a bug. I mean, I know it's meant to improve performance, which is great, but it has to make assumptions about what's going to happen before it knows it, and those assumptions are going to be wrong. How wrong? How can we con it into making better assumptions? Suddenly programming becomes about second guessing the compiler. A…

> it has to make assumptions about what's going to happen before it knows it, and those assumptions are going to be wrong. How wrong

Speculative/optimistic techniques are not limited to branch prediction, you encounter various forms of it pretty much everywhere, without knowing it.

* Your hard drives have a read ahead buffer, fetching in advance future data, just because most reads are immediately followed by an other one for the data just after.

* Your CPU instructions are pre fetched, because most instructions are _not_ jumps, so you will 99% of the time just execute the instruction right after it.

* Instruction reordering where your compiler will decide that future instructions not affected by previous instructions could be run in advance.

* Overall any kind of caching is some form of optimistic technique. You are preparing future results.

If you think about it, optimistic/speculative techniques are ubiquitous, and used even at _high_ abstraction levels.

The famous python mantra of "better ask for forgiveness than permission" embodies that spirit. It encourages a coding style of "try: do() except: nope()" rather than "if check: do()".

Standing "against optimistic/speculative techniques" is standing against transactions, rollbacks, caches. It's just not a viable line of thinking IMHO.

Re: Print(“lol”) doubled the speed of my Go function

#72

Post by Zinu »

mcv wrote: post #53
bakul wrote: post #4

Processor "optimizations" can produce surprising effects. The problem is these optimizations are not programmatically accessible to C (or most modern programming languages) given their simple memory model. Deterministic performance is not easy to obtain. My view is to not bother with such tricks unless absolutely necessary (and be prepared that your changes may actually pessimize performance on a future processor or…

I would agree, but it's hard to argue with a factor 2 performance boost. But these kind of tricks feel like we need to con the compiler into optimising this correctly, which is of course ridiculous. What we probably need instead is if-statements that we can tell what's most likely the correct prediction. Something like: if v > maxV predict true maxV = v continue

It's only factor 2 with an increasing array though. At which point you can just take the last element, that's way faster.

So really you end up having to make assumptions about the input to get the performance boost.

Re: Print(“lol”) doubled the speed of my Go function

#73

Post by anonymoushn »

James_K wrote: post #59
tylerhou wrote: post #25

Earlier quoted context omitted.

"Conditional move is slow" is not the right takeaway. (In fact, conditional move can be much /faster/ than a branch in certain circumstances (e.g. hard-to-predict branches, like binary search).) The reason why this code is slow is because the conditional move is on the critical path for resolving a loop-carried data dependency. In other words, to execute the assignment `maxV = (v if (v > maxV) else maxV)`, the CPU ha…

How exactly does that change anything? It's being run on an increasing array, so the data is assigned every loop. Surely the comparison creates a dependency on the value of the variable which was assigned in the last loop.

For branch instructions but not for conditional move instructions, the processor will happily execute the next instruction before the values are actually ready, then invalidate the work it has done later if it turns out retrospectively to have been wrong. This is called branch prediction.

Re: Print(“lol”) doubled the speed of my Go function

#74

Post by ColinWright »

vsnf wrote: post #38

Kind of tangential, but who are these people who are so comfortable with disassembling a high level language binary, reading assembly, and then making statements about branch prediction and other such low level esoterica? I've only ever meet people like that maybe two or thee times in my career, and yet it seems like every other blog post I read in certain language circles everyone is some kind of ASM and Reverse Eng…

My first machine was a TRS-80. My first large program was a compiler from TRS-80 BASIC to Z80. I subsequently disassembled the ROM in the machine to figure out how things worked.

These skills stay with you, and if you read articles like this then you can keep broadly up-to-date with the insanity that is current CPUs. Things like pipe-lines, branch prediction, and different levels of cacheing are optimisations that you can acquire as you go.

If you're an auto-didact web developer then you never have the opportunity to learn these skills, or the need to do so.

I know a lot of people who are comfortable with doing this, but in my case it's a generational thing. If you want to do it then you can. It's not hard, it's just a different skill from those you already have, though sufficiently related that you wouldn't be starting from scratch.

But starting with modern CPUs can be hard. Learning the basics from older, simpler CPUs can help. Doing some kind of embedded programming might be the way to get started, or working on an emulator.

As always, YMMV.

Re: Print(“lol”) doubled the speed of my Go function

#75

Post by _joel »

gv83 wrote: post #68
keyle wrote: post #47

Earlier quoted context omitted.

They're a dying breed. We're forgetting how to look under the hood and understand "why something works". Case in point, I'm slowly being replaced by Salesforce muppets for all my projects at work. They're little code monkeys with amazon ebook type knowledge, projects cost 20x more and I look like the mad scientist for speaking the truth. The products are worse in every possible metrics, I'm not crazy. The politics at…

> We're forgetting how to look under the hood and understand "why something works". But under the hood there is a hood. And under that hood there is another hood. And under that hood there is a brand new car you don't know how to open the hood, and so on. I cannot devote my life to know everything or I won't be able to provide for my family.

"It's hoods, all the way down..."

Re: Print(“lol”) doubled the speed of my Go function

#76

Post by rajamaka »

gv83 wrote: post #68
keyle wrote: post #47

Earlier quoted context omitted.

They're a dying breed. We're forgetting how to look under the hood and understand "why something works". Case in point, I'm slowly being replaced by Salesforce muppets for all my projects at work. They're little code monkeys with amazon ebook type knowledge, projects cost 20x more and I look like the mad scientist for speaking the truth. The products are worse in every possible metrics, I'm not crazy. The politics at…

> We're forgetting how to look under the hood and understand "why something works". But under the hood there is a hood. And under that hood there is another hood. And under that hood there is a brand new car you don't know how to open the hood, and so on. I cannot devote my life to know everything or I won't be able to provide for my family.

I'm interested in what's under the hood but after working full time scrubbing the bonnet, my motivation is mostly gone.

Re: Print(“lol”) doubled the speed of my Go function

#77

Post by lordlimecat »

vsnf wrote: post #38

Kind of tangential, but who are these people who are so comfortable with disassembling a high level language binary, reading assembly, and then making statements about branch prediction and other such low level esoterica? I've only ever meet people like that maybe two or thee times in my career, and yet it seems like every other blog post I read in certain language circles everyone is some kind of ASM and Reverse Eng…

....And then feel OK resorting to ChatGPT for the explanation.

Seriously that threw me, and maybe it makes sense in this context but it seems strange for someone with such an apparent depth of technical knowledge leaning on an LLM for anything.

Re: Print(“lol”) doubled the speed of my Go function

#78

Post by anonymoushn »

mcv wrote: post #60
dataflow wrote: post #23

Earlier quoted context omitted.

The performance of branches is data-dependent. The performance of conditional moves is data-independent. In this case the branches are predictable, so they perform better here. In general, though, the compiler has no idea whether that's the case, so it makes sense to insert a conditional move to avoid branch misprediction penalties.

But still, why the difference? With or without the "lol", the data dependence is identical, the predictability is identical. Yet with the "lol" it optimises differently. My best guess is that it's because of the added cost to the other branch: printing is expensive, so the compiler really doesn't want to do that, and would prefer to incorrectly predict not-printing than to incorrectly predict printing. If that tradeo…

When compiling code like

  if a > b
    b = a
    continue
  else
    continue
you can replace the entire if-else with a cmov. but if there is a function call in one of the prongs and not in the other, you cannot.

Re: Print(“lol”) doubled the speed of my Go function

#79

Post by smcl »

grose wrote: post #64
smcl wrote: post #37

Does Go have any facility for providing hints to the optimiser (like how some C compilers support #pragmas) that could cause the branch-predicted instruction to be used rather than the slower one?

Seems like the answer is no[1] and profile-guided optimization is recommended instead, https://go.dev/doc/pgo . I would be curious to see if pgo helps with the author's use case. [1] https://groups.google.com/g/golang-nuts/c/1erdKe3aV5k

Ah thanks! That's interesting but a bit weird to me. That response sounds a little bit like someone who feels like they shouldn't do something and is thinking on-the-fly for reasons they can use to justify that feeling.

> We don't want to complicate the language

So I can understand if this complicates the implementation but I don't know if totally optional pragmas or annotations complicates the language itself. Like C has this but I don't think people say "Ah C is alright but the pragmas are a bit confusing and make things complicated".

> experience shows that programmers are often mistaken as to whether branches are likely or not

Your average programmer may mess that up, but those who would give optimisation hints aren't quite your average programmer. And insisting on introducing PGO to your build process (so build, run-with-profile, rebuild-with-profile) for some cases where someone isn't mistaken as to whether branches are likely (or whether some loops run minimum X times, etc) feels a bit needless.

Please remember though that I'm neither a Go programmer nor contributor so I'm really just an outsider looking in, it could be that this is a total non-issue or is really low-priority.

Re: Print(“lol”) doubled the speed of my Go function

#80

Post by lordlimecat »

mcv wrote: post #48
keyle wrote: post #47

Earlier quoted context omitted.

They're a dying breed. We're forgetting how to look under the hood and understand "why something works". Case in point, I'm slowly being replaced by Salesforce muppets for all my projects at work. They're little code monkeys with amazon ebook type knowledge, projects cost 20x more and I look like the mad scientist for speaking the truth. The products are worse in every possible metrics, I'm not crazy. The politics at…

> We're forgetting how to look under the hood and understand "why something works". Partly because that's often not what we're supposed to do; the stuff under the hood "just works" and we're meant to use it to write features, not worry about optimising the stuff that happens under the hood. And partly it's because the stuff under the hood is increasingly weird and bizarre. Branch prediction is weird, and I still don'…

In my world "just works" means "we blew away all of the security controls and best practice to get this thing hobbling across the finish line."

I see COTS products using ldap memberof queries without LDAP_MATCHING_RULE_IN_CHAIN and stating definitively in their documentation that nested groups are bad (despite decades of best practice).

I see product documentation recommending authenticating against LDAP instead of kerberos, despite the underlying libraries having full kerberos support.

I see sslverify: no, and flags to ignore SSH TOFU warnings, and recommendations to avoid SSH gssapi-keyex (WHY?????), and security approached by buying ever more products creating ever more complexity.

Yes, things "just work" in a horrible, 'youre stuck with your vendors forever' sort of way that results in lengthy outages every 6 months due to mounting, intractable technical debt. But things don't have to be this way, you just need people who are willing to ask "why" or "is that necessary" or "can it be better".

Post reply on HN