Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

blogs.embarcadero.com

Post reply on HN
Print view View original

41–50 of 187 posts

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#41

Post by ChuckMcM »

StillBored wrote: post #20
klodolph wrote: post #17

Earlier quoted context omitted.

Make forces you to choose between being able to do full parallel builds or using recursive make, you can’t do both.

I might be misunderstanding something, but the common gnumake does no such thing. https://www.gnu.org/software/make/manual/html_node/Job-Slots...

It may be a Windows thing. I too started reading the post and was thinking, why not -j128 or -j64 depending on if HT was on and then realized that the author's system wasn't one that had been tuned for decades to build software as quickly as possible :-).

It would be an interesting academic exercise to create a setup on Linux that did the same thing but using the extant build tools, and then to do a deep dive into how much of the computer was actually compiling code and how much of it was running the machinery of the OS/IDE/Linkages. A LONG time ago we used to do that at Sun to profile the "efficiency" of a machine in order to maximize the work it did for the user per unit time, vs other things. That sort of activity however became less of a priority as the CPUs literally became 1000x faster.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#42

Post by EricE »

trhway wrote: post #24
ska wrote: post #18

Earlier quoted context omitted.

I suspect the biggest (build time) benefit to most c++ workflows and toolchains was the move to ubiquitous SSD. Prior to that in my experience excepting expensive RAID array dedicated build machines, it was really easy to build a system that would always be IO bound on builds. There of course were tricks to improve things but you still tended to hit that wall unless your CPUs were really under spec. edit: to be clear…

i (and some teamates) actually put HDDs on some workstations as SSD just die after 2-3 years of active build on them and with modern HDDs you have practically unlimited storage while you can have only limited number of 400G builds on SSD (the org has psychological barriers to having more than 1-2Tb SSD in a machine) and the SSD start to have perf issues when at 70-80% capacity . With HDD the build time didn't change…

A way to have your cake and eat it too - check out Primocache. It's pretty inexpensive disk caching software (especially for Windows Server which is where I really leverage it!).

Pair it with an Optane for L2 cache and it will speed up normal SSD use too ;)

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#43

Post by jcelerier »

bullen wrote: post #40

In my experience multi-core compilation does not work. make -j>3 just locks the process and fails.

You just need more ram. I 'ever compile at less than -j$(ncpu). Hard with less than 32 GB tho - a single clang instance can easily eat upwards of 1gb of ram

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#44

Post by ChuckMcM »

klodolph wrote: post #36
StillBored wrote: post #20

Earlier quoted context omitted.

I might be misunderstanding something, but the common gnumake does no such thing. https://www.gnu.org/software/make/manual/html_node/Job-Slots...

I didn’t go into great detail of the reasons, but the jobserver doesn’t address the problem except for the most trivial cases—the core problem is that you can’t cross to a recursive make invocation via multiple edges. This is fairly common in larger projects, so you end up having to do some hackery to manually sequence make invocations if you want to use recursive make (which is pretty awful). Honestly, for large pro…

That is a really excellent observation.

My experience is that "larger projects" to the investment in their build infrastructure to maximize parallelism in order to reduce build times because it pays dividends in terms of turn around time and thus productivity.

One of the joys of working with someone who has been building large projects for a while, is that they just design the build system from the start to be as parallel as practical.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#45

Post by ska »

trhway wrote: post #38
ska wrote: post #29

Earlier quoted context omitted.

Those are pretty beefy workstations, does every developer have one or are these really build servers? As I noted you could always throw money at this and end up somewhere reasonable, but it introduces workflow considerations for your devs.

It is one per dev, and in reality it is at least a couple - I have 4 of them for example. The product itself is a beast too.

That works! It helps that "plenty-of-ram" is more achievable than it used to be, also.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#46

Post by StillBored »

klodolph wrote: post #36
StillBored wrote: post #20

Earlier quoted context omitted.

I might be misunderstanding something, but the common gnumake does no such thing. https://www.gnu.org/software/make/manual/html_node/Job-Slots...

I didn’t go into great detail of the reasons, but the jobserver doesn’t address the problem except for the most trivial cases—the core problem is that you can’t cross to a recursive make invocation via multiple edges. This is fairly common in larger projects, so you end up having to do some hackery to manually sequence make invocations if you want to use recursive make (which is pretty awful). Honestly, for large pro…

Make isn't great, but if you look at the opensource world the vast majority of the large projects are make based. The backbone of your average distro is a thin layer on top of what is mostly automake/autoconf in the individual projects. That is because while I can create a toy project that builds with make in a half dozen lines, big projects can extend it to cover those odd edge cases that break a lot of other "better" build systems. Particularity, when a project starts including a half dozen different languages.

So, while i'm not a make fan, I'm really tired of people pissing on solutions (c also comes to mind) that have been working for decades because of edge cases or problems of their own creation because they don't understand the tool.

A well understood tool is one where people know where the problems lie and work around them. Most "perfect" tools are just project crashing dragons hiding under pretty marketing of immature tooling.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#47

Post by bullen »

jcelerier wrote: post #43
bullen wrote: post #40

In my experience multi-core compilation does not work. make -j>3 just locks the process and fails.

You just need more ram. I 'ever compile at less than -j$(ncpu). Hard with less than 32 GB tho - a single clang instance can easily eat upwards of 1gb of ram

Aha, I only compile on ARM so I got no room to increase RAM...

Is it the same with g++? I have 4GB so I should be able to compile with 4 cores, but the processes only fill 2-3 cores even when I try make -j8 on a 8 core machine and then locks the entire OS until it craps out?!

Something is fishy...

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#48

Post by bserge »

titzer wrote: post #27
renewiltord wrote: post #15

Hahaha, fuck me, CPUs are fast . That's wicked. 15 mins. A billion lines of C. Insane. Wonder if there's some IO speed to be gained from ramdisking the inputs.

17,300 lines/sec per core. That's embarrassingly slow IMHO.

What can perform better?

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#49

Post by PragmaticPulp »

ska wrote: post #18
PragmaticPulp wrote: post #13

Fun experiment. The more pedestrian 5950X or the now bargain 3950X are great for anyone doing a lot of compiling. With the right motherboard they even have ECC RAM support. Game changer for workstations in the $1000–$2000 range. The more expensive Threadripper parts really shine when memory bandwidth becomes a bottleneck. In my experience, compiling code hasn’t been very memory bandwidth limited. However, some of my…

I suspect the biggest (build time) benefit to most c++ workflows and toolchains was the move to ubiquitous SSD. Prior to that in my experience excepting expensive RAID array dedicated build machines, it was really easy to build a system that would always be IO bound on builds. There of course were tricks to improve things but you still tended to hit that wall unless your CPUs were really under spec. edit: to be clear…

SSDs help, but nothing beats core count X clock speed when compiling.

Source code files are relatively small and modern OSes are very good at caching. I ran out of SSD on my build server a while ago and had to use a mechanical HDD. To my surprise, it didn’t impact build times as much as I thought it would.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#50

Post by ska »

PragmaticPulp wrote: post #49
ska wrote: post #18

Earlier quoted context omitted.

I suspect the biggest (build time) benefit to most c++ workflows and toolchains was the move to ubiquitous SSD. Prior to that in my experience excepting expensive RAID array dedicated build machines, it was really easy to build a system that would always be IO bound on builds. There of course were tricks to improve things but you still tended to hit that wall unless your CPUs were really under spec. edit: to be clear…

SSDs help, but nothing beats core count X clock speed when compiling. Source code files are relatively small and modern OSes are very good at caching. I ran out of SSD on my build server a while ago and had to use a mechanical HDD. To my surprise, it didn’t impact build times as much as I thought it would.

Hmm. Maybe the tradeoff has changed since I last tested this (to be fair, a few years ago). But I'm also not focused on build servers especially, it's always been possible to make those reasonably fast. Unless you have a very specific sort of workflow anyway, your devs are doing way more local builds than on the server and that sped up a ton moving to SSD, in my experience anyway. YMMV of course.
Post reply on HN