Lucky sons of gun. We are stuck with Xeons. Have to wait 3 hours for our 20M C/C++ on the 2x14cores Xeon machine after a pull/rebase. Ryzen/TR would probably be faster 2-3x times for the same money, yet it is a BigCo, so no such luck (and our product is certified only for Xeons, so our customers can't run AMD too - thus we're de-facto part of the Great Enterprise Wall blocking AMD from on-premise datacenter).
I upgraded from 2x 12 core xeons to a 64 core thread ripper - compile times dropped from 45m to 12m
Fun experiment. The more pedestrian 5950X or the now bargain 3950X are great for anyone doing a lot of compiling. With the right motherboard they even have ECC RAM support. Game changer for workstations in the $1000–$2000 range. The more expensive Threadripper parts really shine when memory bandwidth becomes a bottleneck. In my experience, compiling code hasn’t been very memory bandwidth limited. However, some of my…
I suspect the biggest (build time) benefit to most c++ workflows and toolchains was the move to ubiquitous SSD. Prior to that in my experience excepting expensive RAID array dedicated build machines, it was really easy to build a system that would always be IO bound on builds. There of course were tricks to improve things but you still tended to hit that wall unless your CPUs were really under spec. edit: to be clear…
i (and some teamates) actually put HDDs on some workstations as SSD just die after 2-3 years of active build on them and with modern HDDs you have practically unlimited storage while you can have only limited number of 400G builds on SSD (the org has psychological barriers to having more than 1-2Tb SSD in a machine) and the SSD start to have perf issues when at 70-80% capacity . With HDD the build time didn't change much - the machines have enough memory for the system to cache a lot (256-512G RAM).
I suspect the biggest (build time) benefit to most c++ workflows and toolchains was the move to ubiquitous SSD. Prior to that in my experience excepting expensive RAID array dedicated build machines, it was really easy to build a system that would always be IO bound on builds. There of course were tricks to improve things but you still tended to hit that wall unless your CPUs were really under spec. edit: to be clear…
i (and some teamates) actually put HDDs on some workstations as SSD just die after 2-3 years of active build on them and with modern HDDs you have practically unlimited storage while you can have only limited number of 400G builds on SSD (the org has psychological barriers to having more than 1-2Tb SSD in a machine) and the SSD start to have perf issues when at 70-80% capacity . With HDD the build time didn't change…
> i actually put HDDs on some workstations as SSD just die after 2-3 years of active build on them
That sounds very low for modern SSDs, even consumer-grade. Have you tried different vendors?
Seems like he tried more complex examples too, but ran into roadblocks like a 2GB limit on executables and running into a commandline length limit restriction that dates back to early DOS days which made it impossible to link. Both of those problems seemed solvable if he was willing to chunk up his application into libraries, maybe 1024 files per library then linked to the main application.
I believe this is one of the reasons for object libraries (or archives, the foo.a files on linux/unix), you can then link in all of the object files from one of those at link time without having to list them all at once. That won't get past the 2GB limit on executables but it will get past the command line length.
This is correct, a .lib file on Windows has a bunch of .obj files in it that you can then link together.
You can also use command files[1] to pass options in instead of using the command line.
Hahaha, fuck me, CPUs are fast . That's wicked. 15 mins. A billion lines of C. Insane. Wonder if there's some IO speed to be gained from ramdisking the inputs.
17,300 lines/sec per core. That's embarrassingly slow IMHO.
i (and some teamates) actually put HDDs on some workstations as SSD just die after 2-3 years of active build on them and with modern HDDs you have practically unlimited storage while you can have only limited number of 400G builds on SSD (the org has psychological barriers to having more than 1-2Tb SSD in a machine) and the SSD start to have perf issues when at 70-80% capacity . With HDD the build time didn't change…
> i actually put HDDs on some workstations as SSD just die after 2-3 years of active build on them That sounds very low for modern SSDs, even consumer-grade. Have you tried different vendors?
If spending hours a day at 100% utility, SSDs will rarely last 5 years.
I suspect the biggest (build time) benefit to most c++ workflows and toolchains was the move to ubiquitous SSD. Prior to that in my experience excepting expensive RAID array dedicated build machines, it was really easy to build a system that would always be IO bound on builds. There of course were tricks to improve things but you still tended to hit that wall unless your CPUs were really under spec. edit: to be clear…
i (and some teamates) actually put HDDs on some workstations as SSD just die after 2-3 years of active build on them and with modern HDDs you have practically unlimited storage while you can have only limited number of 400G builds on SSD (the org has psychological barriers to having more than 1-2Tb SSD in a machine) and the SSD start to have perf issues when at 70-80% capacity . With HDD the build time didn't change…
Those are pretty beefy workstations, does every developer have one or are these really build servers? As I noted you could always throw money at this and end up somewhere reasonable, but it introduces workflow considerations for your devs.
Lucky sons of gun. We are stuck with Xeons. Have to wait 3 hours for our 20M C/C++ on the 2x14cores Xeon machine after a pull/rebase. Ryzen/TR would probably be faster 2-3x times for the same money, yet it is a BigCo, so no such luck (and our product is certified only for Xeons, so our customers can't run AMD too - thus we're de-facto part of the Great Enterprise Wall blocking AMD from on-premise datacenter).
Industrial software will always grow in size to use all available compilation time ... I have seen large Xeon distcc farms and the total build walltime was still measured in hours...