Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

blogs.embarcadero.com

Post reply on HN
Print view View original

51–60 of 187 posts

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#51

Post by Retric »

winkeltripel wrote: post #28
outworlder wrote: post #25

Earlier quoted context omitted.

> i actually put HDDs on some workstations as SSD just die after 2-3 years of active build on them That sounds very low for modern SSDs, even consumer-grade. Have you tried different vendors?

If spending hours a day at 100% utility, SSDs will rarely last 5 years.

If your SSD would be at 100% utilization it’s going to take a lot of HDD to reach that kind of bandwidth. To the point where for high bandwidth loads SSD’s actually cost less even if you have to replace them regularly.

100% utilization and 30x the bandwidth = 30x as many HDD. Alternatively, if HDD’s are an option you’re a long way from 100% utilization.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#52

Post by mywittyname »

kyrra wrote: post #34
winkeltripel wrote: post #28

Earlier quoted context omitted.

If spending hours a day at 100% utility, SSDs will rarely last 5 years.

Depends on the kind of SSD. If it's using SLC, the write endurance is much much higher. If you're going with cheap SSDs (TLC or QLC), your write endurance will suck. see: https://www.anandtech.com/show/6459/samsung-ssd-840-testing-...

SLC seems to be going away pretty quickly, if it hasn't already been phased out. It just can't produce the price / GB of the better tech. Also, that article you linked is almost 10 years old.

You're best bet for long-term reliability is to buy much more capacity than you need and try not to exceed >50% capacity for high write frequency situations. I keep an empty drive around to use as a temp directory for compiling, logging, temp files, etc.

Also, my understanding is that consumer-grade drive need a "cool down" period to allow them to perform wear leveling. So you don't want to be writing to these drives constantly.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#53

Post by fctorial »

ska wrote: post #18
PragmaticPulp wrote: post #13

Fun experiment. The more pedestrian 5950X or the now bargain 3950X are great for anyone doing a lot of compiling. With the right motherboard they even have ECC RAM support. Game changer for workstations in the $1000–$2000 range. The more expensive Threadripper parts really shine when memory bandwidth becomes a bottleneck. In my experience, compiling code hasn’t been very memory bandwidth limited. However, some of my…

I suspect the biggest (build time) benefit to most c++ workflows and toolchains was the move to ubiquitous SSD. Prior to that in my experience excepting expensive RAID array dedicated build machines, it was really easy to build a system that would always be IO bound on builds. There of course were tricks to improve things but you still tended to hit that wall unless your CPUs were really under spec. edit: to be clear…

Don't SSDs have a finite TBW? 50GB of writes everyday (possible on large projects) will consume that in a couple of months.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#54

Post by gm »

That article mentioned Delphi and Object Pascal, and it brought back many fond memories. I absolutely LOVED Delphi and Object Pascal back in the day. So clean and so fun to program in. If Borland hadn't f-ed it up and had stayed around until now, I'd be the biggest Delphi fanboy.

Alas, that was not to be. Modern languages are fun and all, but not Delphi-back-in-the-day level fun :-).

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#55

Post by dboat »

After liking this article, I wanted to check out others on the site, and am shocked at the terrible usability of their front page. I can't finish reading the titles of their articles before the page just keeps moving things around on me. It is so frustrating, which is unfortunate because I would otherwise have been interested to see more of their content. Experience completely ruined by awful design judgment.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#57

Post by zajio1am »

klodolph wrote: post #17
ahepp wrote: post #3

>C++Builder with TwineCompile is a powerful productivity solution for multi-core machines compiling 1 million lines of code very quickly and can work better than the MAKE/GCC parallel compilation Jobs feature due to it’s deep IDE integration You're claiming this plugin has deeper IDE integration than `make`? I find that really, really difficult to believe. And if it's true, it seems like the solution is to either use…

Make forces you to choose between being able to do full parallel builds or using recursive make, you can’t do both.

Why would you do recursive make? That is setup discouraged for decades ...

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#58

Post by fctorial »

trhway wrote: post #24
ska wrote: post #18

Earlier quoted context omitted.

I suspect the biggest (build time) benefit to most c++ workflows and toolchains was the move to ubiquitous SSD. Prior to that in my experience excepting expensive RAID array dedicated build machines, it was really easy to build a system that would always be IO bound on builds. There of course were tricks to improve things but you still tended to hit that wall unless your CPUs were really under spec. edit: to be clear…

i (and some teamates) actually put HDDs on some workstations as SSD just die after 2-3 years of active build on them and with modern HDDs you have practically unlimited storage while you can have only limited number of 400G builds on SSD (the org has psychological barriers to having more than 1-2Tb SSD in a machine) and the SSD start to have perf issues when at 70-80% capacity . With HDD the build time didn't change…

What are the signs of an SSD that's about to die?

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#59

Post by fctorial »

EricE wrote: post #42
trhway wrote: post #24

Earlier quoted context omitted.

i (and some teamates) actually put HDDs on some workstations as SSD just die after 2-3 years of active build on them and with modern HDDs you have practically unlimited storage while you can have only limited number of 400G builds on SSD (the org has psychological barriers to having more than 1-2Tb SSD in a machine) and the SSD start to have perf issues when at 70-80% capacity . With HDD the build time didn't change…

A way to have your cake and eat it too - check out Primocache. It's pretty inexpensive disk caching software (especially for Windows Server which is where I really leverage it!). Pair it with an Optane for L2 cache and it will speed up normal SSD use too ;)

How does it compare to default linux caching algorithm?

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#60

Post by PartiallyTyped »

PragmaticPulp wrote: post #13

Fun experiment. The more pedestrian 5950X or the now bargain 3950X are great for anyone doing a lot of compiling. With the right motherboard they even have ECC RAM support. Game changer for workstations in the $1000–$2000 range. The more expensive Threadripper parts really shine when memory bandwidth becomes a bottleneck. In my experience, compiling code hasn’t been very memory bandwidth limited. However, some of my…

IIRC AMD has EEC support in all x70/x50 motherboards and cpu combinations. If I may, what kind of simulations are you running?

I am trying to build a system for Reinforcement Learning research and seeing many things depend on python, I am not certain how to best optimise the system.

Post reply on HN