Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

blogs.embarcadero.com

Post reply on HN
Print view View original

181–187 of 187 posts

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#181

Post by bullen »

drmpeg wrote: post #95
bullen wrote: post #47

Earlier quoted context omitted.

Aha, I only compile on ARM so I got no room to increase RAM... Is it the same with g++? I have 4GB so I should be able to compile with 4 cores, but the processes only fill 2-3 cores even when I try make -j8 on a 8 core machine and then locks the entire OS until it craps out?! Something is fishy...

A swap file is your friend.

Actually, you win, swap was set to 0 by default! :D

Also I only have 2GB (in 32-bit though)! Xo

High karma HN users are toxic downers.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#182

Post by jcelerier »

bullen wrote: post #181
drmpeg wrote: post #95

Earlier quoted context omitted.

A swap file is your friend.

Actually, you win, swap was set to 0 by default! :D Also I only have 2GB (in 32-bit though)! Xo High karma HN users are toxic downers.

if you are in 32-bits you could still have trouble in the future though.

source: two days ago I had the exact same problem, so I mounted a 32gb swap file over an NFS drive because my SD card was 2gb (don't ask) and it still failed because ld tried to use more than 4gigs of ram

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#183

Post by nwmcsween »

MayeulC wrote: post #101
ska wrote: post #50

Earlier quoted context omitted.

Hmm. Maybe the tradeoff has changed since I last tested this (to be fair, a few years ago). But I'm also not focused on build servers especially, it's always been possible to make those reasonably fast. Unless you have a very specific sort of workflow anyway, your devs are doing way more local builds than on the server and that sped up a ton moving to SSD, in my experience anyway. YMMV of course.

Well, you can always move your code to a ramdisk, I suspect any C(++) isn't more than a few GB anyway ?

You don't even need to do this just cat all the files to /dev/null to prime the cache

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#184

Post by nwmcsween »

tomc1985 wrote: post #177
rbanffy wrote: post #167

Earlier quoted context omitted.

Heat dissipation can be an issue. Writing continuously generates a lot of heat.

Part of the purpose of a burn test is to see how it handles temperature under load. We didn't have the option of adding cooling, many of the product installations took place in a hot climate, and nobody wanted to pay for a hardened part... Anyway, my point is that SSD drive reliability varies wildly

Nvme does, SSDs as in sata doing this burn test will probably work fine

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#185

Post by peter_d_sherman »

This seems to be a little bit related to this quest for fast compilation:

The "mold" linker:

https://github.com/rui314/mold

>"Concretely speaking, I wanted to use the linker to link a Chromium executable with full debug info (~2 GiB in size) just in 1 second. LLVM's lld, the fastest open-source linker which I originally created a few years ago, takes about 12 seconds to link Chromium on my machine. So the goal is 12x performance bump over lld. Compared to GNU gold, it's more than 50x."

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#186

Post by MayeulC »

flukus wrote: post #146
MayeulC wrote: post #101

Earlier quoted context omitted.

Well, you can always move your code to a ramdisk, I suspect any C(++) isn't more than a few GB anyway ?

Most compilers won't fsync will they? The output is likely not being written straight to disk. It's caches all the way down.

Well, using ramdisks will let you compare and make sure the disk isn't the bottleneck, at least.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#187

Post by 55873445216111 »

amarshall wrote: post #170
55873445216111 wrote: post #79

Earlier quoted context omitted.

You can get ECC UDIMMs from Supermicro. They are rebranded Micron DIMMs. ECC memory is not going to go as high of frequencies as you might be looking for. They will only go up to the officially validated speed of the CPUs. https://store.supermicro.com/16gb-ddr4-mem-dr416l-cv02-eu26....

The rated speeds are not as high, but ECC memory can be overclocked just as non-ECC; memory overclock support is mostly up to the motherboard. I have some DDR4-2666 ECC overclocked to 3200 MHz with slightly tighter timings on TRX40.

Good point. I guess what I meant to say is that ECC memory is not marketed for greater than JEDEC speeds.
Post reply on HN