Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

blogs.embarcadero.com

Post reply on HN
Print view View original

141–150 of 187 posts

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#141

Post by okr »

Wowfunhappy wrote: post #138
wtallis wrote: post #129

Earlier quoted context omitted.

SSDs never shrink the reported capacity in response to defective/worn-out blocks. Instead, they have spare area that is not accessible to the user. The SMART data reported by the drive includes an indicator of how much spare area has been used and how much remains. When the available spare area starts dropping rapidly, you're approaching the drive's end of life.

I so happened to know this already, but I must say I've always found the approach somewhat weird. Wouldn't it make more sense to give the user all the available space, and then remove capacity slowly as blocks go bad? I guess they think people would be annoyed? Imagine if we treated batteries like SSDs, not allowing the use of a set amount of capacity so that it can be added back later, when the battery's "real" capa…

I can imagine, OSes in general are not prepared to such reported numbers shrinking. There would be a hole.

What if a moment ago my OS has still 64G and then all of the sudden it only has 63G. Where would the data go? I think something has to make up for the loss.

For me it makes sense to report logically 64G and internally you do the remapping magic.

I wonder, how some OSes deal with a hot-swap of RAM. You have a big virtual address space and all of the sudden there is no physical memory behind it.

Hm.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#142

Post by mrmuagi »

PragmaticPulp wrote: post #49
ska wrote: post #18

Earlier quoted context omitted.

I suspect the biggest (build time) benefit to most c++ workflows and toolchains was the move to ubiquitous SSD. Prior to that in my experience excepting expensive RAID array dedicated build machines, it was really easy to build a system that would always be IO bound on builds. There of course were tricks to improve things but you still tended to hit that wall unless your CPUs were really under spec. edit: to be clear…

SSDs help, but nothing beats core count X clock speed when compiling. Source code files are relatively small and modern OSes are very good at caching. I ran out of SSD on my build server a while ago and had to use a mechanical HDD. To my surprise, it didn’t impact build times as much as I thought it would.

I did a test a while back where I had a workstation compiling linux with SSD and one with a HDD -- it turns out all the files were cached in the memory (measely 8gb). But for general usage and user experience I would reccomend SSD without any question.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#143

Post by wtallis »

Wowfunhappy wrote: post #138
wtallis wrote: post #129

Earlier quoted context omitted.

SSDs never shrink the reported capacity in response to defective/worn-out blocks. Instead, they have spare area that is not accessible to the user. The SMART data reported by the drive includes an indicator of how much spare area has been used and how much remains. When the available spare area starts dropping rapidly, you're approaching the drive's end of life.

I so happened to know this already, but I must say I've always found the approach somewhat weird. Wouldn't it make more sense to give the user all the available space, and then remove capacity slowly as blocks go bad? I guess they think people would be annoyed? Imagine if we treated batteries like SSDs, not allowing the use of a set amount of capacity so that it can be added back later, when the battery's "real" capa…

Shrinking the usable space on a block device is wildly impractical. The SSD has no awareness of how any specific LBA is being used, no way to communicate with the host system to find out what LBAs are safe to permanently remove. You can't just incrementally delete LBAs from the end of the drive, because important data gets stored there, like the backup GPT and OS recovery partitions. Filesystems also don't really like when they get truncated. Deleting LBAs from the middle of a filesystem would be even more catastrophic. The vast majority of software and operating systems are simply not equipped to treat all block devices as thinly-provisioned. [1]

And SSDs already have all the infrastructure for fully virtualizing the mapping between LBAs and physical addresses, because that's fundamental to their ordinary operation. They also don't all start out with the same capacity; a brand-new SSD already starts out with a non-empty list of bad blocks, usually a few per die.

Even if it were practical to dynamically shrink block devices, it wouldn't be worth the trouble. SSD wear leveling is generally effective. When the drive starts retiring worn out blocks en masse, you can expect most of the "good" blocks to end up in the "bad" column pretty soon. So trying to continue using the drive would mean you'd see the usable capacity rapidly diminish until it reached the inevitable catastrophe of deleting critical data. It makes a lot more sense to stop before that point and make the drive read-only while all the data is still intact and recoverable.

[1] Technically, ATA TRIM/NVMe Deallocate commands mean the host can inform the drive about what LBAs are not currently in use, but that always comes with the expectation that they are still available to be used in the future. NVMe 1.4 added commands like Verify and Get LBA Status that allow the host to query about damaged LBAs, but when the drive indicates data has been unrecoverably corrupted, the host expects to be able to write fresh data to those LBAs and have it stored on media that's still usable. The closest we can get to the kind of mechanism you want is with NVMe Zoned Namespaces, where the drive can mark individual zones as permanently read-only or offline. But that's pretty coarse-grained, and handling it gracefully on the software side is still a challenge.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#145

Post by biglost »

MayeulC wrote: post #101
ska wrote: post #50

Earlier quoted context omitted.

Hmm. Maybe the tradeoff has changed since I last tested this (to be fair, a few years ago). But I'm also not focused on build servers especially, it's always been possible to make those reasonably fast. Unless you have a very specific sort of workflow anyway, your devs are doing way more local builds than on the server and that sped up a ton moving to SSD, in my experience anyway. YMMV of course.

Well, you can always move your code to a ramdisk, I suspect any C(++) isn't more than a few GB anyway ?

With gentoo i use tmpfs and it works really well

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#146

Post by flukus »

MayeulC wrote: post #101
ska wrote: post #50

Earlier quoted context omitted.

Hmm. Maybe the tradeoff has changed since I last tested this (to be fair, a few years ago). But I'm also not focused on build servers especially, it's always been possible to make those reasonably fast. Unless you have a very specific sort of workflow anyway, your devs are doing way more local builds than on the server and that sped up a ton moving to SSD, in my experience anyway. YMMV of course.

Well, you can always move your code to a ramdisk, I suspect any C(++) isn't more than a few GB anyway ?

Most compilers won't fsync will they? The output is likely not being written straight to disk.

It's caches all the way down.

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#148

Post by heelix »

m463 wrote: post #70
bmurphy1976 wrote: post #65

Earlier quoted context omitted.

Where can you get decent ECC ram for a reasonable price? I was on the hunt recently for ECC RAM for my new desktop and I gave up and pulled the trigger on low latency non-ECC RAM. Availability seems to be pretty terrible at the moment.

same for everything. 5950? where would you even get one? wait - seems you can get one, just pay 2x list price.

I picked one up at the Minneapolis Microcenter about a week ago at near (or less) than MSRP. I'd been looking for any stock since launch, and that was the first one I'd seen show up in stock and queued up for it. (Same day as a bunch of video cards showed up, so a bit less competition.)

They have an online checkin app that they use to order the queue. You see it show up in stock, add yourself to the queue, and get an early enough spot - show up. No need to wait in the cold. (ymmv on weather, but up here... can be cold)

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#149

Post by [deleted] »

nh2 wrote: post #137
ska wrote: post #50

Earlier quoted context omitted.

Hmm. Maybe the tradeoff has changed since I last tested this (to be fair, a few years ago). But I'm also not focused on build servers especially, it's always been possible to make those reasonably fast. Unless you have a very specific sort of workflow anyway, your devs are doing way more local builds than on the server and that sped up a ton moving to SSD, in my experience anyway. YMMV of course.

Last time I benchmarked C++ compilation on SSDs vs HDDS (compiling the PCL project on Linux which took around 45 minutes), SSD didn't help in a noticeable fashion. I believe that this makes sense: In a typical C++ project like that which use template libraries like CGAL, compilation of a single file takes up to 30 seconds of CPU-only time. Even though each file (thanks to lack of sensible module system) churns throug…

[deleted]

Re: Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores

#150

Post by ianhanschen »

Great read. I wonder if the make -j modification wasn’t scaling things across all cores because it was using the physical core count (number of cores) versus the logical core count (number of core threads).

Or perhaps the code wasn’t modified to spread the work across all processor core groups (a Windows thing to support more than 64 logical cores).

https://bitsum.com/general/the-64-core-threshold-processor-g...

Post reply on HN