Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD's Turin: 5th Gen EPYC Launched

chipsandcheese.com

Post reply on HN
Print view View original

161–170 of 175 posts

Re: AMD's Turin: 5th Gen EPYC Launched

#161

Post by unnah »

kolbe wrote: post #110
thelastparadise wrote: post #100

I wonder how LLM performance is on the higher core counts? With recent DDR generations and many core CPUs, perhaps CPUs will give GPUs a run for their money.

The H100 has 16,000 cuda cores at 1.2ghz. My rough calculation is it can handle 230k concurrent calculations. Whereas a 192 core avx512 chip (assuming it calculates on 16 bit data) can handle 6k concurrent calculations at 4x the frequency. So, about a 10x difference just on compute, not to mention that memory is an even stronger advantage for GPUs.

A Zen 5 core has four parallel AVX-512 execution units, so it should be able to execute 128 16-bit operations in parallel, or over 24k on 192 cores. However I think the 192-core processors use the compact variant core Zen 5c, and I'm not sure if Zen 5c is quite as capable as the full Zen 5 core.

Re: AMD's Turin: 5th Gen EPYC Launched

#162

Post by 7thpower »

noncoml wrote: post #148
7thpower wrote: post #4

Earlier quoted context omitted.

What are the specs? Tangentially related, but I need to go check a18 civ 6benchmarks. The experience on my a15 with small map sizes was surprisingly good.

It's not latest and greatest, 12900K + 64GB DDR4. But even when 12900K came out(2021) Civ 6 was already 5 years old

Still nothing to sneeze at!

Re: AMD's Turin: 5th Gen EPYC Launched

#163

Post by nullc »

zamadatix wrote: post #128
bee_rider wrote: post #111

Earlier quoted context omitted.

512 MB of cache, wow. A couple years ago I noticed that some Xeons I was using had a much cache as the ram in the systems I had growing up (millennial, so, we’re not talking about ancient commodores or whatever; real usable computers that could play Quake and everything). But 512MB? That’s roomy. Could Puppy Linux just be held entirely in L3 cache?

CCDs can't access each other's L3 cache as their own (fabric penalty is too high to do that directly). Assuming it's anything like the 9174F that means it's really 8 groups of 2 cores that each have 64 MB of L3 cache. Still enormous, and you can still access data over the infinity fabric with penalties, but not quite a block of 512 MB of cache on a single 16 core block that it might sound like at first. Zen 4 also ha…

9684x is 1152 MB cache per socket, 12 CCDs * 96MB. A similar X series zen5 is planned.

Though I wish they did some chips with 128GB of high bandwidth dram instead of a extra sized sram caches.

Re: AMD's Turin: 5th Gen EPYC Launched

#164

Post by nullc »

thelastparadise wrote: post #100

I wonder how LLM performance is on the higher core counts? With recent DDR generations and many core CPUs, perhaps CPUs will give GPUs a run for their money.

They're memory bandwidth limited, you can basically just estimate the performance from the time it takes to read the entire model from ram for each token.

Re: AMD's Turin: 5th Gen EPYC Launched

#165

Post by 3abiton »

aurareturn wrote: post #108

Phoronix recently reviewed the 196 core Turin Dense against the AmpereOne 192 core. * Ampere MSRP $5.5K vs $15K for the EPYC. * Turin 196 had 1.6x better performance * Ampere had 1.2x better energy consumption In terms of actual $/perf, Ampere 192 core is 1.7x better than Turin Dense 196 core based on Phoronix's review. So for $5.5k, you can either buy an AmpereOne 192 core CPU (274w) or a Turin Dense 48 core CPU (30…

Very exciting age, and very sad drop for intel, although as many have been warning, they should have seen it coming

Re: AMD's Turin: 5th Gen EPYC Launched

#166

Post by menaerus »

unnah wrote: post #161
kolbe wrote: post #110

Earlier quoted context omitted.

The H100 has 16,000 cuda cores at 1.2ghz. My rough calculation is it can handle 230k concurrent calculations. Whereas a 192 core avx512 chip (assuming it calculates on 16 bit data) can handle 6k concurrent calculations at 4x the frequency. So, about a 10x difference just on compute, not to mention that memory is an even stronger advantage for GPUs.

A Zen 5 core has four parallel AVX-512 execution units, so it should be able to execute 128 16-bit operations in parallel, or over 24k on 192 cores. However I think the 192-core processors use the compact variant core Zen 5c, and I'm not sure if Zen 5c is quite as capable as the full Zen 5 core.

Right, I found this interesting as a thought exercise and took it from another angle.

Since it takes 4 cycles to execute FMA on double-precision 64-bit floats (VFMADD132PD) this translates to 1.25G ops/s (GFLOPS/s) per each core@5GHz. At 192 cores this is 240 GFLOPS/s. For a single FMA unit. At 2x FMA units per core this becomes 480 GFLOPS/s.

For 16-bit operations this becomes 1920 GFLOPS/s or 1.92 TFLOPS/s for FMA workloads.

Similarly, 16-bit FADD workloads are able to sustain more at 2550 GFLOPS/s or 2.55 TFLOPS/s since the FADD is a bit cheaper (3 cycles).

This means that for combined half-precision FADD+FMA workloads zen5 at 192 cores should be able to sustain ~4.5 TFLOPS/s.

Nvidia H100 OTOH per wikipedia entries, if correct, can sustain 50-65 TFLOP/s at single-precision and 750-1000 TFLOPS/s at half-precision. Quite a difference.

Re: AMD's Turin: 5th Gen EPYC Launched

#167

Post by menaerus »

RobinL wrote: post #144
conjecTech wrote: post #126

Earlier quoted context omitted.

The difference in throughput for local versus distributed orchestration would mainly come from serdes, networking, switching. Serdes can be substantial. Networking and switching has been aggressively offloaded from CPU through better hardware support. Individual tasks would definitely have better latency, but I'd suspect the impact on throughput/CPU usage might be muted. Of course at the extremes (very small jobs, ve…

By way of a single example, we've been migrating recently from spark to duckdb. Our jobs are not huge, but too big for a single 'normal' machine. We've gone from a 2.5 hour runtime on a cluster of 10 machines (40,vCPU total) to a 15 minute runtime on a 32vCPU single machine. I don't know for sure, but I think this is largely because it eliminates expensive shuffles and serde. Obviously results vary hugely depending o…

A cluster of 10 machines with 40 vCPUs in total would equate to 4 vCPUs per machine. I am not familiar with Spark internals but in the realm of distributed databases such a setup would generally make no sense at all (to me). So I think you're correct that most of the overhead was caused by machine-to-machine byte juggling. 4 vCPUs is nothing.

I suspect you would be able to cut down the 2.5hr runtime dramatically even with the Spark if you just deployed it as a single instance on that very same 32vCPU machine.

Re: AMD's Turin: 5th Gen EPYC Launched

#168

Post by elil17 »

smolder wrote: post #77

The weirdest one of the bunch is the AMD EPYC 9175F: 16 cores with 512MB of L3 cache! Presumably this is for customers trying to minimize software costs that are based on "per-core" licensing. It really doesn't make much sense to have so few cores at such an expense, otherwise. Does Oracle still use this style of licensing? If so, they need to knock it off. The only other thing I can think of is some purpose like HFT…

Many computational fluid dynamics programs have per core licensing and also benefit from large amounts of cache.

Re: AMD's Turin: 5th Gen EPYC Launched

#169

Post by aecmadden »

smolder wrote: post #77

The weirdest one of the bunch is the AMD EPYC 9175F: 16 cores with 512MB of L3 cache! Presumably this is for customers trying to minimize software costs that are based on "per-core" licensing. It really doesn't make much sense to have so few cores at such an expense, otherwise. Does Oracle still use this style of licensing? If so, they need to knock it off. The only other thing I can think of is some purpose like HFT…

This optimises for a key vmware license mechanism "Per core licensing with a minimum of 16 cores licensed per CPU.".

Re: AMD's Turin: 5th Gen EPYC Launched

#170

Post by dmitrygr »

hedora wrote: post #112
bee_rider wrote: post #111

Earlier quoted context omitted.

512 MB of cache, wow. A couple years ago I noticed that some Xeons I was using had a much cache as the ram in the systems I had growing up (millennial, so, we’re not talking about ancient commodores or whatever; real usable computers that could play Quake and everything). But 512MB? That’s roomy. Could Puppy Linux just be held entirely in L3 cache?

I wonder if you can boot it without populating any DRAM sockets.

That would require either rewriting drivers to never use DMA or making sure that all DMA controllers are able to write into and read from L3 directly.
Post reply on HN