Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computing Performance on the Horizon

brendangregg.com

Post reply on HN
Print view View original

71–75 of 75 posts

Re: Computing Performance on the Horizon

#71

Post by dragontamer »

drewg123 wrote: post #54
dragontamer wrote: post #41

Earlier quoted context omitted.

When a hypothetical 128-core single socket comes out, will there be no workload that prefers to use a 2x128-core dual socket instead? AMD CPUs remain largely dual-socket compatible. Today's 64-core EPYCs can be dual-socketed into 2x64-core beasts. It just seems silly to me that if you're building say 200 computers in 10x racks (20-computers per 10x 40U racks) that you'd prefer single socket over dual-socket. If you'r…

There are a few problems with dual sockets: - The flip side of cuts the computers you need to work with in half is that it doubles the blast radius in case of PSU/fan/mobo/etc failure - If you're interested in I/O, dual sockets can be problematic because few motherboards are "balanced" with an equal number of PCIe slots local to each socket. - NUMA makes everything harder. Even after the work that I've done to make N…

I do appreciate the difficulty of getting software configured correctly.

But I'm of the opinion that software configuration is quicker and easier than redeveloping algorithms to become faster on FPGAs or GPUs.

Its really odd to have a talk about how FPGAs are part of a hypothetical mainstream future (when so few people even know how to code in Verilog, let alone know how to synthesize a systolic array or other obscure parallel architecture). And then turn around and say that Dual-Socket computers are too hard to configure.

Verilog / FPGAs aren't magic. They're just highly configurable logic gates + some preconfigured ALUs that allow for alternative parallel structures. These alternative parallel structures (most commonly a systolic array) are often highly specific to a task. But ultimately: the mode of compute still needs to be super-parallel to beat a CPU.

Remember: CPUs have higher clock-speeds than FPGAs. That's why FPGAs have mini-ALUs inside of them (ex: multipliers), because ASIC beats configurable logic in every spec that matters (GHz, power-efficiency, mm^2 on die).

Re: Computing Performance on the Horizon

#72

Post by eloff »

dragontamer wrote: post #25
eloff wrote: post #23

Earlier quoted context omitted.

I'm talking mostly abour scaling things like app servers where they might not need any communication. But in general if you can't scale horizontally at 10 gbps, you're in for a world of hurt. Numa gets you to 8x scale at best on very expensive very exotic hardware. And then you hit the wall.

I'm mostly talking about 2 socket servers, which are IIRC more common than even single socket servers. Dual socket is a cheap, easy, and common. If only to recycle fans, power supplies and racks, it seems useful.

And single socket is equally cheap, except it takes twice the rack space - but it also gives you redundancy. One server can fail and you can carry on.

The advantage of memory bandwidth vs Ethernet for scaling to x2 really doesn't matter. If it did, you're not horizontally scalable and at best you buy a little time before you hit the wall.

If the price difference isn't much, I would heavily prefer single socket.

Re: Computing Performance on the Horizon

#73

Post by brendangregg »

dragontamer wrote: post #53
brendangregg wrote: post #51

Earlier quoted context omitted.

I don't have a workload I'd prefer to see on 2x128-core: We're already microservices running across a pool of instances, and would prefer a bigger pool of faster instances than a smaller pool of slower ones at the same cost. Once we get a workload running on 100+ cores, I often see a lot of lock contention anyway. Going bigger usually makes that worse (worse ROI). As for datacenter size/cost, it's a good point, but w…

> As for datacenter size/cost, it's a good point, but what if two 1-Socket servers could take up the same space as one 2-Socket server? :-) That may never happen, but some level of space optimization will Oh it certainly exists. Computers are space-optimized to the point of nonsense. IIRC, most people don't even bother to use widely available 1U servers because you run out of power before you fill up 40U racks. Hyper…

Ah, thanks for the details, I wasn't referring to NUMA-induced lock contention, but rather lock contention in general. I've seen a workload hit 64 CPUs with 80% of CPU time in lock contention (and others in the 10-30%). Now, while that means the developer has a big problem to fix, it also has me wondering about single sockets getting big enough -- 128 CPUs is already hard to use well. Back when I first saw multi-socket systems with 2-4 total CPUs, getting the extra CPUs online was all goodness. But adding another 128 CPUs to my already 128-CPU system, well...is there a point where we can say, in general, that we already have enough cores? In the talk I referred to the 850,000-core GPU, and how I couldn't see that ever working as general purpose CPUs in the software of today. With 3D stacking, I think we'll reach a practical core limit on a single socket, and just won't need the complexity (including NUMA) of multi socket anymore.

Re: Computing Performance on the Horizon

#74

Post by dragontamer »

brendangregg wrote: post #73
dragontamer wrote: post #53

Earlier quoted context omitted.

> As for datacenter size/cost, it's a good point, but what if two 1-Socket servers could take up the same space as one 2-Socket server? :-) That may never happen, but some level of space optimization will Oh it certainly exists. Computers are space-optimized to the point of nonsense. IIRC, most people don't even bother to use widely available 1U servers because you run out of power before you fill up 40U racks. Hyper…

Ah, thanks for the details, I wasn't referring to NUMA-induced lock contention, but rather lock contention in general. I've seen a workload hit 64 CPUs with 80% of CPU time in lock contention (and others in the 10-30%). Now, while that means the developer has a big problem to fix, it also has me wondering about single sockets getting big enough -- 128 CPUs is already hard to use well. Back when I first saw multi-sock…

But its the same in GPU-land.

There are plenty of tasks which max out at one block / threadgroup of 1024 CUDA-threads. cudaMemcpy is a silly example (probably maxes out memory bandwidth at just 64 or 32 cores used), but there are plenty of tasks that simply don't scale to the full use of a GPU.

Just because some tasks (many tasks?) fail to use more than 32-GPU cores doesn't mean that GPU-parallelism is useless. It just means that when you program those particular tasks, only use 32-GPU cores!! Then use the GPU-cores on _other_ tasks (possibly in parallel).

IIRC, cudaMalloc, and many other primitives in the CUDA framework, has been shown to have very little parallelism at all. You need to work at keeping this "sequential-code" outside of your inner loops. (Runs on CUDA-stream #0, which for older hardware at least is sequentially scheduled)

----------

1. Some tasks can effectively use infinite cores (SIMD-threads really for GPUs... but same idea since a SIMD-lane can largely emulate a thread as long as you're careful about branch divergence)

2. Some tasks can be parallelized at the application / operator level. Run many applications in parallel ("Makefile parallelism")

3. Some tasks (memcpy) are so memory-bound that parallelism will never help.

4. Some tasks have a better solution that becomes feasible with more compute power.

---------

Lets take a CPU example: H.264 encoding. IIRC, this task barely scales to 8 cores and has diminishing returns beyond that.

But an example of #2 would be Youtube: you have one encoding machine that handles transcoding in parallel. You don't run just 1 instance of the problem (using 8 cores), you run 32 in parallel, and each of those 32-instances can effectively use 8-cores for H264 encoding.

And #4 can still happen: H264 is pretty easy for modern computers with little chance of parallelism. Switching to H.265 or even to AV1 will increase the compute power needed, and allow scaling of single tasks up to 16 to 64 cores. Now your 2x128 hypothetical machine can only run 4 transcoding sessions at a time.

-----------

The dual-socket machine for a transcoding cluster is still superior over a single-socket machine. 10Gbit Ethernet is more than sufficient to handle 4x AV1 sessions (especially because AV1 is slower than realtime), so right there we've cut the number of Ethernet cables in half, which means we've cut the number of 10 Gbit Switches in half.

Having the two sockets share one Ethernet port is an efficiency gain even if you don't have any task-communication going on: if only for the I/O sharing capability of the NUMA Fabric.

Re: Computing Performance on the Horizon

#75

Post by rbanffy »

dragontamer wrote: post #70
nine_k wrote: post #65

Earlier quoted context omitted.

OK, it must be a compute-heavy load that does little random memory access, but works mostly on compact in-cache structures and maybe does sustained sequential memory accesses. With that, it's not suitable to offload to the GPU. What could it be? Serious question.

Database with a large portion of data in-memory. * Second socket increases the memory channels and RAM available: 16-channel dual-EPYC with 8TB of RAM will be faster than 4TB of RAM on single-EPYC 8-channel. * SQL optimizers automatically search for sequential scans, because sequential scans are faster. * While JOIN can be done in GPU space, GPUs have extremely low memory capacity (only 80GB on the latest A100 that c…

More sockets add memory controllers, but we can also think about moving HBM closer to the cores as a L4 cache or scratch memory that’s not expected to be synchronised with other cores/sockets.
Post reply on HN