Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computing Performance on the Horizon

brendangregg.com

Post reply on HN
Print view View original

51–60 of 75 posts

Re: Computing Performance on the Horizon

#51

Post by brendangregg »

dragontamer wrote: post #41
infogulch wrote: post #38

Earlier quoted context omitted.

> Are there really no use-case where 128 Core+ with NUMA offer some advantage? Are there any use cases where 128+ core single socket wouldn't be preferred to a 128+ core multiple socket design that is burdened by NUMA? AMD has been showing us that integrating the interconnects into the CPU package directly and letting it handle all the issues is a better design.

When a hypothetical 128-core single socket comes out, will there be no workload that prefers to use a 2x128-core dual socket instead? AMD CPUs remain largely dual-socket compatible. Today's 64-core EPYCs can be dual-socketed into 2x64-core beasts. It just seems silly to me that if you're building say 200 computers in 10x racks (20-computers per 10x 40U racks) that you'd prefer single socket over dual-socket. If you'r…

I don't have a workload I'd prefer to see on 2x128-core: We're already microservices running across a pool of instances, and would prefer a bigger pool of faster instances than a smaller pool of slower ones at the same cost. Once we get a workload running on 100+ cores, I often see a lot of lock contention anyway. Going bigger usually makes that worse (worse ROI).

As for datacenter size/cost, it's a good point, but what if two 1-Socket servers could take up the same space as one 2-Socket server? :-) That may never happen, but some level of space optimization will, so it's not a simple doubling of size. E.g., Facebook's work in the OCP with 1-socket sleds (or blades):

https://www.opencompute.org/documents/ocp-yosemite-v3-platfo...

Re: Computing Performance on the Horizon

#52

Post by toast0 »

jeffbee wrote: post #2

A delightful set of slides and references that tickles many of my pet topics. In particular, one I’d love to hear more about is why so many deployments are still choosing 2-socket servers by default when managing them is such a pain in the neck and the performance when you do it badly is so poor. Live the life of the future, today: choose single sockets!

All things considered, managing fewer hosts is nicer than more hosts.

In some hardware generations, dual socket has pretty good cost and complexity tradeoffs. And if you benefit from having a large dataset in memory on a single machine, dual socket often gets you twice the DIMM sockets and therefore twice the ram. Quad socket has been very expensive (and not great performance) for quite some time, so that's usually out.

Single socket Epyc looks pretty impressive though; although I'm retired and probably won't get to work with those anytime soon.

Re: Computing Performance on the Horizon

#53

Post by dragontamer »

brendangregg wrote: post #51
dragontamer wrote: post #41

Earlier quoted context omitted.

When a hypothetical 128-core single socket comes out, will there be no workload that prefers to use a 2x128-core dual socket instead? AMD CPUs remain largely dual-socket compatible. Today's 64-core EPYCs can be dual-socketed into 2x64-core beasts. It just seems silly to me that if you're building say 200 computers in 10x racks (20-computers per 10x 40U racks) that you'd prefer single socket over dual-socket. If you'r…

I don't have a workload I'd prefer to see on 2x128-core: We're already microservices running across a pool of instances, and would prefer a bigger pool of faster instances than a smaller pool of slower ones at the same cost. Once we get a workload running on 100+ cores, I often see a lot of lock contention anyway. Going bigger usually makes that worse (worse ROI). As for datacenter size/cost, it's a good point, but w…

> As for datacenter size/cost, it's a good point, but what if two 1-Socket servers could take up the same space as one 2-Socket server? :-) That may never happen, but some level of space optimization will

Oh it certainly exists. Computers are space-optimized to the point of nonsense. IIRC, most people don't even bother to use widely available 1U servers because you run out of power before you fill up 40U racks.

Hyperscalers, such as Google / Netflix / Amazon are a bit different of course (IIRC, you work at one right?), since they can specially build their data centers to have far denser power-delivery and actually support 40-computers or even 80-computers per rack. But more typical offices simply do not have the power-density to run 1U nodes or smaller (Ex: Supermicro 2xNodes in 1U nodes or Supermicro 4xNode in 2U).

In effect: modern computer systems usually run out of power before they run out of rack space. Especially when you consider that every Watt-delivered turns into Heat (Watts) generated, which then requires a more powerful air-conditioner to keep the room within operating specs.

So you're right that modern datacenters probably don't care about size. Space is relatively cheap, power-lines are expensive! 2x Sockets for 2x per 1U == 160 CPUs per 40U rack. 10 such racks would use over a Megawatt of power once we factor in air conditioning, so a typical building just won't handle that.

-------------

> I don't have a workload I'd prefer to see on 2x128-core: We're already microservices running across a pool of instances, and would prefer a bigger pool of faster instances than a smaller pool of slower ones at the same cost. Once we get a workload running on 100+ cores, I often see a lot of lock contention anyway. Going bigger usually makes that worse (worse ROI).

But that "lock contention" you're measuring is something like 250ns to 500ns over a channel that's 800Gbit/sec thick.

EDIT: To be more specific: I'm talking about the MESI messages going over the NUMA fabric.

In contrast, a packet over 10 Gbit Ethernet is basically two orders of magnitude less bandwidth and an order of magnitude more latency (maybe 2500 to 5000 nanoseconds of latency?).

If the application were truly limited by the communication paradigm between the NUMA Fabric, switching to Ethernet or InfiniBand would only slow it down further.

EDIT: Case in point: we don't do spinlocks over Ethernet. I mean, we could in theory (RDMA a region of memory over Ethernet and then hold it as a Spinlock), but we all know its a bad idea. We do spinlocks at L3 and/or the NUMA Fabric level (and maybe we'll do it over PCIe 5.0 / CCIX level, as cache-coherent I/O becomes possible).

Re: Computing Performance on the Horizon

#54

Post by drewg123 »

dragontamer wrote: post #41
infogulch wrote: post #38

Earlier quoted context omitted.

> Are there really no use-case where 128 Core+ with NUMA offer some advantage? Are there any use cases where 128+ core single socket wouldn't be preferred to a 128+ core multiple socket design that is burdened by NUMA? AMD has been showing us that integrating the interconnects into the CPU package directly and letting it handle all the issues is a better design.

When a hypothetical 128-core single socket comes out, will there be no workload that prefers to use a 2x128-core dual socket instead? AMD CPUs remain largely dual-socket compatible. Today's 64-core EPYCs can be dual-socketed into 2x64-core beasts. It just seems silly to me that if you're building say 200 computers in 10x racks (20-computers per 10x 40U racks) that you'd prefer single socket over dual-socket. If you'r…

There are a few problems with dual sockets:

- The flip side of cuts the computers you need to work with in half is that it doubles the blast radius in case of PSU/fan/mobo/etc failure

- If you're interested in I/O, dual sockets can be problematic because few motherboards are "balanced" with an equal number of PCIe slots local to each socket.

- NUMA makes everything harder. Even after the work that I've done to make NUMA useful for Netflix's Open Connect (CDN) on FreeBSD, I'd very much rather just use flat machines wherever I can. NUMA gives lots of opportunities for comically bad performance if any little thing is placed incorrectly.

Re: Computing Performance on the Horizon

#55

Post by dragontamer »

blt wrote: post #50
yjftsjthsd-h wrote: post #33

Earlier quoted context omitted.

> in 2D layout is NP-hard (complete) without efficient polynomial approximations; in 3D layout is low-order polynomial Any chance you could explain to a novice why 3D is easier? To my naive intuition, it would have seemed like the more room to maneuver is offset by having more stuff to route.

Intuitively, we feel that larger solution spaces should make a problem harder because there are more possible solutions to consider. And of course this is true if exhaustive search is the only algorithm. But CS is full of problems where the smaller space is NP-hard and the larger one isn't. Integer linear programming is a prominent example. To resolve the intuition, we can think of the larger space as an "unconstrain…

> Adding constraints usually makes problems harder.

Adding constraints can also make a problem easier. Which is why you've carefully worded your sentence in this manner :-)

You clearly know about those edge cases, but I felt it necessary to elaborate on this point: Additional constraints can make a problem easier OR harder, depending on the problem and the search space. Its very non-intuitive.

Re: Computing Performance on the Horizon

#56

Post by ClumsyPilot »

thechao wrote: post #40
nradov wrote: post #32

Earlier quoted context omitted.

How are we going to cool those 3D chips?

The commenter above is correct: just stop toggling HW. We already do this to a great extent; we’re limited in the number of custom implementations because we can’t wire everything together. 3D chips will have a lot more “dark” logic than current chips, but will be orders-of-magnitude more efficient (& thus powerful) due to deep customization. Also, remember the argument of my timeline is ~50–80 years out from now.

This is totally wishfull thinking. Customised logix means customised code.

The GPU has an h265 and h264 hardware video encoder that has no support on Mac and ia a bitch to get working on linux. Most software doesn't support it most software does not support GPU compute either, and we had that for a decade now.

Fuck, we had SIMD in every CPU for like 30 years, and out of 50 most popular programming languages how many even support it? 5?

Re: Computing Performance on the Horizon

#57

Post by [deleted] »

dragontamer wrote: post #53
brendangregg wrote: post #51

Earlier quoted context omitted.

I don't have a workload I'd prefer to see on 2x128-core: We're already microservices running across a pool of instances, and would prefer a bigger pool of faster instances than a smaller pool of slower ones at the same cost. Once we get a workload running on 100+ cores, I often see a lot of lock contention anyway. Going bigger usually makes that worse (worse ROI). As for datacenter size/cost, it's a good point, but w…

> As for datacenter size/cost, it's a good point, but what if two 1-Socket servers could take up the same space as one 2-Socket server? :-) That may never happen, but some level of space optimization will Oh it certainly exists. Computers are space-optimized to the point of nonsense. IIRC, most people don't even bother to use widely available 1U servers because you run out of power before you fill up 40U racks. Hyper…

[deleted]

Re: Computing Performance on the Horizon

#58

Post by pm90 »

brendangregg wrote: post #46
martinpw wrote: post #10

Slide 26 is interesting - arguing that cloud providers have an advantage for future CPU design since they can analyze so many real world customer workloads directly. In previous roles I have worked with CPU vendors who have been very keen on getting access to profiling data from our workloads for design optimization, and lamenting the fact that it was hard to get such data and they were often limited to synthetic ben…

Thanks, I think it's the start of a new "cloud CPU" era. Cloud vendors have already split workloads into different instance types, so directly analyzing their workloads and developing CPUs for each instance type will lead to further performance wins. This may prompt the creation of even more instance types just to further separate workload types for future CPU specialization. In the future, products like AWS outposts…

In hindsight, I wonder why Intel or other chip makers didn’t invest more in datacenters/hosting companies just to get this valuable data.

Re: Computing Performance on the Horizon

#60

Post by AprilArcus »

nradov wrote: post #32
thechao wrote: post #26

Earlier quoted context omitted.

He sort of implies that “just better hardware” will peter out in the 2030s. I think he’s calling it at least 50 years too soon. Here’s why: (1) I think logic designers are still faffing about in term of optimizing their designs; and (2), I think there’s a lot of smart people thinking “incrementally” through what we’d consider paradigm shifts in HW implementation. That is, our fabs will just naturally segue into 3D, s…

How are we going to cool those 3D chips?

could it be fabbed around a lattice of thermally conductive elements, like a menger sponge?
Post reply on HN