Computing Performance on the Horizon • Hacker News Board

Re: Computing Performance on the Horizon

#11

Post by dragontamer » Mon, Jul 05, 2021, 5:58 PM UTC

FPGAs from Xilinx are very complicated. They are no longer homogeneous 4-LUTs or 6LUTs with dedicated multipliers here and there.

Today's FPGAs are VLIW minicores capable of SIMD execution with custom routing and some LUTs thrown around. They've stepped towards GPU style architecture while retaining the custom logic portions.

FPGAs remain so difficult to use, I find it unlikely that they'd be mainstream in any capacity. GPUs seem like the easier way to get access to HBM + heavy compute, but either way the HBM future is eminent.

------------

GPUs have big questions about ease of use and practicality as it is, even with widespread acceptance of their compute potential. FPGAs are much less known, it's hard for me to imagine a mainstream future of them.

Since memory bounds remains the biggest issue and not compute performance, I bet that the easiest to use accelerator with mass production and cheap access to the highest speed HBM is going to be the winner. GPUs are the current frontrunner, but the Fujitsu ARM CPU has easy access to HBM and could be a wildcard.

POWER10 will be using high performance GDDR6. Not quite HBM, but it signals that IBM is also concerned with the memory bandwidth problem in the near future.

CPUs could very well switch to HBM in some scenarios.

------------

If I were to guess the future: I think that AMD and NVidia have proven that today's systems need high speed routers to practically scale

AMD has their IO die on EPYC. NVidia has NVLink and NVSwitch. That seems to be how to get more dies / sockets without additional NUMA hops.

More efficient networks of chips with explicit switching / routing topologies is the only way to scale. The exact form of this network is still a mystery, but that's my big bet for the future.

HBM is probably the future for high performance. DDR5 for cheaper bulk RAM but HBM on high performance CPUs / GPUs / FPGAs is going to be key.

---------

The insight into RAM bottlenecks is interesting but seems to be point in favor of SMT. If your core is 50% waiting on RAM, then SMT into another thread to perform work while waiting on RAM.

Re: Computing Performance on the Horizon

#12

Post by eloff » Mon, Jul 05, 2021, 5:59 PM UTC

jeffbee wrote: post #7

soulbadguy wrote: post #4

Earlier quoted context omitted.

In world where most work loads are containerized, and where each container can be pinned to numa region doesn't it really matter?

k8s, by default, is oblivious to NUMA topology. You have to enable unreleased features and configure them correctly, which is the unwanted complexity to which I referred earlier. Simply aligning your containers to NUMA domains does not solve the problem that your arriving network frames or your NVMe completion queues can still be on the wrong domain. Isn't it simpler to just have 1 socket and not need to care? The nu…

Yeah, it makes a lot of sense to go with single socket servers unless you can't scale horizontally (e.g. database server). Why deal with the complexity when you can just side step it.

Re: Computing Performance on the Horizon

#13

Post by dragontamer » Mon, Jul 05, 2021, 6:13 PM UTC

eloff wrote: post #12

jeffbee wrote: post #7

Earlier quoted context omitted.

k8s, by default, is oblivious to NUMA topology. You have to enable unreleased features and configure them correctly, which is the unwanted complexity to which I referred earlier. Simply aligning your containers to NUMA domains does not solve the problem that your arriving network frames or your NVMe completion queues can still be on the wrong domain. Isn't it simpler to just have 1 socket and not need to care? The nu…

Yeah, it makes a lot of sense to go with single socket servers unless you can't scale horizontally (e.g. database server). Why deal with the complexity when you can just side step it.

Why would you switch from a 100GBps NUMA connection (800 gigabits per second) over NUMA fabric into a 10 Gbps Ethernet fabric?

If you are scaling horizontally, NUMA is the superior fabric than Ethernet or Infiniband (100Gbps)

Horizontal scaling seems to favor NUMA. 1000 chips over Ethernet is less efficient than 500 dual socket nodes over Ethernet. Anything you can do over Ethernet seems easier and cheaper over NUMA instead.

Re: Computing Performance on the Horizon

#14

Post by wmf » Mon, Jul 05, 2021, 6:20 PM UTC

dragontamer wrote: post #13

eloff wrote: post #12

Earlier quoted context omitted.

Yeah, it makes a lot of sense to go with single socket servers unless you can't scale horizontally (e.g. database server). Why deal with the complexity when you can just side step it.

Why would you switch from a 100GBps NUMA connection (800 gigabits per second) over NUMA fabric into a 10 Gbps Ethernet fabric? If you are scaling horizontally, NUMA is the superior fabric than Ethernet or Infiniband (100Gbps) Horizontal scaling seems to favor NUMA. 1000 chips over Ethernet is less efficient than 500 dual socket nodes over Ethernet. Anything you can do over Ethernet seems easier and cheaper over NUMA…

This is correct if your software is NUMA-optimized (or if auto-NUMA works well for you) but if it isn't you can end up with slowdowns.

Re: Computing Performance on the Horizon

#15

Post by zaroth » Mon, Jul 05, 2021, 6:24 PM UTC

syoc wrote: post #9

jeffbee wrote: post #2

A delightful set of slides and references that tickles many of my pet topics. In particular, one I’d love to hear more about is why so many deployments are still choosing 2-socket servers by default when managing them is such a pain in the neck and the performance when you do it badly is so poor. Live the life of the future, today: choose single sockets!

Rack space can be quite expensive. Sometimes you need a lot of computing power in one or two rack units. Would be interested in what the management pains are. I agree that 2 socket machines require more thought in a lot of scenarios, especially IO heavy workloads.

In my very limited experience it seems like space is much less an issue than power density.

You can fit far more kW/U than the datacenter can possibly cool.

In the commodity space that I rent, I ran out of power before filling even half the rack. I’m sure higher power/cooling density is possible to obtain, but I would think you’re primarily paying for that versus square footage?

Re: Computing Performance on the Horizon

#16

Post by handrous » Mon, Jul 05, 2021, 6:29 PM UTC

martinpw wrote: post #10

Slide 26 is interesting - arguing that cloud providers have an advantage for future CPU design since they can analyze so many real world customer workloads directly. In previous roles I have worked with CPU vendors who have been very keen on getting access to profiling data from our workloads for design optimization, and lamenting the fact that it was hard to get such data and they were often limited to synthetic ben…

An interesting application of a now-familiar pattern: get lots of users, spy on them at massive scale, use those data to dominate some other market in a way that, at most, a single digit count of companies in the world could conceivably compete with (because none but they have anything like the data that you do). See also: everything to do with "AI".

Re: Computing Performance on the Horizon

#17

Post by kortilla » Mon, Jul 05, 2021, 6:30 PM UTC

dragontamer wrote: post #13

eloff wrote: post #12

Earlier quoted context omitted.

Yeah, it makes a lot of sense to go with single socket servers unless you can't scale horizontally (e.g. database server). Why deal with the complexity when you can just side step it.

Why would you switch from a 100GBps NUMA connection (800 gigabits per second) over NUMA fabric into a 10 Gbps Ethernet fabric? If you are scaling horizontally, NUMA is the superior fabric than Ethernet or Infiniband (100Gbps) Horizontal scaling seems to favor NUMA. 1000 chips over Ethernet is less efficient than 500 dual socket nodes over Ethernet. Anything you can do over Ethernet seems easier and cheaper over NUMA…

Your scaling architecture sucks if it depends on that kind of throughput. If you need that you’ve only can kicked your way to more capacity without a real scaling fix.

Re: Computing Performance on the Horizon

#18

Post by dragontamer » Mon, Jul 05, 2021, 6:33 PM UTC

wmf wrote: post #14

dragontamer wrote: post #13

Earlier quoted context omitted.

Why would you switch from a 100GBps NUMA connection (800 gigabits per second) over NUMA fabric into a 10 Gbps Ethernet fabric? If you are scaling horizontally, NUMA is the superior fabric than Ethernet or Infiniband (100Gbps) Horizontal scaling seems to favor NUMA. 1000 chips over Ethernet is less efficient than 500 dual socket nodes over Ethernet. Anything you can do over Ethernet seems easier and cheaper over NUMA…

This is correct if your software is NUMA-optimized (or if auto-NUMA works well for you) but if it isn't you can end up with slowdowns.

Surely that can be fixed with just a well placed numactl command to set node affinity and CPU affinity.

The root article is discussing rewriting code to fit on FPGAs. If NUMA is too complex then... I dunno. The FPGA argument seems dead on arrival.

Re: Computing Performance on the Horizon

#19

Post by jeffbee » Mon, Jul 05, 2021, 6:41 PM UTC

syoc wrote: post #9

jeffbee wrote: post #2

A delightful set of slides and references that tickles many of my pet topics. In particular, one I’d love to hear more about is why so many deployments are still choosing 2-socket servers by default when managing them is such a pain in the neck and the performance when you do it badly is so poor. Live the life of the future, today: choose single sockets!

Rack space can be quite expensive. Sometimes you need a lot of computing power in one or two rack units. Would be interested in what the management pains are. I agree that 2 socket machines require more thought in a lot of scenarios, especially IO heavy workloads.

The OpenCompute "Delta Lake" machine mentioned in the article occupies only one third of 1RU and peaks at 400W. You will certainly be power/cooling limited, rather than volume limited, with that kind of density.

Re: Computing Performance on the Horizon

#20

Post by dragontamer » Mon, Jul 05, 2021, 6:42 PM UTC

kortilla wrote: post #17

dragontamer wrote: post #13

Earlier quoted context omitted.

Why would you switch from a 100GBps NUMA connection (800 gigabits per second) over NUMA fabric into a 10 Gbps Ethernet fabric? If you are scaling horizontally, NUMA is the superior fabric than Ethernet or Infiniband (100Gbps) Horizontal scaling seems to favor NUMA. 1000 chips over Ethernet is less efficient than 500 dual socket nodes over Ethernet. Anything you can do over Ethernet seems easier and cheaper over NUMA…

Your scaling architecture sucks if it depends on that kind of throughput. If you need that you’ve only can kicked your way to more capacity without a real scaling fix.

Depend on? Heavens no.

Dual socket has numerous advantages in density and rack space. The fact that performance is better is pretty much icing on the cake.

It's easier to manage 500 dual socket servers than 1000 single socket servers. Less equipment, higher utilization of parts, etc. Etc.

To suggest dual socket NUMA is going away is... just very unlikely to me. I don't see what the benefits would be at all. Not just performance, but also routine maintenance issues (power, Ethernet, local storage, etc etc)