Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computing Performance on the Horizon

brendangregg.com

Post reply on HN
Print view View original

21–30 of 75 posts

Re: Computing Performance on the Horizon

#21

Post by volta83 »

dragontamer wrote: post #11

FPGAs from Xilinx are very complicated. They are no longer homogeneous 4-LUTs or 6LUTs with dedicated multipliers here and there. Today's FPGAs are VLIW minicores capable of SIMD execution with custom routing and some LUTs thrown around. They've stepped towards GPU style architecture while retaining the custom logic portions. FPGAs remain so difficult to use, I find it unlikely that they'd be mainstream in any capaci…

> If your core is 50% waiting on RAM, then SMT into another thread to perform work while waiting on RAM.

If your core is 50% waiting on RAM, then SMT into another thread, and that other thread will want some memory to work on, so it will also wait on RAM. On Top of it, this second thread now puts extra pressure on the memory subsystem, might cause cache evictions for the other thread, etc etc etc

The moment that you include the memory subsystem into the SMT picture, SMT goes from a "no brainer; waiting on memory? do other work" to a "uhhh... i don't know if this makes things better or worse".

Re: Computing Performance on the Horizon

#22

Post by dragontamer »

volta83 wrote: post #21
dragontamer wrote: post #11

FPGAs from Xilinx are very complicated. They are no longer homogeneous 4-LUTs or 6LUTs with dedicated multipliers here and there. Today's FPGAs are VLIW minicores capable of SIMD execution with custom routing and some LUTs thrown around. They've stepped towards GPU style architecture while retaining the custom logic portions. FPGAs remain so difficult to use, I find it unlikely that they'd be mainstream in any capaci…

> If your core is 50% waiting on RAM, then SMT into another thread to perform work while waiting on RAM. If your core is 50% waiting on RAM, then SMT into another thread, and that other thread will want some memory to work on, so it will also wait on RAM. On Top of it, this second thread now puts extra pressure on the memory subsystem, might cause cache evictions for the other thread, etc etc etc The moment that you…

Not quite.

DDR4 and DDR5 have 50ns (single socket) to 150ns (dual socket) latency.

For a 3GHz processor, that's 150 to 450 cycles.

On any latency bound problem, SMT helps. However, what you say is true on bandwidth bound problems. Given the shear number of pointer hopping that happens in typical OOP code these days (or python / JavaScript), I expect SMT to be of big help to typical applications.

DDR5 will double bandwidth in the near future. But that's not enough: HBM and GDDR6 have a possible future because you can only solve the bandwidth problem with more hardware. No tricks like SMT can help.

Re: Computing Performance on the Horizon

#23

Post by eloff »

dragontamer wrote: post #13
eloff wrote: post #12

Earlier quoted context omitted.

Yeah, it makes a lot of sense to go with single socket servers unless you can't scale horizontally (e.g. database server). Why deal with the complexity when you can just side step it.

Why would you switch from a 100GBps NUMA connection (800 gigabits per second) over NUMA fabric into a 10 Gbps Ethernet fabric? If you are scaling horizontally, NUMA is the superior fabric than Ethernet or Infiniband (100Gbps) Horizontal scaling seems to favor NUMA. 1000 chips over Ethernet is less efficient than 500 dual socket nodes over Ethernet. Anything you can do over Ethernet seems easier and cheaper over NUMA…

I'm talking mostly abour scaling things like app servers where they might not need any communication.

But in general if you can't scale horizontally at 10 gbps, you're in for a world of hurt. Numa gets you to 8x scale at best on very expensive very exotic hardware. And then you hit the wall.

Re: Computing Performance on the Horizon

#24

Post by solarkennedy »

wmf wrote: post #6
soulbadguy wrote: post #4

Earlier quoted context omitted.

In world where most work loads are containerized, and where each container can be pinned to numa region doesn't it really matter?

Does any container runtime/orchestrator perform this optimization yet? Why wait?

Titus (Netflix's container orchestrator that I work on) does this via: https://github.com/Netflix-Skunkworks/titus-isolate

Re: Computing Performance on the Horizon

#25

Post by dragontamer »

eloff wrote: post #23
dragontamer wrote: post #13

Earlier quoted context omitted.

Why would you switch from a 100GBps NUMA connection (800 gigabits per second) over NUMA fabric into a 10 Gbps Ethernet fabric? If you are scaling horizontally, NUMA is the superior fabric than Ethernet or Infiniband (100Gbps) Horizontal scaling seems to favor NUMA. 1000 chips over Ethernet is less efficient than 500 dual socket nodes over Ethernet. Anything you can do over Ethernet seems easier and cheaper over NUMA…

I'm talking mostly abour scaling things like app servers where they might not need any communication. But in general if you can't scale horizontally at 10 gbps, you're in for a world of hurt. Numa gets you to 8x scale at best on very expensive very exotic hardware. And then you hit the wall.

I'm mostly talking about 2 socket servers, which are IIRC more common than even single socket servers.

Dual socket is a cheap, easy, and common. If only to recycle fans, power supplies and racks, it seems useful.

Re: Computing Performance on the Horizon

#26

Post by thechao »

martinpw wrote: post #10

Slide 26 is interesting - arguing that cloud providers have an advantage for future CPU design since they can analyze so many real world customer workloads directly. In previous roles I have worked with CPU vendors who have been very keen on getting access to profiling data from our workloads for design optimization, and lamenting the fact that it was hard to get such data and they were often limited to synthetic ben…

He sort of implies that “just better hardware” will peter out in the 2030s. I think he’s calling it at least 50 years too soon. Here’s why: (1) I think logic designers are still faffing about in term of optimizing their designs; and (2), I think there’s a lot of smart people thinking “incrementally” through what we’d consider paradigm shifts in HW implementation. That is, our fabs will just naturally segue into 3D, spintronics, etc. I think he even mentions 3D circuits? One thing a lot of people miss is that layout of the design is materially different in 3D vs 2D: in 2D layout is NP-hard (complete) without efficient polynomial approximations; in 3D layout is low-order polynomial. The reduction in layout complexity will allow us to design things that are unthinkable right now, due to layout constraints & wire congestion.

Re: Computing Performance on the Horizon

#27

Post by ksec »

>for storage including new uses for 3D Xpoint as a 3D NAND accelerator;

3D XPoint's future is not entirely certain. Intel with their new CEO has remained rather quiet on the subject. Micron are pulling the plug on it and sold the Fab to Texas Instrument. The problem is there isn't a clear path forward with the technology, it make some sense when NAND and DRAM price were high in 2016 - 2019. Once they dropped to a normal level with newer DDR5 and faster SLC NAND or ZNAND with lower latency than XPoint's cost benefits becomes unclear. I guess we will know once Intel's Optane P5800X [1] is out with review. It is quite a beast.

>Multi-Socket is Doomed

Are there really no use-case where 128 Core+ with NUMA offer some advantage?

>Slower Rotational

Seagate [2] is actually working on dual Actuator HDD, think of it as something like internal RAID 0. The rational being as HDD gets bigger the time to fill up those drive increases as well.

>ARM on Cloud

Marvell partly confirms all HyperScalers have intention to build their own ARM CPU. But Google just announced their Tau instances [3], effectively cutting their cost / pref by 50%. Where each vCPU is an entire physical CPU core rather than a x86 thread.

Not much mention on GPGPU.

[1] https://www.intel.com/content/www/us/en/products/docs/memory...

[2] https://www.anandtech.com/show/16544/seagates-roadmap-120-tb...

[3] https://cloud.google.com/blog/products/compute/google-cloud-...

Re: Computing Performance on the Horizon

#28

Post by zozbot234 »

zaroth wrote: post #15
syoc wrote: post #9

Earlier quoted context omitted.

Rack space can be quite expensive. Sometimes you need a lot of computing power in one or two rack units. Would be interested in what the management pains are. I agree that 2 socket machines require more thought in a lot of scenarios, especially IO heavy workloads.

In my very limited experience it seems like space is much less an issue than power density. You can fit far more kW/U than the datacenter can possibly cool. In the commodity space that I rent, I ran out of power before filling even half the rack. I’m sure higher power/cooling density is possible to obtain, but I would think you’re primarily paying for that versus square footage?

What do you need that power density for? It's a rack, not a supercomputer. (I sure hope it's not "mining coins" or anything like that.)

Re: Computing Performance on the Horizon

#29

Post by zozbot234 »

dragontamer wrote: post #22
volta83 wrote: post #21

Earlier quoted context omitted.

> If your core is 50% waiting on RAM, then SMT into another thread to perform work while waiting on RAM. If your core is 50% waiting on RAM, then SMT into another thread, and that other thread will want some memory to work on, so it will also wait on RAM. On Top of it, this second thread now puts extra pressure on the memory subsystem, might cause cache evictions for the other thread, etc etc etc The moment that you…

Not quite. DDR4 and DDR5 have 50ns (single socket) to 150ns (dual socket) latency. For a 3GHz processor, that's 150 to 450 cycles. On any latency bound problem, SMT helps. However, what you say is true on bandwidth bound problems. Given the shear number of pointer hopping that happens in typical OOP code these days (or python / JavaScript), I expect SMT to be of big help to typical applications. DDR5 will double band…

Memory bandwidth is just barely trying to keep up with cores, frequencies and IPC amounts. Bandwidth available per core is still going to drop. So newer development workflows that optimize for this bottleneck are going to be very relevant.

Re: Computing Performance on the Horizon

#30

Post by Robotbeat »

dragontamer wrote: post #11

FPGAs from Xilinx are very complicated. They are no longer homogeneous 4-LUTs or 6LUTs with dedicated multipliers here and there. Today's FPGAs are VLIW minicores capable of SIMD execution with custom routing and some LUTs thrown around. They've stepped towards GPU style architecture while retaining the custom logic portions. FPGAs remain so difficult to use, I find it unlikely that they'd be mainstream in any capaci…

FPGAs are hard to work with in part because the tools are extremely proprietary. But that is starting to change. Open source FPGA tools are becoming more common and more powerful.
Post reply on HN