Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computing Performance on the Horizon

brendangregg.com

Post reply on HN
Print view View original

61–70 of 75 posts

Re: Computing Performance on the Horizon

#61

Post by RantyDave »

ClumsyPilot wrote: post #56
thechao wrote: post #40

Earlier quoted context omitted.

The commenter above is correct: just stop toggling HW. We already do this to a great extent; we’re limited in the number of custom implementations because we can’t wire everything together. 3D chips will have a lot more “dark” logic than current chips, but will be orders-of-magnitude more efficient (& thus powerful) due to deep customization. Also, remember the argument of my timeline is ~50–80 years out from now.

This is totally wishfull thinking. Customised logix means customised code. The GPU has an h265 and h264 hardware video encoder that has no support on Mac and ia a bitch to get working on linux. Most software doesn't support it most software does not support GPU compute either, and we had that for a decade now. Fuck, we had SIMD in every CPU for like 30 years, and out of 50 most popular programming languages how many…

I would argue that most software doesn't need GPU compute.

Re: Computing Performance on the Horizon

#62

Post by [deleted] »

zozbot234 wrote: post #28
zaroth wrote: post #15

Earlier quoted context omitted.

In my very limited experience it seems like space is much less an issue than power density. You can fit far more kW/U than the datacenter can possibly cool. In the commodity space that I rent, I ran out of power before filling even half the rack. I’m sure higher power/cooling density is possible to obtain, but I would think you’re primarily paying for that versus square footage?

What do you need that power density for? It's a rack, not a supercomputer. (I sure hope it's not "mining coins" or anything like that.)

[deleted]

Re: Computing Performance on the Horizon

#63

Post by wmf »

ineedasername wrote: post #59

3d CPU stacking seems interesting where surface area is a limited resource, but otherwise it seems like it would significantly complicate cooling things efficiently. Or isy assumption wrong?

You're right; you don't want to stack hot silicon on top of other hot silicon.

Re: Computing Performance on the Horizon

#64

Post by brendangregg »

pm90 wrote: post #58
brendangregg wrote: post #46

Earlier quoted context omitted.

Thanks, I think it's the start of a new "cloud CPU" era. Cloud vendors have already split workloads into different instance types, so directly analyzing their workloads and developing CPUs for each instance type will lead to further performance wins. This may prompt the creation of even more instance types just to further separate workload types for future CPU specialization. In the future, products like AWS outposts…

In hindsight, I wonder why Intel or other chip makers didn’t invest more in datacenters/hosting companies just to get this valuable data.

Yes, this was in my predictions: Processor vendors may offer their own clouds (either built or acquired) to get the same level of workload access.

Re: Computing Performance on the Horizon

#65

Post by nine_k »

rbanffy wrote: post #43

I have an enormous respect for Brandon Gregg, but this "one socket ought to be enough for anyone" is something I saw too many people get burned with. I mean, it should, but who knows what the next version of Slack will need...

OK, it must be a compute-heavy load that does little random memory access, but works mostly on compact in-cache structures and maybe does sustained sequential memory accesses. With that, it's not suitable to offload to the GPU.

What could it be? Serious question.

Re: Computing Performance on the Horizon

#66

Post by oscardssmith »

zozbot234 wrote: post #29
dragontamer wrote: post #22

Earlier quoted context omitted.

Not quite. DDR4 and DDR5 have 50ns (single socket) to 150ns (dual socket) latency. For a 3GHz processor, that's 150 to 450 cycles. On any latency bound problem, SMT helps. However, what you say is true on bandwidth bound problems. Given the shear number of pointer hopping that happens in typical OOP code these days (or python / JavaScript), I expect SMT to be of big help to typical applications. DDR5 will double band…

Memory bandwidth is just barely trying to keep up with cores, frequencies and IPC amounts. Bandwidth available per core is still going to drop. So newer development workflows that optimize for this bottleneck are going to be very relevant.

One thing that I think this will mean is the end of blas. Blas served us well for about 50 years, but one of the big problems it has is that it leads to code that takes multiple passes over memory. I think the future lies in systems that do code generation to better fuse loops of arbitrary code together. LoopVectorization.jl for example is able to generate blas level code for arbitrary computation, and as such can often be faster than blas since you don't have to use one of the specific hand optimized kernels.

Re: Computing Performance on the Horizon

#67

Post by rektide »

jeffbee wrote: post #2

A delightful set of slides and references that tickles many of my pet topics. In particular, one I’d love to hear more about is why so many deployments are still choosing 2-socket servers by default when managing them is such a pain in the neck and the performance when you do it badly is so poor. Live the life of the future, today: choose single sockets!

> In particular, one I’d love to hear more about is why so many deployments are still choosing 2-socket servers by default

bit of a guess but connectivity has been coming at a stupidly high premium for too long. there are sweet sweet blade chassis with price optimized less than full power 1p designs but 10g is still kind of novel there. bigger form factors are starting to see 25gbit at not-astronomical prices & switches in some rare cases are reasonable too.

power supplies, storage, networking... a computer has a lot of not-entirely-ancillary needs. having multiple chips sharing the perhipetals should make sense, should be cheap. it's not though. the SMP tax is huge huge huge.

thing is we don't need smp. we just need multihost peripherals. we needs nic's that like the grouphug ocp board can support 4 separate modes via pcie srv-io, a nice that can present multiple different virtual functions that different hosts can use. NVME similarly could be multiport- was was. power supplies are shared in ocp designs, with big bus rails, some 48v.

I'm notad yet but sure seems dead obvious to me the future of multi-socket is non-coherency. build a big board with a couple different isolated computers on it, but connected via shared nic or nics to the top-of-rack. we get close with the 3 per width ope compute systems but those each need to be self contained, and there's an obvious leap in efficiency to be had by merging those three separate computers onto a single motherboard, while sharing some network, maybe storage devices. also like throw in some gratis pcie ntb maybe for a medium speed (32gbps on pcie 4.0 x16) direct server to server interconnect. ideally add another ntb unit on most chips so we can make a little mediums speed nearly free ring, or other topology.

choose single sockets but choose many of them, each sharing some common peripherals.

Re: Computing Performance on the Horizon

#68

Post by rektide »

jeffbee wrote: post #7
soulbadguy wrote: post #4

Earlier quoted context omitted.

In world where most work loads are containerized, and where each container can be pinned to numa region doesn't it really matter?

k8s, by default, is oblivious to NUMA topology. You have to enable unreleased features and configure them correctly, which is the unwanted complexity to which I referred earlier. Simply aligning your containers to NUMA domains does not solve the problem that your arriving network frames or your NVMe completion queues can still be on the wrong domain. Isn't it simpler to just have 1 socket and not need to care? The nu…

generally a huge fan of kubernetes but it's stunning what a did-it-ourselves dirtbag k8s opted to be every step of the way with regard to scheduling.

Facebook has really really good talks about managing process scheduling at scale, talking about how they leverage cgroups to do the right thing.

kubernetes seems to not give a fuck. they have their own resource systems they cooked up. shit gets scheduled in a huge massive cgroup. any order or control is userland, totally ignorant to the kernel control. there's not hierarchies, no priorities, everything is absolute, schedule or die. it's such a ginormous piece of shit, so in unbelievably willfully ignorant to all the good kernel technology that exists. it tries to make sure the kernel never has a role & that's just a huge mistake, just deeply tragic.

one noteable side effect ofany is that while the the kernel has many ways to make multi-tenant scheduling fairly reasonable, kubernetes has a variety of wild hair brained schemes, all of which detour around how easy the job would be if different pods could be scheduled in different cgroups. but that's somehow too blindingly obvious for kubernetes, which instead tries to mediate what to run entirely by itself.

Re: Computing Performance on the Horizon

#69

Post by thechao »

ClumsyPilot wrote: post #56
thechao wrote: post #40

Earlier quoted context omitted.

The commenter above is correct: just stop toggling HW. We already do this to a great extent; we’re limited in the number of custom implementations because we can’t wire everything together. 3D chips will have a lot more “dark” logic than current chips, but will be orders-of-magnitude more efficient (& thus powerful) due to deep customization. Also, remember the argument of my timeline is ~50–80 years out from now.

This is totally wishfull thinking. Customised logix means customised code. The GPU has an h265 and h264 hardware video encoder that has no support on Mac and ia a bitch to get working on linux. Most software doesn't support it most software does not support GPU compute either, and we had that for a decade now. Fuck, we had SIMD in every CPU for like 30 years, and out of 50 most popular programming languages how many…

Engineers will use the HW when it’s their only choice for year-over-year performance gains — I like to think of it as the “great unwinding”.

Re: Computing Performance on the Horizon

#70

Post by dragontamer »

nine_k wrote: post #65
rbanffy wrote: post #43

I have an enormous respect for Brandon Gregg, but this "one socket ought to be enough for anyone" is something I saw too many people get burned with. I mean, it should, but who knows what the next version of Slack will need...

OK, it must be a compute-heavy load that does little random memory access, but works mostly on compact in-cache structures and maybe does sustained sequential memory accesses. With that, it's not suitable to offload to the GPU. What could it be? Serious question.

Database with a large portion of data in-memory.

* Second socket increases the memory channels and RAM available: 16-channel dual-EPYC with 8TB of RAM will be faster than 4TB of RAM on single-EPYC 8-channel.

* SQL optimizers automatically search for sequential scans, because sequential scans are faster.

* While JOIN can be done in GPU space, GPUs have extremely low memory capacity (only 80GB on the latest A100 that costs $10,000+). CPU will be faster because you can keep a much larger dataset hot in RAM. Your 80GB of VRAM on a GPU means nothing if your dataset is in the multi-TB range. (8TB of CPU-RAM on the other hand, serves as a reasonable cache)

Post reply on HN