Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Popping the GPU Bubble

moondream.ai

Post reply on HN
Print view View original

11–20 of 53 posts

Re: Popping the GPU Bubble

#11

Post by kibibu »

SCdF wrote: post #9
nl wrote: post #3

> you find that the GPU often sits idle, not for lack of work, but because the CPU hasn't told it what to do next yet. This phenomenon is called a GPU bubble. This is true, but I've never heard anyone refer to this as a GPU bubble before. I think most people hear "GPU bubble" and think of a financial bubble of some kind.

It appears to be a real term? https://docs.vulkan.org/tutorial/latest/Synchronization/Asyn... Very odd, but perhaps more familiar to graphics programmers? I will say I'd probably call it a stall, which is exactly what the Vulkan docs call it moments later, so :shrug:

"bubble" used to be used a lot more when talking about very deep pipelines, eg Pentium 4 depth.

Re: Popping the GPU Bubble

#12

Post by [deleted] »

SCdF wrote: post #9
nl wrote: post #3

> you find that the GPU often sits idle, not for lack of work, but because the CPU hasn't told it what to do next yet. This phenomenon is called a GPU bubble. This is true, but I've never heard anyone refer to this as a GPU bubble before. I think most people hear "GPU bubble" and think of a financial bubble of some kind.

It appears to be a real term? https://docs.vulkan.org/tutorial/latest/Synchronization/Asyn... Very odd, but perhaps more familiar to graphics programmers? I will say I'd probably call it a stall, which is exactly what the Vulkan docs call it moments later, so :shrug:

[deleted]

Re: Popping the GPU Bubble

#13

Post by radq »

blueblazin wrote: post #2

I really appreciate this type of articles. I feel like a lot of knowledge in LLM training and inference is locked inside the heads of practitioners. Similar to compiler engineers before. To work in LLM training/inference you’re expected to know this stuff but to know this stuff you need to be working in the space.

Thank you for the kind words. We will write and share more of these.

Re: Popping the GPU Bubble

#14

Post by vkazanov »

nl wrote: post #3

> you find that the GPU often sits idle, not for lack of work, but because the CPU hasn't told it what to do next yet. This phenomenon is called a GPU bubble. This is true, but I've never heard anyone refer to this as a GPU bubble before. I think most people hear "GPU bubble" and think of a financial bubble of some kind.

I saw it in literature on cpu pipelines in quotes, never without.

Re: Popping the GPU Bubble

#15

Post by IshKebab »

vkazanov wrote: post #14
nl wrote: post #3

> you find that the GPU often sits idle, not for lack of work, but because the CPU hasn't told it what to do next yet. This phenomenon is called a GPU bubble. This is true, but I've never heard anyone refer to this as a GPU bubble before. I think most people hear "GPU bubble" and think of a financial bubble of some kind.

I saw it in literature on cpu pipelines in quotes, never without.

I've never seen it in quotes, but yeah it is a very common term in pipelined CPUs.

Re: Popping the GPU Bubble

#16

Post by _zoltan_ »

nl wrote: post #3

> you find that the GPU often sits idle, not for lack of work, but because the CPU hasn't told it what to do next yet. This phenomenon is called a GPU bubble. This is true, but I've never heard anyone refer to this as a GPU bubble before. I think most people hear "GPU bubble" and think of a financial bubble of some kind.

while the title is misreading, when reading GPU profiling data, we do call these bubbles - where the GPU _could_ do something, but it's idle.

any time your GPU is idle = you are losing $$$ = your TCO is going up. you don't want that.

Re: Popping the GPU Bubble

#17

Post by _zoltan_ »

barries11 wrote: post #7
rusk wrote: post #6

Earlier quoted context omitted.

The term I would use would be “underutilised”

"stall" is the best term I can think of as in "pipeline stall". Better term, anyone?

it's not stalled, as that would imply that it waits for something, which is not necessarily the case with bubbles. most often it shows lack of proper pipelining or wrong pipeline dependencies (pipeline A waits for pipeline B, pipeline C waits for pipeline B, while pipeline B waits for an event X, now you've just made all three pipelines stalled on event X - not good).

Re: Popping the GPU Bubble

#19

Post by rusk »

barries11 wrote: post #7
rusk wrote: post #6

Earlier quoted context omitted.

The term I would use would be “underutilised”

"stall" is the best term I can think of as in "pipeline stall". Better term, anyone?

When an engine stalls, the implication is that the chain reaction that drives it is failing - I don’t think that is the case with a GPU as it will quite happily sit there drawing watts til you give it things. In systems nomenclature the inverse term for bubble is utilisation. This or that link is or node is using x% of its capacity. Indeed, if you monitor your GPU with nvidia-smi you will see that very term in the instrumentation.

Re: Popping the GPU Bubble

#20

Post by spaqin »

nl wrote: post #3

> you find that the GPU often sits idle, not for lack of work, but because the CPU hasn't told it what to do next yet. This phenomenon is called a GPU bubble. This is true, but I've never heard anyone refer to this as a GPU bubble before. I think most people hear "GPU bubble" and think of a financial bubble of some kind.

Pretty sure that would be "[GPU performance] bottlenecked [by the CPU]" in most common terms.
Post reply on HN