Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Popping the GPU Bubble

moondream.ai

Post reply on HN
Print view View original

31–40 of 53 posts

Re: Popping the GPU Bubble

#31

Post by alfiedotwtf »

blueblazin wrote: post #2

I really appreciate this type of articles. I feel like a lot of knowledge in LLM training and inference is locked inside the heads of practitioners. Similar to compiler engineers before. To work in LLM training/inference you’re expected to know this stuff but to know this stuff you need to be working in the space.

> Similar to compiler engineers before.

I guess the difference here being that we have ample compiler literature and practically know 99% of all there is to know about compilers that exist in the wild vs this new field.

Until we’ve gathered and agreed on a few “dragon books” for LLMs and have explored all there is to LLMs, you’re probably right - know-how will be with the practitioners and in source code until it’s distilled (pun intended).

Re: Popping the GPU Bubble

#32

Post by alfiedotwtf »

tux3 wrote: post #21
kibibu wrote: post #11

Earlier quoted context omitted.

"bubble" used to be used a lot more when talking about very deep pipelines, eg Pentium 4 depth.

Or in the case of my poor Verilog, even very short pipelines :(

And before that, graphics programmers called it vertical retrace :upsidedown:

Re: Popping the GPU Bubble

#33

Post by Melatonic »

alfiedotwtf wrote: post #31
blueblazin wrote: post #2

I really appreciate this type of articles. I feel like a lot of knowledge in LLM training and inference is locked inside the heads of practitioners. Similar to compiler engineers before. To work in LLM training/inference you’re expected to know this stuff but to know this stuff you need to be working in the space.

> Similar to compiler engineers before. I guess the difference here being that we have ample compiler literature and practically know 99% of all there is to know about compilers that exist in the wild vs this new field. Until we’ve gathered and agreed on a few “dragon books” for LLMs and have explored all there is to LLMs, you’re probably right - know-how will be with the practitioners and in source code until it’s d…

Better comparison would be low level code running on smaller chips. Intersection of hardware and software engineering

Re: Popping the GPU Bubble

#34

Post by fragmede »

radq wrote: post #26
fragmede wrote: post #22

That's a terrible name for that and I can't say that Hanlon's razor applies. Bubble that everyone's knowingly referring to is the stock market collapsing like in 2001. To choose a headline that can be mistaken for that just to get clicks is shit. You could've called it GPU-CPU pipeline stall, but no, you intentionally chose a name that would be confused for something else just to get clicks?

This is what people in the field call it. I'm sorry you're offended.

You. You are people in the field. You can choose to name it anything else in the article that you just wrote. "We call it the GPU-CPU pipeline stall, but others might call it the GPU bubble."

Re: Popping the GPU Bubble

#35

Post by dingdingdang »

cubefox wrote: post #30
fragmede wrote: post #22

That's a terrible name for that and I can't say that Hanlon's razor applies. Bubble that everyone's knowingly referring to is the stock market collapsing like in 2001. To choose a headline that can be mistaken for that just to get clicks is shit. You could've called it GPU-CPU pipeline stall, but no, you intentionally chose a name that would be confused for something else just to get clicks?

Yeah the title is obviously clickbait.

Yeah, it works though, and the content is genuine enough which I guess trumps the issue with the title for me ;)

Re: Popping the GPU Bubble

#37

Post by Eisenstein »

nl wrote: post #3

> you find that the GPU often sits idle, not for lack of work, but because the CPU hasn't told it what to do next yet. This phenomenon is called a GPU bubble. This is true, but I've never heard anyone refer to this as a GPU bubble before. I think most people hear "GPU bubble" and think of a financial bubble of some kind.

I thought it was normal for the AI field to confuse people by repurposing other terms of art? To: "transformer", "lora", "diffusion", "hallucination", etc, we can now add "bubble".

Re: Popping the GPU Bubble

#38

Post by aleph_minus_one »

amelius wrote: post #36

The real GPU bubble will be when AI companies figure out they can better make their own ASICs and ditch all their GPUs onto the market.

In data center operations, GPUs have some specific lifetime. Because datacenter GPUs are currently so expensive and hard to get, they don't get dropped on the market at some point (even if a better replacement has arrived), but used as long as possible.

Even if the AI companies decide to use their own ASICs, they will rather slowly, but continuously introduce them, while removing GPUs that have reached their end of life.

Re: Popping the GPU Bubble

#40

Post by amelius »

aleph_minus_one wrote: post #38
amelius wrote: post #36

The real GPU bubble will be when AI companies figure out they can better make their own ASICs and ditch all their GPUs onto the market.

In data center operations, GPUs have some specific lifetime. Because datacenter GPUs are currently so expensive and hard to get, they don't get dropped on the market at some point (even if a better replacement has arrived), but used as long as possible. Even if the AI companies decide to use their own ASICs, they will rather slowly, but continuously introduce them, while removing GPUs that have reached their end of l…

Yes, short term this is right. But at some point PyTorch will have a model.toVHDL() method, and we'll have a PCBWAY-style website for tapeout of the circuit. Nvidia's future looks less bright than they think and their GPU market will certainly pop.
Post reply on HN