Popping the GPU Bubble • Hacker News Board

Re: Popping the GPU Bubble

#51

Post by knollimar » Wed, Jul 01, 2026, 10:38 AM UTC

amelius wrote: post #50

knollimar wrote: post #43

Earlier quoted context omitted.

Doesn't that assume that VHDL is trivial? I feel like there are tons of performance tradeoffs or hardware designers wouldn't have jobs

No it does not assume that. Some very smart people will write that model.toVHDL() function. And keep in mind that a DL model is only a very small subset of what you can use VHDL for, and most models will have a very similar implementation in hardware from a conceptual point of view. And don't take it too literally, VHDL could be replaced by other hardware design languages, maybe even at lower abstraction levels.

Not trying to take it literally, but aren't there costs vs performance tradeoffs? Like the py.toHDL would have like (maxSize,maxCost,minThroughput) as free and that would determine energy usage?

And a GPU is already pretty optimized for inference, no? Like isn't it a bunch of FP mults? I don't think HDLs do well with that, either.

Re: Popping the GPU Bubble

#52

Post by rjzzleep » Wed, Jul 01, 2026, 3:21 PM UTC

esperent wrote: post #29

rjzzleep wrote: post #10

Earlier quoted context omitted.

Gentle reminder that while most money is spent on LLM inference, the vast majority of useful AI use is in fact not LLMs. Also, more and more work is poured into making small models. One thing I like about the whole export controls saga is that people are finding creative ways to squeeze performance out of these devices as witnessed in this post. But, if you then look at solutions like vLLM, vLLM will just fill whatev…

> the vast majority of useful AI use is in fact not LLMs Can you explain what you mean here? Are you talking about small neural networks doing specific tasks?

All sorts of optimizations. Of course vision is huge. Lots of production use in all sorts of manufacturing. Lam research had a few talks a semiconductor manufacturing optimization. There is also CUDA assisted RAN.

Maybe AI is a bit of a misnomer, since everything ML at some point just started getting called AI.

Re: Popping the GPU Bubble

#53

Post by fragmede » Fri, Jul 03, 2026, 6:20 AM UTC

ksbd-pls-finish wrote: post #45

fragmede wrote: post #34

Earlier quoted context omitted.

You. You are people in the field. You can choose to name it anything else in the article that you just wrote. "We call it the GPU-CPU pipeline stall, but others might call it the GPU bubble."

The term is much older than the current GPU craze though. Ypu're trying to regulate how experts in a field communicate, which is... Weird.

I'm trying to effect cultural change. People who do that are gonna be considered "weird". That's what that word means.