HipKittens: Fast and furious AMD kernels • Hacker News Board

Re: HipKittens: Fast and furious AMD kernels

#41

Post by AMDAnon » Sat, Nov 15, 2025, 5:35 AM UTC

JonChesterfield wrote: post #37

AMDAnon wrote: post #14

Earlier quoted context omitted.

AMD pays the bare minimum in software to get a product out the door. The company does not even have working performance testing and regressions routinely get shipped to customers. Benchmarks the executives see are ad hoc and not meaningful. HipKittens is an improvement but AMD does not have the ability to understand or track kernel performance so it'll be ignored. This isn't fixable overnight. Company-wide DevOps and…

This doesn't sound right. I definitely got yelled at over trivial performance regressions which looked like noise so people were measuring performance. They've paid serious amounts in RSUs over the last six years. Not top of market by any stretch but firmly in the category of engineers don't care what the steak costs. Bonus might be team dependent, I remember being annoyed and nicely surprised by it in different year…

> I definitely got yelled at over trivial performance regressions which looked like noise so people were measuring performance.

It depends on team, we have some testing, and progress is being made. But it's not "working" or comprehensive as we get complaints from our big customers. We should be replicating their setup internally and not have them catch problems.

> Not top of market by any stretch but firmly in the category of engineers don't care what the steak costs.

We need to pay top of market to steal people from our competitors. We can't pay less than Nvidia and outcompete them. Paying less is a signal we're aiming for second and to copy the market leader.

Re: HipKittens: Fast and furious AMD kernels

#42

Post by bigyabai » Sat, Nov 15, 2025, 5:39 AM UTC

observationist wrote: post #15

LtdJorge wrote: post #7

Earlier quoted context omitted.

The vast amount of CUDA libraries for anything you can think of. I think there’s where they have the biggest leverage.

AI is going to be so ubiquitous, something principled and open is going to supersede cuda at some point, as HTML5 did for Flash. CUDA isn't like an x86 vs ARM situation where they can use hardware dominance for decades, it's a higher level language, and being compatible with a wide range of systems benefits NVIDIA and their competitors. They're riding out their relative superiority for now, but we're going to see a s…

> but we're going to see a standards and interoperability correction sometime soon, imo.

I thought this too, in 2015. OpenCL looked really promising, but Apple bailed and neither AMD nor Intel had the funding to keep up with Nvidia's research. It sorta floundered, even though Nvidia GPUs smugly ran OpenCL code with benchmark-leading performance.

Nvidia won the datacenter because of hardware. You could release a perfect CUDA-to-Vulkan translator tomorrow, and they still wouldn't be dethroned until better hardware replaced it. Intel is swirling the drain, Qualcomm is hedging their bets on mobile, AMD is (still) too underfunded - Apple is the only company with the design chops and TSMC inroads to be a serious threat, and they can't release a datacenter product to save their life. It's understandable why people think Nvidia is a monopoly, Team Green is pulling a full-on "Luigi wins by doing nothing" in 2025: https://knowyourmeme.com/memes/luigi-wins-by-doing-absolutel...

The market has almost no pressure to decouple from Nvidia - nobody else has mature solutions. It requires a preestablished player to make a similarly risky play, which might rule out everyone who's sitting at the table.

Re: HipKittens: Fast and furious AMD kernels

#43

Post by ekropotin » Sat, Nov 15, 2025, 5:48 AM UTC

bratao wrote: post #2

One thing I don't understand about Nvidia’s valuation is that right now a small number of algorithms have 'won,' such as Transformers. The data is very important. Compared to the past where customized code was much more common, such as modeling code and HPC, the ecosystem was very important and it was almost impossible to implement all CUDA and related code. Competitors now only need to optimize for a narrow set of a…

It’s all about deeply entrenched ecosystem NVIDIA had been building around CUDA for decades. It’d super hard to replicate this hardware-software platform.

Plus strategic partnerships with cloud providers.

And InfinityBand, yes

Re: HipKittens: Fast and furious AMD kernels

#44

Post by FuckButtons » Sat, Nov 15, 2025, 5:55 AM UTC

AMDAnon wrote: post #14

wewewedxfgdf wrote: post #4

You'd think AMD would swing in on something like this and fund it with the money needed to succeed. I have no knowledge of it but my guess is no, AMD never misses an opportunity to miss an opportunity - when it comes to GPUs and AI.

AMD pays the bare minimum in software to get a product out the door. The company does not even have working performance testing and regressions routinely get shipped to customers. Benchmarks the executives see are ad hoc and not meaningful. HipKittens is an improvement but AMD does not have the ability to understand or track kernel performance so it'll be ignored. This isn't fixable overnight. Company-wide DevOps and…

Madness. I see the accountants are in charge then.

Re: HipKittens: Fast and furious AMD kernels

#45

Post by ivape » Sat, Nov 15, 2025, 5:57 AM UTC

latchkey wrote: post #32

georgehotz wrote: post #25

Full disclosure, we have a contract with AMD to get Llama 405B training on MI350X on MLPerf. Things are turning around for AMD. If you have an AMD card, go to pytorch.org, click Linux+ROCm and install PyTorch. 3 years ago, this was hopeless. Today, most mainline things work. I ran nanochat on MI300X and it just worked. I think that's true about MI350X now too. The MI350X machine is stable. They are clearly behind NVI…

As CEO of an AMD NeoCloud for the past 2 years, it is so nice to hear all this and also see the turn around. It is what I bet my business on from the start and I can concur with what George is saying 100%. The out of box experience can be a bit rough around the edges on bleeding edge stuff, but it isn't anything near as bad as it used to be. For example, a month ago nanochat wasn't working well and now it is. The imp…

That’s interesting, I was specifically looking for AMD hardware being offered by neoclouds, they seem to be rare.

I like your bet though. The difference between NVDA and AMD has never really existed on a hardware level for decades. AMD has always been on par, and software is software, it will catch up.

AMD will be a stock many people will miss because the opportunity has presented itself at the height of AI bubble talk, and this will leave many in the dust. Doubling and tripling of their market cap is pretty much a forgone conclusion.

Re: HipKittens: Fast and furious AMD kernels

#46

Post by FuckButtons » Sat, Nov 15, 2025, 6:02 AM UTC

Aurornis wrote: post #29

AMDAnon wrote: post #17

Earlier quoted context omitted.

The MBAs have always been in charge to an extent. But the real issue is we don't want to invest in beating Nvidia on quality. Otherwise we wouldn't be doing stock buybacks and instead use the money on poaching engineers. The mindset is that we maintain a comfortable second place by creating a shittier but cheaper product. That is how AMD has operated since 1959 as a second source to Fairchild Semiconductor and Intel.…

> What they should do is bump TC by 60-70% and simultaneously lay off 50% of the engineers. I was part of company with a similar problem. If AMD’s situation is similar to what I dealt with, it’s more complicated. When you start doing deep cut layoffs at the IC level combined with expectations of big salary increases for those who remain, the office politics escalate to a level I didn’t know was possible. All of those…

"Maybe because they have so much free time to do office politics because they’re not doing much work.”

I mean, isn’t that always the way? Honestly, I feel like you could do a lot worse than just firing most of the people who demonstrate above average social skills. Sure, some would be fired unnecessarily, but I can’t think of any engineers that have seemed almost pathologically shy that also didn’t want to work hard.

Re: HipKittens: Fast and furious AMD kernels

#47

Post by ivape » Sat, Nov 15, 2025, 6:03 AM UTC

bratao wrote: post #2

One thing I don't understand about Nvidia’s valuation is that right now a small number of algorithms have 'won,' such as Transformers. The data is very important. Compared to the past where customized code was much more common, such as modeling code and HPC, the ecosystem was very important and it was almost impossible to implement all CUDA and related code. Competitors now only need to optimize for a narrow set of a…

I don’t think NVDA will have anything like a real moat, and more like whatever the difference was between iOS and Android. The gist of it is, the big bang of AI has happened and that universe is rapidly expanding, just like it once did for smart phones. There is the Apple of AI which is NVDA, and then there is Android (AMD). Moats are irrelevant here because the universe has just started rapidly expanding for them.

Apple didn’t really “win” out against Android, and it would be a very wrong way of measuring what actually happened. Yet, Apple could have been seen as more premium during various points of that timeline. The truth of the matter was, it was never a swimming race at any point in that smartphone timeline. It was simply a flood that you could convince yourself was an orderly race.

I believe the same is happening now, and it’s in Nvidias interest to maintain the narrative that there is a race and they are winning it. Believing something like this during the smartphone era would have been foolish.

Re: HipKittens: Fast and furious AMD kernels

#48

Post by sho » Sat, Nov 15, 2025, 6:17 AM UTC

AMDAnon wrote: post #17

observationist wrote: post #16

Earlier quoted context omitted.

The MBAs are in charge, and now AMD is the new Intel? It's not only not fixable overnight, but it's not fixable at all if the leadership thinks they can coast on simply being not as bad as Intel, and Intel has a helluva lot of inertia and ability to simply sell OEM units on autopilot. Sounds like the AMD board needs to get their heads out of their asses and shake up leadership.

The MBAs have always been in charge to an extent. But the real issue is we don't want to invest in beating Nvidia on quality. Otherwise we wouldn't be doing stock buybacks and instead use the money on poaching engineers. The mindset is that we maintain a comfortable second place by creating a shittier but cheaper product. That is how AMD has operated since 1959 as a second source to Fairchild Semiconductor and Intel.…

Came into this thread hoping for good news about GPUs and instead there's some surprisingly thoughtful management discussion!

> What they should do is bump TC by 60-70% and simultaneously lay off 50% of the engineers.

Tell me you're an engineer without telling me you're an engineer. The problem is they don't know which half and they can't know. It's an issue of legibility and transparency - put yourself into the shoes of the C-suite. You're staring down a complete black box of, what, 5,000 people. How can you possibly know who's good and who's not? Think of the information they have at hand - what the chain of command tells them. What if the chain of command itself is the problem? Think about how you yourself could protect a bad employee if you were a manager. You could! How can they possibly find the truth?

People rightly hate stack ranking, but you can see why ideas like that exist - attempts to come up with organizational pruning algorithms that are resistant to the managers themselves being the problem.

And this is also why CEOs incoming with a turnaround mission often do a clean sweep and stack the c-suite with all their friends. Not because they're giving jobs to their mates - although sure, that does happen - but because they're trying to establish at least a single layer of trust, which can then in time hopefully be extended downwards. But it all takes time, and for some organizations, they never do manage it. When unlimited orgs all compete for the same limited number of good managers - well, some of them are going to lose.

Ironically I'm bullish on AI being able to greatly help with all of this. Maybe running on AMD GPUs...

Re: HipKittens: Fast and furious AMD kernels

#49

Post by semessier » Sat, Nov 15, 2025, 6:19 AM UTC

without having implemented inference, just by looking at it from a math perspective this is base linear algebra/BLAS. I am very much wondering what a lean inference optimized API with covering 80% of all use cases across dtypes and sparsity would look like. Probably a far cry from what's in CUDA and probably all that's needed for practical inference.

Re: HipKittens: Fast and furious AMD kernels

#50

Post by slavik81 » Sat, Nov 15, 2025, 6:54 AM UTC

JonChesterfield wrote: post #37

AMDAnon wrote: post #14

Earlier quoted context omitted.

AMD pays the bare minimum in software to get a product out the door. The company does not even have working performance testing and regressions routinely get shipped to customers. Benchmarks the executives see are ad hoc and not meaningful. HipKittens is an improvement but AMD does not have the ability to understand or track kernel performance so it'll be ignored. This isn't fixable overnight. Company-wide DevOps and…

This doesn't sound right. I definitely got yelled at over trivial performance regressions which looked like noise so people were measuring performance. They've paid serious amounts in RSUs over the last six years. Not top of market by any stretch but firmly in the category of engineers don't care what the steak costs. Bonus might be team dependent, I remember being annoyed and nicely surprised by it in different year…

I don't think anon is correct, but I can understand how they'd come to their conclusions. I certainly didn't choose AMD to maximize my pay, though it's always been a comfortable salary.

With regards to performance, there are some things tracked carefully and other things that are not tracked at all. I suspect that is why some folks think we're really good at it and others think we're terrible. There's lots of room for improvement, though. Excitement over trivial performance regressions is more a sign of immaturity than of good tracking.