Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CUDA Moat Still Alive

semianalysis.com

Post reply on HN
Print view View original

21–30 of 176 posts

Re: CUDA Moat Still Alive

#21

Post by the_king »

delusional wrote: post #11
the_king wrote: post #5

> It’s not just that it’s immature software, they need to change how they do development. I remember geohot saying something similar about a year ago

"The software needs to be better" is (and was) an easy call to make for anyone paying attention. The problem is that "AMD just needs to do better" is not and will never be an implementable strategy. Engineering isn't just about money. It's also about the process of exploring all the edge cases. "We recommend that AMD to fix their GEMM libraries’ heuristic model such that it picks the correct algorithm out of the box…

Sounds more like they were (and still are) being sloppy. “be better” is one thing. “runs without fatal crash” is what semi is talking about.

Re: CUDA Moat Still Alive

#22

Post by rikafurude21 »

dboreham wrote: post #8

Sounds like a buy signal for AMD. If you run the right branch and set the right env cars, the thing flies.

Would be a a buy signal if their actions (better drivers) show that they are seriously working on improving software. "This could be great _if_ you go through the trouble of doing it right!" is not persuasive, and any sane person would go with green if they know they have the choice between troubleshooting shitty software and things just working. Look at the george hotz archive youtube channel and watch the videos where he's debugging the amd drivers, it damn near ruins the man. And george is not the type to give up at the first roadblock, there are multiple 5-8 hour videos where he just tries to get the thing to work. The mad man ended up just writing his own driver lol.

Re: CUDA Moat Still Alive

#23

Post by SideQuark »

cyberax wrote: post #15

> CUDA Moat Still Alive Wrong conclusion. AMD is slower than NVidia, but not _that_ much slower. They are actually pretty cost-competitive. The just need to do some improvements, and they'll be a very viable competitor.

The amount of effort this team took, literally co-opting AMD engineers, and working for 5 months, to get closer but not yet usable, means they are not even close to usable. What team wanting to do ML training/inference can afford so much down time for zero benefit? How many except a few big ones can get AMD to devote so many resources simply for that team?

And, if you’re training a model costing you millions, the last thing you need is a buggy, untested stack, breaking training or perhaps worse giving you noise that makes your models perform worse or increases training time.

By the time AMD gets usable out of the box at this point, NVidia will have moved further ahead.

Re: CUDA Moat Still Alive

#25

Post by nrp »

dzdt wrote: post #24

[flagged]

In this case it’s because Dylan Patel of Semianalysis interviews Lisa Su regularly and presumably has a direct line to her, and because Lisa and the rest of AMD leadership are absolutely reading the article. It’s unclear if Pat would have (e.g. I don’t think Pat ever sat down for a chat with Semianalysis like Lisa has).

Re: CUDA Moat Still Alive

#26

Post by mountainriver »

nrp wrote: post #13

> Give AMD Engineers more compute and engineering resources to fix and improve the AMD ecosystem, they have very few internal gpu boxes relative to what Nvidia provides to their engineers. This is real. We’ve found ourselves having to give hardware to engineers at AMD because they’re unable to get allocation of it internally.

This seems so insane, is anyone actually doing the work to provide an alternative to CUDA? Maybe Google?

Re: CUDA Moat Still Alive

#27

Post by Archit3ch »

nrp wrote: post #13

> Give AMD Engineers more compute and engineering resources to fix and improve the AMD ecosystem, they have very few internal gpu boxes relative to what Nvidia provides to their engineers. This is real. We’ve found ourselves having to give hardware to engineers at AMD because they’re unable to get allocation of it internally.

Coming up next: "We bought AMD stock on the open market and used it to compensate AMD engineers".

Re: CUDA Moat Still Alive

#28

Post by incrudible »

dboreham wrote: post #8

Sounds like a buy signal for AMD. If you run the right branch and set the right env cars, the thing flies.

1. AMD always had a lot of hype already priced in, it is no different with AI.

2. AMD has always shipped a bad software stack, it is no different with AI.

Re: CUDA Moat Still Alive

#30

Post by latchkey »

dboreham wrote: post #8

Sounds like a buy signal for AMD. If you run the right branch and set the right env cars, the thing flies.

Anthony has been doing training on 4 of our MI300x systems and has been getting great results... but of course, he is a genius, and writing his own code...

https://x.com/HotAisle/status/1870984996171006035

https://x.com/zealandic1/status/1869857713280430349

https://x.com/zealandic1/status/1868810042168033623

Post reply on HN