Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CUDA Moat Still Alive

semianalysis.com

Post reply on HN
Print view View original

51–60 of 176 posts

Re: CUDA Moat Still Alive

#51

Post by schmidtleonard »

roenxi wrote: post #19
the_king wrote: post #5

> It’s not just that it’s immature software, they need to change how they do development. I remember geohot saying something similar about a year ago

I expect everyone has been saying it for a while, the calls are just getting more strident and public as it becomes clear that AMD's failures are strategic rather than tactical. And as people try to build business on their half-hearted attempts. I still think it is a mistake to say that CUDA is a moat. IMO the problem here is that AMD still doesn't seem to think that GPGPU compute is a thing. They don't seem to under…

Yeah, 80% margins on matrix multiplication should be a puddle not a moat but AMD is more scared of water than the witch that melts in Wizard of Oz so I guess the puddle is a moat after all.

Re: CUDA Moat Still Alive

#52

Post by shrubble »

Back in the late 1990s I met the ATI guys, and they were slipshod then as well. That the ATI legacy of special-casing things lives on is sadly, not too surprising for me.

Re: CUDA Moat Still Alive

#53

Post by redox99 »

dzdt wrote: post #24

[flagged]

People call out Gelsinger all the time.

And yes, she is definitely responsible for this. Probably more than Gelsinger.

At Intel it is not so obvious what they should have done to improve their fabs to better compete with TSMC, which is groundbreaking tech where you often have to make risky bets.

At AMD it was pretty obvious what had to be done to better compete in AI, and it was basic software engineering, not lithography wizardry. Totally achievable by just spending money, increasing head count, hiring top talent, and firing underperformers.

They have so much low hanging fruit that could have been solved just by hiring 5 or 10 software engineers.

Re: CUDA Moat Still Alive

#54

Post by ryao »

> AMD is attempting to vertically integrate next year with their upcoming Pollara 400G NIC, which supports Ultra Ethernet, hopefully making AMD competitive with Nvidia.

Infiniband is an industry standard. It is weird to see the industry invent yet another standard to do effectively the same thing just because Nvidia is using it. This “Nvidia does things this way so let’s do it differently” mentality is hurting AMD:

  * Nvidia has a unified architecture so let’s split ours into RDNA and CDNA.
  * Nvidia has a unified driver, so let’s make a different driver for every platform.
  * Nvidia made a virtual ISA (PTX) for backward compatibility. Let’s avoid that.
  * Nvidia is implementing tensor cores. Let’s avoid those on RDNA. Then implement them on CDNA and call them matrix cores.
  * Nvidia is using Infiniband like the rest of the HPC community. Let’s use Ethernet.
I am sure people can find more examples. Also, they seem to have realized their mistake in splitting their architecture into RDNA and CDNA, since they are introducing UDNA in the future to unify them like Nvidia does.

Re: CUDA Moat Still Alive

#56

Post by dralley »

ryao wrote: post #54

> AMD is attempting to vertically integrate next year with their upcoming Pollara 400G NIC, which supports Ultra Ethernet, hopefully making AMD competitive with Nvidia. Infiniband is an industry standard. It is weird to see the industry invent yet another standard to do effectively the same thing just because Nvidia is using it. This “Nvidia does things this way so let’s do it differently” mentality is hurting AMD: *…

You're painting this like AMD is off to play in their own sandbox when it's more like the entire industry is trying to develop an alternative to Infiniband.

Ultra Ethernet is a joint project between dozens of companies organized under the Linux Foundation.

https://www.phoronix.com/news/Ultra-Ethernet-Consortium

>> The Linux Foundation has established the Ultra Ethernet Consortium "UED" as an industry-wide effort founded by AMD, Arista, Broadcom, Cisco, Eviden, HPE, Intel, Meta, and Microsoft for designing a new Ethernet-based communication stack architecture for high performance networking.

You probably can't call it "industry standard" yet but the goal is obviously for it to become one.

Re: CUDA Moat Still Alive

#57

Post by ryao »

cyberax wrote: post #34
SideQuark wrote: post #23

Earlier quoted context omitted.

The amount of effort this team took, literally co-opting AMD engineers, and working for 5 months, to get closer but not yet usable, means they are not even close to usable. What team wanting to do ML training/inference can afford so much down time for zero benefit? How many except a few big ones can get AMD to devote so many resources simply for that team? And, if you’re training a model costing you millions, the las…

Sure. But this work is done, and can be reused by others. Meanwhile, Nvidia hardware is expensive and still is in short supply. AMD might look quite tempting.

There is a difference between doing something just for yourself and making it usable by others.

Re: CUDA Moat Still Alive

#58

Post by ryao »

dboreham wrote: post #8

Sounds like a buy signal for AMD. If you run the right branch and set the right env cars, the thing flies.

I expect Nvidia shares to increase tomorrow because of the article while AMD shares are not likely to do well. It is odd how we read the same thing and came to opposite conclusions.

Re: CUDA Moat Still Alive

#59

Post by schmidtleonard »

Lerc wrote: post #35
Archit3ch wrote: post #27

Earlier quoted context omitted.

Coming up next: "We bought AMD stock on the open market and used it to compensate AMD engineers".

You joke, but it is almost a genuine investment opportunity here for a large player. Spend a billion on AMD shares, Spend another Billion on a out-of-house software team to solve the software solution to more than double the share price. Taking into account that there are players that already own billions in AMD shares, they could probably do that as well. On the other hand perhaps it would be better for them, as maj…

I don't have the inside baseball but I have seen those weird as hell interviews with Lisa Su where she gets asked point blank about the software problems and instead of "working on it, stay tuned" -- an answer that costs nothing to give -- she deflects into "performance is what matters," which is the kind of denial that rhymes exactly with the problems they are having. No, the horsepower of your F1 racecar doesn't matter if the engine doesn't start and there's a wheel missing! You need to fix those problems before the horsepower can matter! Please tell me you are fixing the starter and the wheel!

Hopefully I am reading too much into this. Hopefully she doesn't have any weird hangups over investing in software and it all just takes time to Do It Right after GPGPU got starved in the AMD winter. But if it is a weird hangup then yeah, 100%, ownership needs to get management in line because whiffing a matmul benchmark years into a world where matmul is worth trillions just ain't it.

Re: CUDA Moat Still Alive

#60

Post by dogma1138 »

ryao wrote: post #47
dogma1138 wrote: post #20

Earlier quoted context omitted.

Anyone who looks at the mess that is ROCm and the design choices they made could easily see that. GPU support lagged behind for years, no support for APUs and no guaranteed forward compatibility were clear signs that as a whole they have no idea what they are doing when it comes to building and shipping a software ecosystem. To that you can add the long history of both AMD and ATI before they merged releasing dog shi…

> GPU support lagged behind for years, no support for APUs and no guaranteed forward compatibility were clear signs that as a whole they have no idea what they are doing when it comes to building and shipping a software ecosystem. This is likely self inflicted. They decided to make two different architectures. One is CDNA for HPC and the other is RDNA for graphics. They are reportedly going to rectify this with UDNA…

This was happening before CDNA was even a thing.

They didn’t release support even for all GPUs from the same generation and dropped support for GPUs sometime within 6 months of releasing a version that actually “worked”.

The entire core architecture behind ROCM is rotten.

P.S. NVIDIA usually has multiple CUDA feature levels even within a generation. The difference is that a) they always provide a fallback option, and usually this doesn’t require any manual intervention and b) is that as long as you define the minimum target framework when you build the binary you are guaranteed to run on all past hardware that is supported by the feature level you targeted and on all future hardware.

Post reply on HN