31–40 of 221 posts
Earlier quoted context omitted.
Is there no hope for AMD anymore? After George Hotz/Tinygrad gave up on AMD I feel there’s no realistic chance of using their chips to break the CUDA dominance.
https://x.com/dylan522p/status/1871287937268383867
I have come across quite few startups who are trying a similar idea: break the nvidia monopoly by utilizing AMD GPUs (for inference at least): Felafax, Lamini, tensorwave (partially), SlashML. Even saw optimistic claims like CUDA moat is only 18 months deep from some of them [1]. Let's see. [1] https://www.linkedin.com/feed/update/urn:li:activity:7275885...
Peculiar business model, at a glance. It seems like they're doing work that AMD ought to be doing, and is probably doing behind the scenes. Who is the customer for a third-party GPU driver shim?
Earlier quoted context omitted.
Not really. AMD is constitutionally incapable of shipping anything but mid range hardware that requires no innovation. The only reason why they are doing so well in CPUs right now is that Intel has basically destroyed itself without any outside help.
And I'm supposed to believe that HN is this amazing platform for technology and science discussions, totally unlike its peers...
I have come across quite few startups who are trying a similar idea: break the nvidia monopoly by utilizing AMD GPUs (for inference at least): Felafax, Lamini, tensorwave (partially), SlashML. Even saw optimistic claims like CUDA moat is only 18 months deep from some of them [1]. Let's see. [1] https://www.linkedin.com/feed/update/urn:li:activity:7275885...
Peculiar business model, at a glance. It seems like they're doing work that AMD ought to be doing, and is probably doing behind the scenes. Who is the customer for a third-party GPU driver shim?
Earlier quoted context omitted.
https://x.com/dylan522p/status/1871287937268383867
That's almost word for word what geohotz said last year?
I assume the part where she said there's "gaps in the software stack", because that's the only part that's attributed to her.
But I must be wrong because that hasn't been in dispute or in the news in a decade, it's not a geohot discovery from last year.
Hell I remember a subargument of a subargument re: this being an issue a decade ago in macOS dev (TL;Dr whether to invest in opencl)
Earlier quoted context omitted.
CPUs have pitiful RAM bandwidth compared to GPUs. The speeds aren't so different but GPU RAM busses are wiiiiiiiide.
Compute Express Link (CXL) should mostly solve limited RAM with CPU: 1) Compute Express Link (CXL): https://en.wikipedia.org/wiki/Compute_Express_Link PCIe vs. CXL for Memory and Storage: https://news.ycombinator.com/item?id=38125885
My years old pleb tier non-HBM GPU has more than 4 times the bandwidth you would get from a PCIe Gen 7 x16 link, which doesn't even officially exist yet.
Earlier quoted context omitted.
Isn't part of it because the first-party libraries like cuDNN are only available through CUDA? Nvidia has poured a ton of effort into tuning those libraries so it's hard to justify not using them.
Unlike training, ML inference is almost always bound by memory bandwidth as opposed to computations. For this reason, tensor cores, cuDNN, and other advanced shenanigans make very little sense for the use case. OTOH, general-purpose compute instead of fixed-function blocks used by cuDNN enables custom compression algorithms for these weights which does help, by saving memory bandwidth. For example, I did custom 5 bit…
Note: this is old work, and much of the team working on TVM, and MLC were from OctoAI and we have all recently joined NVIDIA.
Is there no hope for AMD anymore? After George Hotz/Tinygrad gave up on AMD I feel there’s no realistic chance of using their chips to break the CUDA dominance.
https://www.modular.com/blog/introducing-max-24-6-a-gpu-nati...