Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD may get across the CUDA moat

hpcwire.com

Post reply on HN
Print view View original

281–290 of 312 posts

Re: AMD may get across the CUDA moat

#281

Post by LoganDark »

jacobgorm wrote: post #279
LoganDark wrote: post #264

Earlier quoted context omitted.

IIRC, ease of implementation (for the GPU kernels), and cross-compatibility (the same bytecode can be loaded by multiple models of GPU).

The ease of implementation using CUDA means that your code because effed for life, because it is no longer valid C/C++, unless you totally litter it with #ifdefs to special case for CUDA. In my own proprietary AI inference pipeline I've ended up code-generating to a bunch of different backends (OpenCL SpirV, Metal, CUDA, HLSL, CPU w. OpenMP), giving no special treatment to CUDA, and the resulting code is much cleaner…

> The ease of implementation using CUDA means that your code because effed for life

yes, yes it absolutely does. establishing market dominance as everyone wants to use CUDA but almost nobody wants to write their kernel twice.

Re: AMD may get across the CUDA moat

#282

Post by hot_gril »

anon291 wrote: post #271
hot_gril wrote: post #80

Earlier quoted context omitted.

I'm lazy, so it's 99% for me. I don't even mess with AMD CPUs; I know they're not exactly the same instruction set as Intel, and more importantly they work with a different (and less mainstream) set of mobos, so I don't want em. If AMD manages to pull more customers their way, that's great, it just means lower Intel premium for me.

I have no idea what you're talking about. Amd and Intel match on the isa in any case you'd see typically. Moreover, Intel is currently using AMDs instruction set. X86_64 was designed my amd and used to be called AMD64

It's never this simple. Their SIMD extensions differ for one, or at least did in the past.

Re: AMD may get across the CUDA moat

#283

Post by pezezin »

hot_gril wrote: post #282
anon291 wrote: post #271

Earlier quoted context omitted.

I have no idea what you're talking about. Amd and Intel match on the isa in any case you'd see typically. Moreover, Intel is currently using AMDs instruction set. X86_64 was designed my amd and used to be called AMD64

It's never this simple. Their SIMD extensions differ for one, or at least did in the past.

How long ago is that past?

If we are talking about 3D-Now, that is long dead and buried. If we are talking about the latest AVX-whatever, not even Intel is consistent, with different processor families supporting different subsets and applying different clock policies.

Re: AMD may get across the CUDA moat

#284

Post by mikepurvis »

iopq wrote: post #222
mgaunard wrote: post #193

Earlier quoted context omitted.

I don't know what world you live in, but this is a problem for any software development. You need to ensure that there is only one version of any library used globally throughout the code and that the set of versions is compatible with each other, and preferably you also want everything to be built against the same toolchain with the same flags. That usually means onboarding third-party libraries into your own build…

In NixOS, I can install multiple versions of libraries Or rather, I install no versions of libraries because NixOS will put them all in the store in different folders, and will compile the executable to use the correct path (or patch the elf when needed) it has an issue with pip because it's allergic to just randomly executing things as part of package management, but pip in general is wtf

Ironically I'm having this problem in a Nix build context because of the broken approach Nix takes to packaging bazel—which itself is largely a consequence of the larger issue I'm grouching about here: unbundling tensorflow's locked dependencies is very hard to do when the underlying source is written to assume it's only targeting the exact version specified in the build rules. You can't just switch it to target the gproto in nixpkgs because then you get compilation failures.

Re: AMD may get across the CUDA moat

#285

Post by incognition »

latchkey wrote: post #212
incognition wrote: post #200

Earlier quoted context omitted.

Fp64??

https://en.wikipedia.org/wiki/Double-precision_floating-poin... NVIDIA fp32 (H100) has 2x more TFLOPS than AMD's fp32 (MI250) and AI doesn't need fp64 precision.

Lol it was meant as I wouldn't be caught dead using fp64

Re: AMD may get across the CUDA moat

#286

Post by [deleted] »

cyrux004 wrote: post #184
ShamelessC wrote: post #161

Earlier quoted context omitted.

> The way he stole Fail0verflow's work with the PS3 security leak after failing to find a hypervisor exploit for months absolutely soured any respect I had for him at the time That sounds interesting. I tried googling about it but can't really find much other than that failoverflow found a key and didn't release it, and then geohot released his own subsequently. I'd love to hear more about how directly he "stole" the…

People here generally try to bash people who are much smarter than them, throwing shade at their background. They will say that he abandoned his first company, gave up on tiny grad but both of them are very much alive projects

[deleted]

Re: AMD may get across the CUDA moat

#287

Post by mschuetz »

weebull wrote: post #252
mschuetz wrote: post #224

Earlier quoted context omitted.

Not just feature parity, but proper UX. Things need to just work, without spending hours or days to make them work.

Blame Nvidia. They are the ones the got the industry hooked on a proprietry API.

Why would I blame NVIDIA? If it wasn't for them, we'd still only have needlessly cumbersome APIs and ecosystems. They did what Khronos always failed to do: They created something that is both easy, powerful and fast. Khronos always heavily neglects the easy part.

Re: AMD may get across the CUDA moat

#288

Post by Zetobal »

65a wrote: post #205
Havoc wrote: post #57

Earlier quoted context omitted.

>if this unicorn were to show up A unicorn like that showed up a couple hours ago. Someone posted a guide for getting llama to run on a 7900xtx https://old.reddit.com/r/LocalLLaMA/comments/170tghx/guide_i... It's still slow and janky but this really isn't that far away. I don't buy that AMD can't make this happen if they actually tried. Go on fiverr, get them to compile a list of top 100 people in the DIY LLM space,…

I was running llama on a w7900 a month ago, with 48gb of VRAM and excellent performance. ROCm support got a lot better really recently.

Inference ≠ Training

Re: AMD may get across the CUDA moat

#289

Post by codemk8 »

javchz wrote: post #26

CUDA is the only reason I have an Nvidia card, but if more projects start migrating to a more agnostic environment, I'll be really grateful. Running Nvidia in Linux isn't as much fun. Fedora and Debian can be incredibly reliable systems, but when you add an Nvidia card, I feel like I am back in Windows Vista with kernel crashes from time to time.

From my recent experience, ROCm and hipcc allow you to port cuda programs fairly easily to AMD arch. The compiling is so much faster than nvcc too.

Re: AMD may get across the CUDA moat

#290

Post by [deleted] »

IntelMiner wrote: post #27
mattnewton wrote: post #4

Earlier quoted context omitted.

Re: startups, Geohotz raised a few million for this already. https://tinygrad.org/

Didn't he do what he always does. Rake in a ton of money, fart around and then cash out exclaiming it's everyone else's fault? The way he stole Fail0verflow's work with the PS3 security leak after failing to find a hypervisor exploit for months absolutely soured any respect I had for him at the time

[deleted]
Post reply on HN