Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Resources for general purpose GPU development on Apple's M* chips?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 86 posts

Re: Ask HN: Resources for general purpose GPU development on Apple's M* chips?

#41

Post by amelius »

morphle wrote: post #30
twoodfin wrote: post #23

Earlier quoted context omitted.

Apple wants total freedom to rework lower levels of the stack down to the hardware, without worrying about application compatibility, hence their answer will continue to be Metal.

I agree that it allows Apple to redefine Apple Silicon instruction sets without having do explain it to 3rd party software developers, but it is certainly not the main reason they hide the technical documentation of the chips.

> but it is certainly not the main reason they hide the technical documentation of the chips

What is the main reason?

Re: Ask HN: Resources for general purpose GPU development on Apple's M* chips?

#42

Post by billti »

If you know CUDA, then I assume you know a bit already about GPUs and the major concepts. There’s just minor differences and different terminology for things like “warps” etc.

With that base, I’ve found their docs decent enough, especially coupled with the Metal Shader Language pdf they provide (https://developer.apple.com/metal/Metal-Shading-Language-Spe...), and quite a few code samples you can download from the docs site (e.g. https://developer.apple.com/documentation/metal/performing_c...).

I’d note a lot of their stuff was still written in Objective-C, which I’m not that familiar with. But most of that is boilerplate and the rest is largely C/C++ based (including the Metal shader language).

I just ported some CPU/SIMD number crunching (complex matrices) to Metal, and the speed up has been staggering. What used to take days now takes minutes. It is the hottest my M3 MacBook has ever been though! (See https://x.com/billticehurst/status/1871375773413876089 :-)

Re: Ask HN: Resources for general purpose GPU development on Apple's M* chips?

#43

Post by morphle »

amelius wrote: post #41
morphle wrote: post #30

Earlier quoted context omitted.

I agree that it allows Apple to redefine Apple Silicon instruction sets without having do explain it to 3rd party software developers, but it is certainly not the main reason they hide the technical documentation of the chips.

> but it is certainly not the main reason they hide the technical documentation of the chips What is the main reason?

>What is the main reason?

I can't guess what is the main reason. There might not even be a main reason, as many groups of people at Apple and its shareholders decided this over the years.

(Also see my speculations below in this thread).

So not in any order of importance to Apple:

1) Create the same moat as NVIDIA has with CUDA.

2) Ability to re-define the microcode instruction set of all the dozens of different Apple Silicon chips now and in the future without having to worry about backwards compatibility. Each Apple Silicon chip simply recompiles code at runtime (similar to my adaptive compiler).

3) Zero hardware documentation needed, much cheaper PR and faster time to market, also making it harder to reverse engineer or repair.

4) Security. Security by obscurity

5) Keeping the walled garden up longer.

6) Frustrating reverse engineering of Apple software. You must realize Apple competes with their own third party developers. Apple can optimize code on the GPU and ANE, third party developers can not and are forbidden too by Apple.

7) Frustrating reverse engineering of Apple hardware.

8) It won't make Apple more sales if 3rd party developers can write faster and more energy efficient GPU and NPU software.

9) Legal and patent infringements considerations

10) Future compiler improvements

11 ) Trade secrets

Re: Ask HN: Resources for general purpose GPU development on Apple's M* chips?

#44

Post by AnthonyMouse »

_zoltan_ wrote: post #34
morphle wrote: post #19

Earlier quoted context omitted.

There certainly is a reason and indeed you don't see it because Apple downplays these things in their PR. It might be the same reason that is behind NVDIA's CUDA moat. CUDA lock-in prevented competitors like AMD and Intel to convince programmers and their customers to switch away from CUDA. So there was no software ported to their competitive GPU's. So you get anti-trust lawsuits [1]. I think you should put yourself…

CUDA didn't prevent anything at least not in the way you believe. Intel and AMD had no competitive offer, period. They still don't. NVIDIA is simply offering an ecosystem that is battle tested and is ready out of the box. Look at the recent semianalysis test to see how not ready AMD is, who would be the only company to have a real shot at this. Their HW on paper is better or equal, yet their software ecosystem is now…

> Look at the recent semianalysis test to see how not ready AMD is, who would be the only company to have a real shot at this. Their HW on paper is better or equal, yet their software ecosystem is nowhere ready.

Reading that was kind of odd. It seems like their conclusion was that on paper AMD should be significantly less expensive and significantly faster, whereas in practice they're significantly less expensive and slightly slower because of unoptimized software, which actually seems like it'd still be a pretty good deal. Especially if the problem is the software, because then the hardware could get better with a software update after you buy it.

They also spend a lot of time complaining about how much trouble it is to install the experimental releases with some improvements that aren't in the stable branch yet, but then the performance difference was only big in a few cases and in general the experimental version was only a couple of percent faster, which either way should end up in the stable release in the near future.

And they do a lot of benchmarks on interconnect bandwidth which, fair enough, Nvidia currently has some hardware advantage. But that also mainly matters to the small handful of companies doing training for huge frontier models and not to the far larger number of people doing inference or training smaller models.

It feels like they were more frustrated because they were using the hardware as the problems were being solved rather than after, even though the software is making progress and many of the issues have already been resolved or are about to be.

Re: Ask HN: Resources for general purpose GPU development on Apple's M* chips?

#45

Post by MuffinFlavored »

morphle wrote: post #7

You can help with the reverse engineering of Apple Silicon done by a dozen people worldwide, that is how we find out the GPU and NPU instructions[1-4]. There is over 43 trillion float operations per second to unlock at 8 terabit per second 'unified' memory bandwidth and 270 gigabits per second networking (less on the smaller chips).... [1] https://github.com/AsahiLinux/gpu [2] https://github.com/dougallj/applegpu [3]…

This would get rid of needing Metal to be the blackbox and enable things like "nvptx CUDA" equivalent / https://libc.llvm.org/gpu/ right?

Very interesting. A steal for $22k but I guess very niche for now...

Re: Ask HN: Resources for general purpose GPU development on Apple's M* chips?

#46

Post by amelius »

morphle wrote: post #43
amelius wrote: post #41

Earlier quoted context omitted.

> but it is certainly not the main reason they hide the technical documentation of the chips What is the main reason?

>What is the main reason? I can't guess what is the main reason. There might not even be a main reason, as many groups of people at Apple and its shareholders decided this over the years. (Also see my speculations below in this thread). So not in any order of importance to Apple: 1) Create the same moat as NVIDIA has with CUDA. 2) Ability to re-define the microcode instruction set of all the dozens of different Apple…

9) hiding known and/or unknown patent infringements

Re: Ask HN: Resources for general purpose GPU development on Apple's M* chips?

#47

Post by morphle »

MuffinFlavored wrote: post #45
morphle wrote: post #7

You can help with the reverse engineering of Apple Silicon done by a dozen people worldwide, that is how we find out the GPU and NPU instructions[1-4]. There is over 43 trillion float operations per second to unlock at 8 terabit per second 'unified' memory bandwidth and 270 gigabits per second networking (less on the smaller chips).... [1] https://github.com/AsahiLinux/gpu [2] https://github.com/dougallj/applegpu [3]…

This would get rid of needing Metal to be the blackbox and enable things like "nvptx CUDA" equivalent / https://libc.llvm.org/gpu/ right? Very interesting. A steal for $22k but I guess very niche for now...

Yes, knowing the exact CPU and ANE assembly instructions (or the underlying microcode!!) allows for general purpose software to adaptively compile processes on all the core types, not just the CPU ones. Its won't always be faster, you get more cache misses (some cores don't have cache) and different DMA and thread scheduling, some registers can't fit the floats or large integers, etc etc.

But yes, it will be possible to use all 140 cores of the M2 Ultra or the 36 cores of the M4. There will be an M6 Extreme some day, maybe 500 cores?

Actually, the GPU and ANE cores themselves are built from teams of smaller cores, maybe a few dozens, hundreds or thousand in all, same as in most NVDIA chips.

>A steal for $22k but I guess very niche for now...

A single iPhone or Mac app (a game, an LLM, pattern recognition, security app, VPN, de/encryption, video en/dec coder) that can be sped up by 80%-200% can afford my faster assembly level API.

A whole series of hardware level zero-day exploits for iPhone and Mac would become possible, now that won't be very niche at all. It is worth millions to reverse Apple Silicon instruction sets.

Re: Ask HN: Resources for general purpose GPU development on Apple's M* chips?

#48

Post by likeabbas »

codr7 wrote: post #38
amelius wrote: post #15

Apple is known to actively discourage general purpose computing. Better try a different vendor.

Preferably one that sells computers, not fashion statements.

It's not a fashion statement, it's a fucking deathwish

Re: Ask HN: Resources for general purpose GPU development on Apple's M* chips?

#49

Post by JackYoustra »

morphle wrote: post #7

You can help with the reverse engineering of Apple Silicon done by a dozen people worldwide, that is how we find out the GPU and NPU instructions[1-4]. There is over 43 trillion float operations per second to unlock at 8 terabit per second 'unified' memory bandwidth and 270 gigabits per second networking (less on the smaller chips).... [1] https://github.com/AsahiLinux/gpu [2] https://github.com/dougallj/applegpu [3]…

any place you have your current progress written up on? Any methodology I could help contribute on? I've read each one of the four links you've given over the years and it seems vague with how far people have currently gotten and exact issues.

Re: Ask HN: Resources for general purpose GPU development on Apple's M* chips?

#50

Post by MuffinFlavored »

morphle wrote: post #47
MuffinFlavored wrote: post #45

Earlier quoted context omitted.

This would get rid of needing Metal to be the blackbox and enable things like "nvptx CUDA" equivalent / https://libc.llvm.org/gpu/ right? Very interesting. A steal for $22k but I guess very niche for now...

Yes, knowing the exact CPU and ANE assembly instructions (or the underlying microcode!!) allows for general purpose software to adaptively compile processes on all the core types, not just the CPU ones. Its won't always be faster, you get more cache misses (some cores don't have cache) and different DMA and thread scheduling, some registers can't fit the floats or large integers, etc etc. But yes, it will be possible…

What would a "llvm compilable" hello world look like that matches the libc GPU example for "AGX" (Apple Graphics)? It's not possible from MacOS, right? It'd have to be done from Linux?
Post reply on HN