81–86 of 86 posts
Earlier quoted context omitted.
idk about “known” considering they basically created OpenGL
If OpenGL is your most up-to-date reference for Apple supporting general purpose computing then I think it absolutely emphasizes how little work they've put in.
Earlier quoted context omitted.
sounds (at least at a high level) similar to EXO[1] [1] https://github.com/exo-explore/exo
Here a video of testing Exo to run huge LLMs on a cluster of M4 Macs[1] more cheaply than with a cluster of NVDIA RTX 4090s. [1] https://www.youtube.com/watch?v=GBR6pHZ68Ho
However, this is interesting and I'm confused why aren't they showcasing the test-run of a larger model that actually necessitates distributing the workload across the cluster.
Earlier quoted context omitted.
> Metal is the answer. Everything else is just implementation detail as GP said. You can say this as long as you want, Nvidia makes money hand-over-fist supporting CUDA alongside OpenCL and DirectX. It's all just business to them - they don't have to play the same game as Apple because they're just not quite so petty with the ecosystem politics. Look at MacOS, for example. Plenty of legacy software never was supporte…
Apple are using their own servers for inference, that's the whole private cloud compute thing. Siri and other things use models and probably aren't running on it (though it's not announced), but those are older. > Apple's GPU design is stuck in iPhone mode and they refuse to change their approach with Apple Silicon desktop hardware. Looks competitive to me. https://venturebeat.com/ai/you-can-now-run-the-most-powerful…
Not for everything, though. Any ChatGPT/OpenAI-based inference request is being sent to Nvidia GPUs that run models too large for even the biggest Mac servers. You cannot refute this simply because Apple does not sell DGX-like server products. Even the rackmount Apple Silicon is still orders-of-magnitude off on the kind of performance you can get from a 1u GPU rack.
> Looks competitive to me.
When compared on equal grounds, Apple doesn't even have a GPU that beats Nvidia's 30XX series on power efficiency: https://browser.geekbench.com/opencl-benchmarks
If it "looks competitive" to you, then I invite you to look closer than just qualitative evidence. Apple's 3nm desktop designs are losing in straight-shot comparisons with Nvidia's 8nm products.
Earlier quoted context omitted.
If OpenGL is your most up-to-date reference for Apple supporting general purpose computing then I think it absolutely emphasizes how little work they've put in.
Metal is not particularly bad.
Accelerate framework might be what you're looking for here, but by most accounts that hasn't improved very much recently. And even still, Accelerate is no analog for OpenCL or CUDA.
Earlier quoted context omitted.
While understanding the GPU’s microarchitecture might be useful for exploits it’s definitely not worth “millions”.
You are right. The zero-day exploits might be worth roughly a million each, but not the family tree of native GPU's, ANE, CPU instruction sets and microarchitecture on which they would be based. My apology for writing unclearly, English is not my native language. I'm surprised it is yours. Saving on energy, programming effort and purchase cost of a supercomputer in case of M4 instruction sets and microarchitecture kn…
Earlier quoted context omitted.
Metal is not particularly bad.
Metal is neither a general-purpose compute API or a known example of Apple supporting it. If you consider Metal "general purpose" then so is DirectX or even Playstation and Nintendo shaders simply because you can sum matrices. Accelerate framework might be what you're looking for here, but by most accounts that hasn't improved very much recently. And even still, Accelerate is no analog for OpenCL or CUDA.