81–90 of 1001 posts
On M4 Max 128GB we're seeing ~100 tok/s generation on a 30B parameter model in our from scratch inference engine. Very curious what the "4x faster LLM prompt processing" translates to in practice. Smallish, local 30B-70B inference is genuinely usable territory for real dev workflows, not just demos. Will require staying plugged in though.
"Scaling up performance from M5 and offering the same breakthrough GPU architecture with a Neural Accelerator in each core, M5 Pro and M5 Max deliver up to 4x faster LLM prompt processing than M4 Pro and M4 Max, and up to 8x AI image generation than M1 Pro and M1 Max." Are they doubling down on local LLMs then? I still think Apple has a huge opportunity in privacy first LLMs but so far I'm not seeing much execution.…
I think its just marketing, and the marketing is working. Look how many people bought Minis and ended up just paying for API calls anyway. (Saw it IRL 2x, see it on reddit openclaw daily) I don't mind it, I open Apple stock. But I'm def not buying into their rebranding of integrated GPU under the guise of Unified Memory.
"Scaling up performance from M5 and offering the same breakthrough GPU architecture with a Neural Accelerator in each core, M5 Pro and M5 Max deliver up to 4x faster LLM prompt processing than M4 Pro and M4 Max, and up to 8x AI image generation than M1 Pro and M1 Max." Are they doubling down on local LLMs then? I still think Apple has a huge opportunity in privacy first LLMs but so far I'm not seeing much execution.…
So yes, the LLM should be inferencing on your prompt, but it should also be inferencing on 25,000 other things … in parallel.
Those are the compute needs.
We just need compute everywhere as fast as possible.
Why doesn't this excite me anymore?
Whoah, both the Pro and Max CPUs feature 18 cores. This hasn't happened since M1 Pro/Max. This is a surprise. Also, the mix of cores have changed drastically. - 6 "Super cores" - 12 "Performance cores" I'm guessing these are just renamed performance and efficiency cores from previous generations. This is a massive change from the M4 Max: - 12 performance cores - 4 efficiency cores This seems like a downgrade (in core…
So they renamed performance to mean efficiency and are now using super in place of performance?
Whoah, both the Pro and Max CPUs feature 18 cores. This hasn't happened since M1 Pro/Max. This is a surprise. Also, the mix of cores have changed drastically. - 6 "Super cores" - 12 "Performance cores" I'm guessing these are just renamed performance and efficiency cores from previous generations. This is a massive change from the M4 Max: - 12 performance cores - 4 efficiency cores This seems like a downgrade (in core…
The base M5 has super/efficiency cores.
The Pro and Max have super/performance cores.
Earlier quoted context omitted.
I've upgraded to Tahoe at 26.2, zero complaints from my side. Haven't had any runaway memory leaks or similar that were reported.
Closing Tabs in Safari till takes more than a second though. And if you hold Cmd-W to close all of them it just completely locks up and crashes. Still not fixed since the release of Safari 26. Literally unusable
"Scaling up performance from M5 and offering the same breakthrough GPU architecture with a Neural Accelerator in each core, M5 Pro and M5 Max deliver up to 4x faster LLM prompt processing than M4 Pro and M4 Max, and up to 8x AI image generation than M1 Pro and M1 Max." Are they doubling down on local LLMs then? I still think Apple has a huge opportunity in privacy first LLMs but so far I'm not seeing much execution.…
But memory bandwidth (bottleneck for LLM inference) is only marginally improved, 614 GB/s vs 546 GB/s for M4/M5 Max - where is this 4x improvement coming from? I think I'll pass on upgrading.
Why doesn't this excite me anymore?