You have to pay separately for the charger now. £99, what a bargain.
[deleted]
MacBook Pro with M5 Pro and M5 Max
351–360 of 1001 posts
Re: MacBook Pro with M5 Pro and M5 Max
#352Earlier quoted context omitted.
The memory bandwith on M4 Max is 546 GB/s, M5 Max is 614GB/s, so not a huge jump. The new tensor cores, sorry, "Neural Accelerator" only really help with prompt preprocessing aka prefill, and not with token generation. Token generation is memory bound. Hopefully the Ultra version (if it exists) has a bigger jump in memory bandwidth and maximum RAM.
Do any frameworks manage to use the neural engine cores for that? Most stuff ends up running Metal -> GPU I thought
Re: MacBook Pro with M5 Pro and M5 Max
#353Earlier quoted context omitted.
The RTX 5090 only has 32gb of VRAM. So the tradeoff is NVIDIA is for blazing speed in a tiny memory pool, but Apple Silicon has a larger memory pool at moderate speed.
Or, there's the DGX Spark, which effectively neutralizes both of these trade-offs, and is the same price as the RTX 5090.
It's not 5090 performance though.
Re: MacBook Pro with M5 Pro and M5 Max
#354Earlier quoted context omitted.
Apple's AI strategy really kind of threads the needle cleverly. "AI" (LLMs) may or may not have a bubble-pop moment, but until it does Apple get to ride it on these press releases and claims. But if the big-pop occurs, then Apple winds up with really fantastic hardware that just happens to be good at AI workloads (as well as general computing). For example, image classification (e.g. face recognition/photo tagging),…
those things could likely just run fine on the gpu though
Not if GPU RAM is a limiter. Which it is for most models.
Unified memory is a serious architectural improvement.
How many GPUs does it take to match the RAM, and make up for the additional communication overhead, of a RAM-maxed Mac? Whatever the answer, it won’t fit in a MacBook Pro’s physical and energy envelopes. Or that of an all-in-one like the Studio.
Re: MacBook Pro with M5 Pro and M5 Max
#355Earlier quoted context omitted.
I've been so disappointed in Apple's lack of execution on this. There is so much potential for fantastic local models to run and intelligently connect to cloud models. I just don't get why they're dropping the ball so much on this.
Because it won’t sell enough hardware to matter to them. They aren’t dropping the ball, they are being smart and prudent.
Downvote all you want. Point blank, they are dropping the ball.
Re: MacBook Pro with M5 Pro and M5 Max
#356Earlier quoted context omitted.
A bit strange to use time to first token instead of throughput. Latency to the first token is not like a web page where first paint already has useful things to show. The first token is "The ", and you'll be very happy it's there in 50ms instead of 200ms... but then what you really want to know is how quickly you'll get the rest of the sentence (throughput)
No you don't. Not as a sticky mushy human with emotions watching tokens drip in. There's a lot of feeling and emotion not backed by hard facts and data going around, and most people would rather see something happening even if it takes longer overall. Hence spinner.gif, that doesn't actually remotely do a damned thing, but it gives users reassurance that they're waiting for something good. So human psychology makes t…
Some kinds of spinners serve as a coal-mine canary indicating if the app has gotten wedged. Not hugely useful, but also not entirely useless.
Re: MacBook Pro with M5 Pro and M5 Max
#357Earlier quoted context omitted.
100 tok/s sounds pretty good. What do you get with 70B? With 128GB, you need quantization to fit 70B model, right? Wondering if local LLM (for coding) is a realistic option, otherwise I wouldn't have to max out the RAM.
I run gpt-oss 120b model on ollama (the model is about 65 GB on disk) with 128k context size (the model is super optimized and only uses 4.8 GB of additional RAM for KV cache at this context size) on M4 Max 128 GB RAM Mac Studio and I get 65 tokens/s.
Have you tried the dense(27B,9B) Qwen3.5 models? Or any diffusion models (Flux Klein, Zimage)? I'm trying to gauge how much of a perf boost I'd get upgrading from an m3 pro.
For reference:
| model | size | params | backend | threads | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | --------------: | -------------------: |
| qwen35 ?B Q5_K - Medium | 6.12 GiB | 8.95 B | MTL,BLAS | 6 | pp512 | 288.90 ± 0.67 |
| qwen35 ?B Q5_K - Medium | 6.12 GiB | 8.95 B | MTL,BLAS | 6 | tg128 | 16.58 ± 0.05 |
| model | size | params | backend | threads | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | --------------: | -------------------: |
| gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | MTL,BLAS | 6 | pp512 | 615.94 ± 2.23 |
| gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | MTL,BLAS | 6 | tg128 | 42.85 ± 0.61 |
Klein 4B completes a 1024px generation in 72seconds.Re: MacBook Pro with M5 Pro and M5 Max
#358Earlier quoted context omitted.
> Look how many people bought Minis and ended up just paying for API calls anyway. (Saw it IRL 2x, see it on reddit openclaw daily) Aren't the OpenClaw enjoyers buying Mac Minis because it's the cheapest thing which runs macOS, the only platform which can programmatically interface with iMessage and other Apple ecosystem stuff? It has nothing to do with the hardware really. Still, buying a brand new Mac Mini for that…
Can't they simply run MacOS on a VM on existing Mac hardware?
Not if you want it to be able to use the hardware identifiers to register for use with iMessage.
Re: MacBook Pro with M5 Pro and M5 Max
#359Re: MacBook Pro with M5 Pro and M5 Max
#360Earlier quoted context omitted.
We had a workshop 6 months ago and while I've always been sceptical of OpenAI,etc's silly AGI/ASI claims, the investments have shown the way to a lot of new technology and has opened up a genie that won't be put back into the bottle. Now extrapolating in line with how Sun servers around year 2000 cost a fortune and can be emulated by a 5$ VPS today, Apple is seeing that they can maybe grab the local LLM workloads if…
I feel like the push by devs towards Metal compatibility has been 10x than AMD. I assume that's because the majority of us run MacBooks.
The only "push" towards Metal compatibility there's been has been complaints on github issues. Not only has none of the work been done, absolutely nobody in their right mind wants to work on Metal compatibility. Replacing proprietary with proprietary is absolutely nobody's weekend project. or paid project.