My M2 still does everything I need, but the M6 is seriously tempting.
Yeah. I keep looking at these updates but then my M2 feels blazing fast and I tell myself I'll wait for the next update.
Apple introduces M6 and M5 Ultra
821–830 of 1001 posts
Re: Apple introduces M6 and M5 Ultra
#822[dead]
Re: Apple introduces M6 and M5 Ultra
#823All I wanna know is how many tokens per second on 100B MoE??
Have we cracked 100 yet.
When we crack 100 tok/s at 2M context for <$10K I’ll buy one.
Re: Apple introduces M6 and M5 Ultra
#824A little expensive, but very powerful.
Re: Apple introduces M6 and M5 Ultra
#825I was blown away by M1 Pro, and used it for 4 years before it became a little sluggish, and replacing it with an Asus S16 running on Ryzen AI 9 HX 370 (an excellent laptop). One of the main reasons was missing my old Linux setup. I've briefly tested M5 Pro in an Apple store and was surprised by how quick it felt and did anything. A tangible and significant difference, and I really feel it would be good getting it, or…
> a little sluggish > surprised by how quick it felt Aren't these properties of the (G)UI rather than of the processor? I've been using systems 20 years ago that were super snappy. The same software would still be snappy today obviously. But the software has become bloated to the point that you need to run the latest generation of CPUs such that things are not sluggish. In principle you don't need 32-core GHz CPUs to…
> Aren't these properties of the (G)UI rather than of the processor?
They are a result of all of them.
Re: Apple introduces M6 and M5 Ultra
#826Earlier quoted context omitted.
Is there anything comparable that runs Linux, doesn't necessarily look as good, but is perhaps (a lot) cheaper/fixable? Or is this really pretty optimal? I mean this is not nvidia based right? It's all custom? So we can use it under Asahi perhaps? I want to get something for my company to run local models, wondering what would be a good option.
I guess, what I mean is: Why are these tiny aluminum boxes so optimal? I just want my butt ugly repairable beast machine to do the same trick. Why is my ram not unified? I have an iGPU in my server, but it can't access the 64 GB ram (I got last year for 150 euro) directly or something? It's on the CPU right? Why did only Apple go for this architecture? So many questions...
well the good news is you can indeed already just have unified cpu/gpu memory on linux with an igpu on good old replaceable ram. i've done it on 8th gen intel stuff i picked up dirt cheap used. for the most part, running on the gpu wasn't faster (nor appreciably slower) than the cpu for the things i was doing, just more power efficient. overall bandwidth is relatively limited regardless which would be the bigger difference comparing against the m chips. and of course, good luck if you're hoping stuff like opencl support hasn't long ago been ripped out of whatever software you might perfectly reasonably expect to run this way today
Re: Apple introduces M6 and M5 Ultra
#827Earlier quoted context omitted.
Do current models run on the NPU or GPU? Wondering if Apple will have something like a TPU.
Apple has a dedicated "neural engine" which is designed as an inference NPU. Where as Google's TPU has a dual focus, both inference and training, which is a more complex design.
That TPU training part I get, but from what I have seen the NPU is rarely used for inference by LLMs. They still use the GPU, no?
Re: Apple introduces M6 and M5 Ultra
#828...Additionally, M5 Ultra features a massive amount of high-bandwidth unified memory, up to 512GB, and delivers a staggering 1.2TB/s of unified memory bandwidth that is 50 percent higher than M3 Ultra.
That's faster than majority of GPUs available today: 5090 ~1.8TB 5080 1TB 5070 0.72TB My R9700 0.64TB :(
Re: Apple introduces M6 and M5 Ultra
#829Earlier quoted context omitted.
M1 Pro 2021 16GB RAM Feels as snappy and impressive as the first day I got it. I never need to close apps, or reboot. I record multitrack audio on it without any perceptible latency. I write code on it. It's the best computer I've ever owned by a significant margin.
MacBook is great hardware, but I really hate MacOS. It feels perpetually cellophane wrapped, and I don't like any of the window management. Finder is particularly awful too. You should be able to deeply tweak how window management works, but Mac won't let you do that. I'd run Linux on the hardware if it wasn't a hack.
I like Linux and used it as my daily driver for a long time but I find I’m addicted to macOS’ ctrl-scroll zoom feature as my eyesight deteriorates.
I’ve not found anything that works as smoothly for Gnome. So I keep going back to macOS with Rectangle to solve its UX issues.
Re: Apple introduces M6 and M5 Ultra
#830I was blown away by M1 Pro, and used it for 4 years before it became a little sluggish, and replacing it with an Asus S16 running on Ryzen AI 9 HX 370 (an excellent laptop). One of the main reasons was missing my old Linux setup. I've briefly tested M5 Pro in an Apple store and was surprised by how quick it felt and did anything. A tangible and significant difference, and I really feel it would be good getting it, or…
scouti