Sick. Particularly stoked for the 10gb network card ($100 option) when using the Mac Mini as a server. Just wish the memory + NVMe prices could come back down to pre ai-goldrush prices. As $2999 for the M5 Pro with 64GB RAM feels painfully over-priced.
Apple introduces M6 and M5 Ultra
41–50 of 1001 posts
Re: Apple introduces M6 and M5 Ultra
#42I know they are a phone company, but I think they should focus on local models software, not only hardware.
Software wise there's plenty to choose from already. Ollama/llama.cpp, LM Studio, Lemonade, vllm etc. Anything Apple would bring to the table?
Maybe https://mlx-framework.org
FWIW, Ollama, LM Studio and Lemonade (and oMLX) also wrap Apple's MLX framework.
Re: Apple introduces M6 and M5 Ultra
#43> M5 Ultra features a massive amount of high-bandwidth unified memory, up to 512GB, and delivers a staggering 1.2TB/s of unified memory bandwidth that is 50 percent higher than M3 Ultra. Apple never needed to participate in the AI race to zero. Because they were already at the finish line years ago building their own chips that can run large >100B parameter AI models locally.
I was so blown away at all the discourse surrounding "Apple fumbling on models". They should never have been in the model game to begin with. Apple crushes hardware over the last decade and that's a huge advantage today. In the end, massive models have proven to be very strong, but small models have proven to be good enough (especially with the recent Qwen 2.8 27B drop) and that's where I imagine the future will lie…
I suspected Apple would let everyone else blow all their money then, when the dust settles, deliver a better experience to end users and clean up.
Re: Apple introduces M6 and M5 Ultra
#44~12k for 80 core gpu with 256gb, 14k in October for 512gb. Seems like that could make for a very descent on prem inference server.
It cant be 14k for 512GB because 96 -> 256 upgrade alone cost $4000
Re: Apple introduces M6 and M5 Ultra
#45The 512GB Ultra is amazing, sure. But who is it for exactly? VC funded big spender founders? In that case why would they need local AI? The ultra rich enthusiast? But there can't be too many of those. So who actually buys these?
For some , the idea that some proprietary information could potentially leak is enough to justify any price.
Re: Apple introduces M6 and M5 Ultra
#4696GB -> 256GB upgrade costs 4000 GBP in UK or $5460. $34 for GB. In US its $4000 upgade so $25 for 1GB. Also: > 512GB memory option for M5 Ultra coming late October
That US price is before sales tax no?
Re: Apple introduces M6 and M5 Ultra
#47The 512GB Ultra is amazing, sure. But who is it for exactly? VC funded big spender founders? In that case why would they need local AI? The ultra rich enthusiast? But there can't be too many of those. So who actually buys these?
A number of people in these comments, it would seem.
So, rich enthusiasts it is, then
Re: Apple introduces M6 and M5 Ultra
#48I know they are a phone company, but I think they should focus on local models software, not only hardware.
their strength has been hardware for over 30 years now
Re: Apple introduces M6 and M5 Ultra
#4996GB -> 256GB upgrade costs 4000 GBP in UK or $5460. $34 for GB. In US its $4000 upgade so $25 for 1GB. Also: > 512GB memory option for M5 Ultra coming late October
VAT?
I think the $34/GB figure might be inclusive of VAT and the $4560 not, which would be $28.5 otherwise. Not sure.