> Apple’s most powerful Mac raises the bar for local AI Wow, "Local AI" mentioned in the subheading above the fold - it's really awesome to see Apple leaning into this use case and I think it will definitely pay off for them going forward. Fingers crossed Apple is able to put some engineering effort towards shipping with one of the frontier open weight models included and optimized exactly for the machine.
New Mac Studio with M5 Max and M5 Ultra
361–370 of 571 posts
Re: New Mac Studio with M5 Max and M5 Ultra
#362Due to pricing insanity (not that Apple prices weren’t insane before the ram/ssd shortages) I’m not in the market but I do wonder if my next computer should be a Mac Studio instead of a MBP that lives its life docked. Might be better to just run a Studio and Neo for the very few times I actually need remote capabilities.
I've been thinking about this a fair bit recently. We make a lot of price/performance compromises for having an attached screen and keyboard on our computer. That was what got me started. Then I remembered the days of having to go to a special corner of the house to use a computer, vs now when I have a computer with me all the time. In my bag, on the sofa, on the train. Hell, I'm writing this on the work MBP while wa…
I can tailscale into my home network, but what next? VNC/TeamViewer/Remote Desktop setups are all mediocre imho.
Re: New Mac Studio with M5 Max and M5 Ultra
#363Earlier quoted context omitted.
> not that Apple prices weren’t insane before the ram/ssd shortages Funny that after the price started to increase last year (I think October/November), there was a window of a few months where Apple prices stayed the same as they were before, thus making apple prices actually good when compared to the rest of the market. It was a unique opportunity to have acquired a 512G M3 ultra for $10k.
not as crazy a config, but I got a 128GB M5max the day before the price increase for $2K less. still proud of the call to buy it.
Despite being outdated in terms of compute, it stills let me run very good recent models locally, with Deepseek V4 Flash 0731 being the greatest one right now, and hopefully Qwen 3.8 Flash will also fit well when it is released tomorrow!
The M1 ultra definitely leaves to be desired in terms of its token speeds, but I think 20 tps generation and ~200 tps prompt processing (which is what I get with DSv4 flash), is already enough to do a lot of serious work when you combine with the decent prompt caching provided by llama.cpp.
Re: New Mac Studio with M5 Max and M5 Ultra
#364Due to pricing insanity (not that Apple prices weren’t insane before the ram/ssd shortages) I’m not in the market but I do wonder if my next computer should be a Mac Studio instead of a MBP that lives its life docked. Might be better to just run a Studio and Neo for the very few times I actually need remote capabilities.
Would you get away with a Mini?
Re: New Mac Studio with M5 Max and M5 Ultra
#365Earlier quoted context omitted.
I've been thinking about this a fair bit recently. We make a lot of price/performance compromises for having an attached screen and keyboard on our computer. That was what got me started. Then I remembered the days of having to go to a special corner of the house to use a computer, vs now when I have a computer with me all the time. In my bag, on the sofa, on the train. Hell, I'm writing this on the work MBP while wa…
Is there an option to "use the remote computer just as it was local" even between two Macs? Sure, terminal/ssh is enough for most use-cases, but this is something I expected from Apple as builtin. I can tailscale into my home network, but what next? VNC/TeamViewer/Remote Desktop setups are all mediocre imho.
Re: New Mac Studio with M5 Max and M5 Ultra
#366Earlier quoted context omitted.
I've been thinking about this a fair bit recently. We make a lot of price/performance compromises for having an attached screen and keyboard on our computer. That was what got me started. Then I remembered the days of having to go to a special corner of the house to use a computer, vs now when I have a computer with me all the time. In my bag, on the sofa, on the train. Hell, I'm writing this on the work MBP while wa…
BRING BACK THE COMPUTER ROOM
But the couch is just so comfy.
Re: New Mac Studio with M5 Max and M5 Ultra
#367Earlier quoted context omitted.
Another option is a 'maxed out' mini e.g M5 Pro/64GB. Mini is super easy to travel with if you know there'll be a 'dock' at the far end e.g office home or whatever.
So I can somewhat speak to this, as I actually did this for a while c. 2021 and 2022. The issue (apart from my Mac mini still having the old, bigger form factor back then) is that this requires a full shutdown (obviously), and that is more friction than opening a closed laptop lid. It just takes a moment for every app and background service etc. to settle back in after, and depending on how your brain works, you migh…
The mini sips a truly tiny amount of power. I wonder what the smallest/lightest UPS could could bundle it with would be.
But at this point.. you’ve got a shit laptop.
Re: New Mac Studio with M5 Max and M5 Ultra
#368Earlier quoted context omitted.
There is no "future proof" for >1T param models, there is no present or future where you can run a model that size on consumer hardware.
6 Mac Studios for 100k USD. Consider it a rule of thumb now - 100k to run 1T params, scales linearly.
Re: New Mac Studio with M5 Max and M5 Ultra
#369Earlier quoted context omitted.
That's in the same ballpark as two 128GB AI machines like the Asus GX10 or DGX Spark or Strix Halo. And, it seems very likely to perform better than either of those for inference. And, 256GB brings some pretty good models into play. But, that doesn't make it a good deal. It just means the Apple tax doesn't apply when stacked up against AI machines and with memory prices being so out of whack. I'm still planning to wa…
Is there anything in the works or planned that suggests the RAMpocalypse will end anytime soon? e.g. new fabs being built, permitted, planned, etc...
Otherwise you'll have to wait to see if the AI circular financing club collapses- if you still have a job, there should be deals to be had...
Re: New Mac Studio with M5 Max and M5 Ultra
#370Earlier quoted context omitted.
I've been thinking about this a fair bit recently. We make a lot of price/performance compromises for having an attached screen and keyboard on our computer. That was what got me started. Then I remembered the days of having to go to a special corner of the house to use a computer, vs now when I have a computer with me all the time. In my bag, on the sofa, on the train. Hell, I'm writing this on the work MBP while wa…
Is there an option to "use the remote computer just as it was local" even between two Macs? Sure, terminal/ssh is enough for most use-cases, but this is something I expected from Apple as builtin. I can tailscale into my home network, but what next? VNC/TeamViewer/Remote Desktop setups are all mediocre imho.