Apple was also allegedly "caught off guard" by the Macbook Neo demand. I don't really see how they couldn't see the Local AI demand or demand for a cheaper Macbooks. This just reads like marketing imo.
the machines arent optimized for it is why. the main driving factor is large unified memory which makes large(r) models possible, but there isnt the gpu horsepower to back it up. essentially, it fits the corner of the market that wants large models and is ok with running them slowly which doesnt sound like it would be a large market.
Macs don’t have very good prefill, though. So it’s important to use a model serving stack that has excellent prompt caching and use a harness that won’t bust the cache.
I’m cross-shopping DGX Sparks and M5 Studios, and having a hard time deciding because they have exactly opposite characteristics for prefill and decode.