Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple caught off guard by AI demand for Mac Mini and Mac Studio

macrumors.com

Post reply on HN
Print view View original

211–220 of 635 posts

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#211

Post by ac29 »

jasode wrote: post #68
Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

>I realize I’m somewhat limited (16GB RTX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. I just ordered a new Mac Studio M5 Max 128GB $5899 ($6400 with tax) to be able to run the bigger "consumer size" models in the 70B parameter range (~96 GB). That said, I have no illusions that this expensive setup with a Qwen Flash coding LLM will be comparable t…

> AMD Strix Halo is about to release 192GB system which is a big upgrade to their current 128GB ai pc

Big upgrade to memory capacity but memory speed is only going up by a few percent, so its still going to be slow with more than a few B active params (I have one)

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#212

Post by veegee »

kridsdale1 wrote: post #197
alasdair_ wrote: post #172

There is a lot of "AI demand" that isn't just running inference on an LLM whose weights you downloaded. I'm training a model using reinforcement learning with self-play. I can and do use vast.ai when scaling but for experiments it's far faster, and cheaper, to run it locally until the bugs are all figured out. Just provisioning a new instance and copying the relevant checkpoints and things can take 25 minutes. It's z…

Likewise. I have a huge demand personally to run AI noise-filtering models on many TB per month of raw video files. It takes about 3 days per file. Apples ProRes codec is only licensed to run in high quality mode on a Mac, and so my Nvidia PC can’t do what I need. Thus, I own the beefiest Mac Studio you can currently buy. I would pay more for more TFlops. I have done local LLM on there but it wasn’t interesting. Far…

[dead]

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#213

Post by lumost »

Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

There are many enterprise environments where running modern models is... difficult. Rather than fighting security for months, a user could petition for a mac studio and have rough cost parity with a z.ai subscription.

if they are a heavy user, perhaps they string 4x together.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#214

Post by bel8 »

GeekyBear wrote: post #208
moezd wrote: post #202

Classic monopoly move: Control the user base, then control hardware. Any decent always-on local LLM setup with Apple devices will have to compete with these behemoths now. Great.

> Classic monopoly move: Control the user base, then control hardware Classic monopoly move by who? Apple created MLX as an open source framework to allow users to run any open model locally.

just to clarify, models already ran locally without MLX years before it existed, on non-Apple environments.

MLX was just Apple's bridge to what already ran in other hardware.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#216

Post by ashish01 »

alasdair_ wrote: post #172

There is a lot of "AI demand" that isn't just running inference on an LLM whose weights you downloaded. I'm training a model using reinforcement learning with self-play. I can and do use vast.ai when scaling but for experiments it's far faster, and cheaper, to run it locally until the bugs are all figured out. Just provisioning a new instance and copying the relevant checkpoints and things can take 25 minutes. It's z…

What are you training on using self play? Like alpha go? Curious what your setup is like .

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#217

Post by overfeed »

Aurornis wrote: post #179
m4rtink wrote: post #173

Earlier quoted context omitted.

Do you believe Gemini will costs the same in 45 months or even exist, given Google track record ?

The options available across the board are getting cheaper and better all the time. There is no reason to believe that equivalent level model output will be more expensive in 12 months, let alone almost 4 years from now. Of all the good reasons to use local AI (privacy, etc), worrying about not having access to cheap models in 4 years is not one of them.

> There is no reason to believe that equivalent level model output will be more expensive in 12 months

It's almost never a drop-in replacement, and having to check and adjust integrations and workflows with new models gets old fast. My task was perfectly solved by the old model, I don't need a newer, "better" one - especially at higher prices ("more cost-effective" my foot). Local models lets one choose a model and freeze the downstream integrations forever, without being forced on the 6/8-month upgrade treadmill by aggressively short, scarcity-driven hosted model-deprecation schedules.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#218

Post by mv4 »

wilkystyle wrote: post #171
tiahura wrote: post #144

Earlier quoted context omitted.

A new base model mac mini is $900. That is 45 month of Gemini. Gemini 4.7 Flash will give better OCR results that Qwen or GLM w/ 10GB.

This is such a tired argument and it seems to be parroted every single time someone talks about local models on hacker news. Yes, of course the most economical path is to hand over all your data and become fully dependent on a cloud provider who is already operating as scale, hoping that they won't change/remove models, hamstring capabilities, or raise prices. If this were a thread about hosting your own email or blo…

It's quite shocking to me how many experienced, tech-savvy people, who used to care about cookies and ad tracking - are now willingly sending their business strategies, highly confidential contracts, and intimate personal issues to a cloud provider because "it is only $0.0x per million tokens!".

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#219

Post by GeekyBear »

DannyBee wrote: post #27
setgree wrote: post #14

It's fun to see that even an extremely large company can find unexpected product market fit [0]. Per this article, "The company reportedly did not possess an engineering team dedicated to business customers or staff focused on developer relations, and lacked an enterprise AI strategy." That sounds insane in retrospect, but I think there's just inherent uncertainty in what people actually need and will use things for.…

It's also fun to see how many people here believed this was all some clear deliberate strategy in the first place rather than an accident.

They didn't "accidentally" add tensor units to the GPU cores in the M5 generation.

However, I don't think they expected the level of Enterprise interest they saw.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#220

Post by kridsdale1 »

adolph wrote: post #151
adastra22 wrote: post #131

Earlier quoted context omitted.

Yeah, The NVIDIA Way goes into a lot of detail on how and why the pivot from graphics to AI happened. This is a prime example of “you make your own luck.” Jensen engineered an organization that was primed to recognize and pounce on the next big thing, and it ended up being AI. But they saw it coming WAY in advance (like 2011/2012, not 2017) because they were explicitly on the lookout.

> an organization that was primed to recognize and pounce on the next big thing e.g.: previous crypto hype-cycle https://www.pcgamer.com/nvidia-cmp-graphics-card-availabilit...

Crypto was a stupid fad, but Nvidia certainly made a lot of money, so being a vendor to a fad is not stupid.
Post reply on HN