Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple caught off guard by AI demand for Mac Mini and Mac Studio

macrumors.com

Post reply on HN
Print view View original

231–240 of 636 posts

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#231

Post by al_borland »

c0nsumer wrote: post #176
Kirby64 wrote: post #163

Earlier quoted context omitted.

> I have an older M2 Mac mini that does the OCR and visual description of all my screenshots. Screenshots are stored on my NAS. Doesn't Apple do this already within it's OS all locally? It certainly does it for OCR and categorization. EDIT: Also, no reason to use a generic LLM for this. This functionality exists in something like Immich (both OCR and 'context categorization'), and doesn't tie you into the Apple ecosy…

I personally use Apple Photos for this. It stores the original, plus makes them nicely searchable, so I have a Hazel action that takes screenshots from the desktop (and from my NAS where mobile devices back them up) and imports them. Works out really well.

The standard system search will also search for text in images now. I think at first it was Photos only, but now its system wide.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#232

Post by postalcoder »

alasdair_ wrote: post #172

There is a lot of "AI demand" that isn't just running inference on an LLM whose weights you downloaded. I'm training a model using reinforcement learning with self-play. I can and do use vast.ai when scaling but for experiments it's far faster, and cheaper, to run it locally until the bugs are all figured out. Just provisioning a new instance and copying the relevant checkpoints and things can take 25 minutes. It's z…

I’ve been happy training and running inference for small language models on my M4 Mac.

Inference with MLX is surprisingly zippy. I’m running a classification task on the entire HN comment dataset and it’s projected to take about two and a half days, which is not bad considering we’re talking about tens of millions of comments.

Yes, I could do it much more quickly by throwing Modal GPUs at it but this is low-priority work. I might as well throw my M4 a bone.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#233

Post by neverrroot »

AdmiralAsshat wrote: post #46

Mac Mini's were really nice HTPC candidates, too, before the AI boom. Like all things genuinely useful and affordable, they were snatched from the hands of normal consumers by a bunch of schmucks chasing the latest gold rush.

This is just so incredibly disrespectful to so many people.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#234

Post by jtap »

alasdair_ wrote: post #172

There is a lot of "AI demand" that isn't just running inference on an LLM whose weights you downloaded. I'm training a model using reinforcement learning with self-play. I can and do use vast.ai when scaling but for experiments it's far faster, and cheaper, to run it locally until the bugs are all figured out. Just provisioning a new instance and copying the relevant checkpoints and things can take 25 minutes. It's z…

Same, but with vision models. Unfortunately, I might be at my limit locally. I have three models that I'm using to find and identify objects in pictures. The largest dataset and model now takes about 8 hours per epoch on my Mac M4 with 16G memory.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#235

Post by okdood64 »

Whatarethese wrote: post #81

There are so many used M4 Mac Minis for sale on marketplace for $600+ now that the people I guess got bored of the local models and decided to see if they could make their money back as they probably bought them when they were $399 on sale earlier this year. I lowball them every time.

> I lowball them every time.

And?

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#236

Post by GeekyBear »

bel8 wrote: post #214
GeekyBear wrote: post #208

Earlier quoted context omitted.

> Classic monopoly move: Control the user base, then control hardware Classic monopoly move by who? Apple created MLX as an open source framework to allow users to run any open model locally.

just to clarify, models already ran locally without MLX years before it existed, on non-Apple environments. MLX was just Apple's bridge to what already ran in other hardware.

MLX is an open source framework that allows you to run open models in a manner optimized for Apple's hardware.

You can take advantage of larger amounts of memory, higher memory bandwidth, and clustering multiple systems.

How is using an open source framework to run open models a monopoly move?

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#237

Post by 0cf8612b2e1e »

jmalicki wrote: post #230
mv4 wrote: post #218

Earlier quoted context omitted.

It's quite shocking to me how many experienced, tech-savvy people, who used to care about cookies and ad tracking - are now willingly sending their business strategies, highly confidential contracts, and intimate personal issues to a cloud provider because "it is only $0.0x per million tokens!".

> It's quite shocking to me how many experienced, tech-savvy people, who used to care about cookies and ad tracking - are now willingly sending their business strategies, highly confidential contracts, and intimate personal issues to a cloud provider because "it is only $0.0x per million tokens!". Because there are more privacy guarantees there, depending on the provider. "But what if they violate their contract!" is…

  But what if they violate their contract!" is some pretty tin-foil hat stuff.
The foundation of these businesses is stealing IP in bulk.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#238

Post by jmalicki »

0cf8612b2e1e wrote: post #237
jmalicki wrote: post #230

Earlier quoted context omitted.

> It's quite shocking to me how many experienced, tech-savvy people, who used to care about cookies and ad tracking - are now willingly sending their business strategies, highly confidential contracts, and intimate personal issues to a cloud provider because "it is only $0.0x per million tokens!". Because there are more privacy guarantees there, depending on the provider. "But what if they violate their contract!" is…

But what if they violate their contract!" is some pretty tin-foil hat stuff. The foundation of these businesses is stealing IP in bulk.

The foundation of the businesses training AI models.

So don't use them for inference.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#239

Post by anslopic »

anslopic wrote: post #97
Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

[flagged]

[flagged]

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#240

Post by mixmastamyk »

Aurornis wrote: post #179
m4rtink wrote: post #173

Earlier quoted context omitted.

Do you believe Gemini will costs the same in 45 months or even exist, given Google track record ?

The options available across the board are getting cheaper and better all the time. There is no reason to believe that equivalent level model output will be more expensive in 12 months, let alone almost 4 years from now. Of all the good reasons to use local AI (privacy, etc), worrying about not having access to cheap models in 4 years is not one of them.

The big providers are losing on average tens of billions a year on these services, so yes prices must go up. Even Moore’s won’t help in the medium-term due to shortages and difficulty/reluctance to vastly increase capacity.
Post reply on HN