Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple Silicon Exec Explains Mac Mini AI Demand and On-Device Future

macrumors.com

Post reply on HN
Print view View original

261–270 of 327 posts

Re: Apple Silicon Exec Explains Mac Mini AI Demand and On-Device Future

#261

Post by [deleted] »

bigyabai wrote: post #244
ingenieroariel wrote: post #233

Earlier quoted context omitted.

I wrote: "we should all be buying a fully loaded Mac Studio (128GB of ram, 20 CPU cores, a lot of GPU and Neural cores.)" April, 2023 We are both late and early. https://news.ycombinator.com/item?id=35527692

You should not buy a fully loaded Mac Studio for AI unless you absolutely NEED macOS. You will be wasting so much electricity idling on prefill while your GPU pulls 150-250w from the wall. Buy an Nvidia Spark, then whatever cheap Mac you want to use as a thin client. There's no reason to force Apple Silicon's round peg into a square hole like AI inference.

[deleted]

Re: Apple Silicon Exec Explains Mac Mini AI Demand and On-Device Future

#262

Post by throw1234567891 »

brandensilva wrote: post #203
throw1234567891 wrote: post #82

Earlier quoted context omitted.

512GB M3 Ultra is out of stock, not coming back, and there’s nothing like it on the consumer market. That’s the reason they go for so much.

But there will be and his point is they cut off supply to make room for the new M5 ultra which I hope has 768GB or more of memory.

Let’s see, I hope!

Re: Apple Silicon Exec Explains Mac Mini AI Demand and On-Device Future

#263

Post by awakeasleep »

HaloZero wrote: post #258
awakeasleep wrote: post #189

Earlier quoted context omitted.

One aspect you're missing is that people running a claw type agent thing need to run it on a Mac to automate software in the Apple ecosystem. Neo-Siri in iOS 27 removes the need for a lot of this, but before then, if you want to ask a robot about information that is stored in Apple notes, or to send an iMessage, a Mac mini is your only practical option.

What does the new Siri do that circumvents the need for a mac? To access Siri you still need one no?

the new Siri does the apple-ecosystem tasks you would have needed a claw type agent for natively

Re: Apple Silicon Exec Explains Mac Mini AI Demand and On-Device Future

#264

Post by nsagent »

_verandaguy wrote: post #250
operatingthetan wrote: post #245

Earlier quoted context omitted.

>I'm fatigued by it all at this point. This is the prevailing opinion of people even outside of tech.

I know public opinion polling supports that, but the parts of my social circle which are outside of tech seem to be, at worst, apathetic (and at best enthusiastic, though that's not a big fraction). That said, I think it's a good thing that this sentiment is coming to the forefront.

I was actually surprised to hear my brother-in-law deride LLMs as being useless for areas he has expertise in when I visited for the 4th of July.

He was complaining that he would ask how to perform a certain repair on a car, and the LLMs he tried (ChatGPT & Grok) would give him a long involved process and he'd ask why not do it this simpler way and it would say, oh you're right! He just found it gave bad advice and realized (rightly) that in areas he has less expertise in he has no way to judge how good the outputs are.

This is from a guy who loves tech, historically worshipped Elon, loves his Tesla, and (rightfully again) didn't buy into SpaceX because he thought it was overvalued.

In the past when I visited for holidays he was liable to have a positive outlook on LLMs and their utility. Seems telling that he's starting to see the cracks.

Re: Apple Silicon Exec Explains Mac Mini AI Demand and On-Device Future

#265

Post by no_no_no_yes »

onion2k wrote: post #7

Running models on-device on a Mac is immensely annoying though. Figuring out what will work out of BF16, FP8, BF16+FP8, NVFP4, INT8, GGUF ... the list goes on ... is 'non-obvious' at best. Apple do little to support with tooling. There's MLX, but unless you're happy to transform a model to that format yourself you'll be lagging a long way behind. Apps like LMStudio, Ollama, Draw Things, etc do a great job of simplify…

In general I just don't like not using a cross platform architecture of some kind.

MLX is fine, but the cross platform alternatives (ONNX) are terrible, GGUF often lacks and doesn't have the "easy convert" that some of the commenters below say MLX has.

Re: Apple Silicon Exec Explains Mac Mini AI Demand and On-Device Future

#266

Post by operatingthetan »

nsagent wrote: post #264
_verandaguy wrote: post #250

Earlier quoted context omitted.

I know public opinion polling supports that, but the parts of my social circle which are outside of tech seem to be, at worst, apathetic (and at best enthusiastic, though that's not a big fraction). That said, I think it's a good thing that this sentiment is coming to the forefront.

I was actually surprised to hear my brother-in-law deride LLMs as being useless for areas he has expertise in when I visited for the 4th of July. He was complaining that he would ask how to perform a certain repair on a car, and the LLMs he tried (ChatGPT & Grok) would give him a long involved process and he'd ask why not do it this simpler way and it would say, oh you're right! He just found it gave bad advice and r…

Yep. In my areas of expertise I can easily catch it coming up with wrong information, bad calculations etc. So laypeople are probably being led astray quite often.

Re: Apple Silicon Exec Explains Mac Mini AI Demand and On-Device Future

#267

Post by doikor »

nsagent wrote: post #264
_verandaguy wrote: post #250

Earlier quoted context omitted.

I know public opinion polling supports that, but the parts of my social circle which are outside of tech seem to be, at worst, apathetic (and at best enthusiastic, though that's not a big fraction). That said, I think it's a good thing that this sentiment is coming to the forefront.

I was actually surprised to hear my brother-in-law deride LLMs as being useless for areas he has expertise in when I visited for the 4th of July. He was complaining that he would ask how to perform a certain repair on a car, and the LLMs he tried (ChatGPT & Grok) would give him a long involved process and he'd ask why not do it this simpler way and it would say, oh you're right! He just found it gave bad advice and r…

> it would say, oh you're right!

This is easily the biggest problem with the current models. The models are just way too eager to please / say yes to the point that the models are happy to lie/make shit up if it means it can say yes.

Re: Apple Silicon Exec Explains Mac Mini AI Demand and On-Device Future

#268

Post by j45 »

habosa wrote: post #219

Apple has totally failed to deliver interesting AI experiences so far ... and I still think they're going to be the dominant provider of AI in 5 years. We're just one or two advances in chips / models / both away from being able to run very good local models for free on mid-tier Apple devices. The privacy, cost, and latency story there will be too much for OpenAI/Anthropic/Google to beat. Just writing this down so I…

I'm not sure that's what this article is about.

Apple is doing something very different. Their AI experience for end users definitely has been a little behind.

Apple Silicon, however, has been quite unique for the last 4-6 years and it's increasing overlap with LLMS.

The model/chip optimizations are definitely improvements, the thing that is really standing out the past 2 years is how much the open source model community has been making possible, especially when you know a group of use cases.

Re: Apple Silicon Exec Explains Mac Mini AI Demand and On-Device Future

#269

Post by ttul »

habosa wrote: post #219

Apple has totally failed to deliver interesting AI experiences so far ... and I still think they're going to be the dominant provider of AI in 5 years. We're just one or two advances in chips / models / both away from being able to run very good local models for free on mid-tier Apple devices. The privacy, cost, and latency story there will be too much for OpenAI/Anthropic/Google to beat. Just writing this down so I…

Here’s the two main reasons why local inference won’t compete any time soon with the cloud:

1. Most useful LLM work is done in parallel. A Mac Mini can run one LLM inference thread at a time. The cloud can spool up dozens and spread that inference across efficiently batched operations over a fleet of hardware.

2. Faster inference hardware such as the chips from Cerebras and Groq cannot be run locally. But the advantages of running >5x the token throughput per thread can’t be overstated. Add in the multi-threading advantage and it’s a knock-out punch for local LLMs.

Local inference has a role: if you’re working with extremely private matters or you want an uncapped model that will talk dirty or generate NSFW photos, local is the only option. I think Apple and others will continue to also run a lot of useful workloads locally such as text editing suggestions, speech to text, text to speech, and image manipulation. As local hardware improves, these capabilities will get better too.

But, for most LLM work, the cloud will continue to dominate for a long time to come, if not forever.

Re: Apple Silicon Exec Explains Mac Mini AI Demand and On-Device Future

#270

Post by tl »

Aurornis wrote: post #225
tl wrote: post #209

Earlier quoted context omitted.

I asked an Apple (via a sales rep who visited our company to showcase in internal iPad healthcare app) to please do this for iCloud when iCloud Drive was in-development. We would have easily paid $50,000 for a rack-able Mac Pro you could point "managed" devices at. Apple simply cannot comprehend the ask.

> Apple simply cannot comprehend the ask. Apple knows the market demand for this type of device. You may have paid $50,000 for it, but you’re only one customer. At Apple scale they need to focus their finite resources on the products that serve the largest market demand. $50,000 rack mount servers are not a large demand.

You'd be surprised. This was the same conversation where Numbers was pitched as an Excel replacement.
Post reply on HN