Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Experimenting with Local LLMs on macOS

blog.6nok.org

Post reply on HN
Print view View original

271–276 of 276 posts

Re: Experimenting with Local LLMs on macOS

#271

Post by wpm »

alt227 wrote: post #239
aurareturn wrote: post #107

Earlier quoted context omitted.

I think if Cook had vision, he could have started something called Apple Enterprise and sold Apple Silicon as a server and made AI chips. I agree he’s too conservative and has no product vision. Great manager though.

Apple silicon does not compete well in multicore spaces. People seem to think that because it can run single core things really well on a laptop, it can do anything. Servers regularly have 100-200 cpu cores maxing out of rapid fire threads. This is not what Apple silicon excels at. On top of that, it only performs so well on consumer devices because they control the hardware and OS and can tune both together. Creatin…

Both Intel and AMD sell server CPUs with fewer than 100, hell, fewer than 32 cores.

There is of course a market for that. Not everyone needs a $4000 electric bill. Apple just can’t take the typical lions share of the profits in that market so they don’t bother.

Re: Experimenting with Local LLMs on macOS

#272

Post by segmondy »

oblio wrote: post #198
segmondy wrote: post #102

Earlier quoted context omitted.

I know this is false, DeepSeekv3.1, GLM4.5, KimiK2-0905, Qwen-235B are all solid open models. Last night, I vibed rough 1300 lines of C server code in about an hour. 0 compilation error, ran without errors and got the job done. I want to meet this experienced programmer that can knock out 1300 lines of C code in an hour.

Can you run any of those models without $20 000 worth of hardware that uses as much power and makes as much noise as a small factory?

I run them with under $3,000 hardware and inference is about 500-600watts with no noise.

Re: Experimenting with Local LLMs on macOS

#273

Post by rpdillon »

grim_io wrote: post #246
rpdillon wrote: post #100

Earlier quoted context omitted.

I mean, not really? Yeah, I pay to go to the movies and sit in a theater that they let me buy a ticket for, but that doesn't mean people that want to set up a nice home theater are ridiculous, they just care more about controlling and customizing their experience.

Some would argue that the home theater is a superior experience to a crowded, far away movie theater where the person's head in front of you takes up a quarter of the screen. The same can't be said for local inference. It is always interior in experience and quality. A reasonable home theater pays for itself over time if you watch a lot of movies. Plus you get to watch shows as well, which the limited theater program…

> The same can't be said for local inference. It is always interior in experience and quality.

Not really. I do it because it offers me more control. That's higher quality in my book.

Re: Experimenting with Local LLMs on macOS

#274

Post by jbs789 »

giancarlostoro wrote: post #233
jbs789 wrote: post #97

Earlier quoted context omitted.

Sounds like you’ve got a solid handle on things - go do it!

Give me a majority share in AAPL if that's what you want ;)

There are a things we can all do in our own lives, not necessarily running Apple. I for one am grateful not to be in the public spotlight running Apple! Everyone has opinions. It’s what you do with them that counts.

Re: Experimenting with Local LLMs on macOS

#275

Post by alt227 »

zargon wrote: post #260
alt227 wrote: post #240

Earlier quoted context omitted.

No offense to you personally, but I find it very funny when people hear marketing copy for a product and think it can do anything they said it can. Apple silicon is still just a single consumer grade chip. It might be able to run certain end user software well, but it cannot replace a server rack of GPUs.

I don’t think this is a fair take in this particular situation. My comment is in response to Simon Willison, who has a very popular blog in the LLM space. This isn’t company marketing copy; it’s trusted third parties spreading this misleading information.

Fair enough, apologies for assuming.

Re: Experimenting with Local LLMs on macOS

#276

Post by [deleted] »

otterley wrote: post #253
alt227 wrote: post #239

Earlier quoted context omitted.

Apple silicon does not compete well in multicore spaces. People seem to think that because it can run single core things really well on a laptop, it can do anything. Servers regularly have 100-200 cpu cores maxing out of rapid fire threads. This is not what Apple silicon excels at. On top of that, it only performs so well on consumer devices because they control the hardware and OS and can tune both together. Creatin…

> Apple silicon does not compete well in multicore spaces. Can you elaborate on this? Maybe with some useful metrics?

[deleted]
Post reply on HN