Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MacBook M5 Pro and Qwen3.5 = Local AI Security System

sharpai.org

Post reply on HN
Print view View original

141–150 of 162 posts

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#141

Post by vercaemert »

Melatonic wrote: post #114
nateb2022 wrote: post #55

Earlier quoted context omitted.

I disagree. Let's take the M1 vs the M5 ( https://www.macrumors.com/2025/11/10/apple-silicon-m1-to-m5-... ): - 6× faster CPU/GPU performance - 6× faster AI performance - 7.7× faster AI video processing - 6.8× faster 3D rendering - 2.6× faster gaming performance - 2.1× faster code compiling Over the span of 5 years. Plus, realistically what makes an "ai" server different from a computer? This "lineage info of the fami…

It doesnt matter if computers keep getting faster - it just matters if eventually they get to the point where everything is good enough for good AI. That being said I feel like were gonna get to that point for most other stuff way sooner than AI (and already have for many pieces of software)

I love this conundrum.

I have a good analogy. 10 years ago, I was convinced that a 24-inch 1080p monitor at arm's length was perfection. There could never be any reason to improve over it. I could do everything I ever wanted to, to a standard I would never need to improve upon.

Yet here we are. The simplest and most obvious improvement is a 24" 4k monitor at 200% scaling. Basically, better in every way.

There's a discussion to be had about whether you need the better setup, which I think is your point, but there's no denying you'd want it (all other variables the same).

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#142

Post by algo_trader »

aegis_camera wrote: post #2

The M5 Pro just dropped, so here's a real AI workload instead of another Geekbench score. We run Qwen3.5 as the brain of a fully local home security system and benchmarked it against OpenAI cloud models on a custom 96-test suite. The Qwen3.5-9B scores 93.8% — within 4 points of GPT-5.4 — while running entirely on the M5 Pro at 25 tok/s, 765ms TTFT, using only 13.8 GB of unified memory. The 35B MoE variant hits 42 tok…

> fully local home security system

R u running the GPU at full throttle 24x7? Have you encounters silicon failures over time?

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#143

Post by jmalicki »

runako wrote: post #101
Octoth0rpe wrote: post #85

Earlier quoted context omitted.

Even if that happened tomorrow, I suspect we'd have _at least_ a decade of people tweaking/optimizing designs on the same node to squeeze meaningful performance upgrades out. Eg, coming up with hardware support for new int/float formats that make more sense for the models of 2029, running matrix operators on ram chips directly, etc.

I remember back in the early 2000s when people thought we were running out of steam on the advancements front. This was roughly around the time when CPU clocks stopped getting faster. Pentium hit 3GHz in 2005, Intel Core Ultra 5 performance cores are generally around this exact speed 20 years later. Since at least the 640kb quip, betting against progress or the appetite for progress has been a losing bet.

Honestly post 2005 things did slow down dramatically for typical single core workloads.

In the late 90s and early 2000s the mantra was "why waste time optimizing your software? By the time you're done the next gen of CPUs will have made up the difference."

Now the increase is more about moving to GPUs and power efficiency etc. We still have increases, but the rate of speedup has slowed down a lot.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#144

Post by tristor »

aegis_camera wrote: post #125
tristor wrote: post #99

Earlier quoted context omitted.

I've been doing a lot of experimentation with Qwen3.5 models locally, and I've found for other tasks that the Opus 4.6 distilled versions of the model ("Qwopus") tend to perform better for other tasks. But this is mostly based on the quality of output, not necessarily from a performance perspective. I'll report back once I get around to running the benchmark. I'm also interested in applying local AI tools onto my loc…

I just received one report that UniFi is using RTSPs, one fix is to loosen the RTSP string pattern, a release version is uploading ( 0.2.7 ). I'll find one UniFi camera to test secure RTSP streaming.

I tried to run the benchmark just now and ran into some issues. I have screenshots of the misbehavior. Do you have an email address I can reach out to?

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#145

Post by aegis_camera »

tristor wrote: post #144
aegis_camera wrote: post #125

Earlier quoted context omitted.

I just received one report that UniFi is using RTSPs, one fix is to loosen the RTSP string pattern, a release version is uploading ( 0.2.7 ). I'll find one UniFi camera to test secure RTSP streaming.

I tried to run the benchmark just now and ran into some issues. I have screenshots of the misbehavior. Do you have an email address I can reach out to?

I don't know if I can post the email here hopefully hn doesn't filter it out: service at sharpai.org

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#147

Post by tristor »

aegis_camera wrote: post #145
tristor wrote: post #144

Earlier quoted context omitted.

I tried to run the benchmark just now and ran into some issues. I have screenshots of the misbehavior. Do you have an email address I can reach out to?

I don't know if I can post the email here hopefully hn doesn't filter it out: service at sharpai.org

Thanks, I sent you an email with a couple of bugs I ran into and screenshots.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#148

Post by aegis_camera »

algo_trader wrote: post #142
aegis_camera wrote: post #2

The M5 Pro just dropped, so here's a real AI workload instead of another Geekbench score. We run Qwen3.5 as the brain of a fully local home security system and benchmarked it against OpenAI cloud models on a custom 96-test suite. The Qwen3.5-9B scores 93.8% — within 4 points of GPT-5.4 — while running entirely on the M5 Pro at 25 tok/s, 765ms TTFT, using only 13.8 GB of unified memory. The 35B MoE variant hits 42 tok…

> fully local home security system R u running the GPU at full throttle 24x7? Have you encounters silicon failures over time?

[flagged]

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#149

Post by spiderfarmer »

vercaemert wrote: post #141
Melatonic wrote: post #114

Earlier quoted context omitted.

It doesnt matter if computers keep getting faster - it just matters if eventually they get to the point where everything is good enough for good AI. That being said I feel like were gonna get to that point for most other stuff way sooner than AI (and already have for many pieces of software)

I love this conundrum. I have a good analogy. 10 years ago, I was convinced that a 24-inch 1080p monitor at arm's length was perfection. There could never be any reason to improve over it. I could do everything I ever wanted to, to a standard I would never need to improve upon. Yet here we are. The simplest and most obvious improvement is a 24" 4k monitor at 200% scaling. Basically, better in every way. There's a dis…

At some point specs don’t matter. I don’t wonder about the processor in my thermostat either. I don’t know how many horsepower my XC90 has. I don’t know the rated power of my chainsaw.

All I care about is: do they work, are they ‘safe’, are they comfortable, etc.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#150

Post by hparadiz »

Aurornis wrote: post #127
hparadiz wrote: post #6

Currently the barrier to entry for local models is about $2500. Funny thing is $2500 is about the amount my parents paid for a 166 MHZ machine in 1995.

The top local mode in this benchmark is Qwen3.5-9B (Q4_K_M), which is not a big model. 9B = 9 billion parameters. Q4_K_M is the quantization which will come in somewhere around 4.5 bits per weight. It will run well on a $500 Mac Mini.

I'm actually running it on my AMD 6900 XT right now with 16GB of RAM but looking at my options for upgrading my local model. Can't say I'm a fan of these entry level machines to be honest. I wanna be able to run it with 100k context.
Post reply on HN