Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MacBook M5 Pro and Qwen3.5 = Local AI Security System

sharpai.org

Post reply on HN
Print view View original

91–100 of 162 posts

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#91

Post by psyclobe »

jagged-chisel wrote: post #41
psyclobe wrote: post #28

I have always envisioned a ai server being part of a family's major purchases e.g. when they buy a house, appliance, etc. they also buy a 'ai system'. Machine hardware evolution is slowing down, pretty soon you can buy one big ass server that will last potentially decades as it would be purpose built for ai. Things like 'context based home security' yeah thats just, automatic, free, part of the ai system. Everyone wi…

And it's not going to happen any time soon because there's no recurring revenue to be gained from users/homeowners for such a thing.

Well, custom/bespoke training for your families particular needs perhaps, performed once every 5 years.

I mean I envision analog/custom/bespoke ai hardware that is fundamentally 'good enough'. I mean as the market increases its need for these systems and as time progresses at some point it'll like warhammer 30k where these 'standard template constructs' are smart enough to basically teach you anything.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#92

Post by psyclobe »

beoberha wrote: post #59
psyclobe wrote: post #28

I have always envisioned a ai server being part of a family's major purchases e.g. when they buy a house, appliance, etc. they also buy a 'ai system'. Machine hardware evolution is slowing down, pretty soon you can buy one big ass server that will last potentially decades as it would be purpose built for ai. Things like 'context based home security' yeah thats just, automatic, free, part of the ai system. Everyone wi…

I don’t think there’s anything different between what you’re suggesting and a homelab. Most people do not have a homelab and are happy to offload services like photo storage or security to remote providers.

I'm thinking 'everyone needs an air conditioner', kinda need. Instead of 'some nerds run servers'. And this 'ac' is your 'ai'.

Maybe even subsidized by the government. This will be a fundamental need.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#93

Post by psyclobe »

zamadatix wrote: post #72
psyclobe wrote: post #28

I have always envisioned a ai server being part of a family's major purchases e.g. when they buy a house, appliance, etc. they also buy a 'ai system'. Machine hardware evolution is slowing down, pretty soon you can buy one big ass server that will last potentially decades as it would be purpose built for ai. Things like 'context based home security' yeah thats just, automatic, free, part of the ai system. Everyone wi…

If you bought a big ass server for your home 10 years ago it probably wouldn't have even have had a GPU/AI accelerator at all. If it did, it would have been something with wimpy compute and VRAM because you needed the video encoder/decoder for security cameras or the like. I'm not sure that really gives confidence hardware has really slowed down enough to invest in it for decades. Single core CPU performance has but…

Yeah but, how long do mainframes last? Think of the COBOL systems used in government. No reason to update them, they worked forever; their job is discrete and they performed it well enough where intense updating wasn't a requirement.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#94

Post by psyclobe »

icedchai wrote: post #90
psyclobe wrote: post #28

I have always envisioned a ai server being part of a family's major purchases e.g. when they buy a house, appliance, etc. they also buy a 'ai system'. Machine hardware evolution is slowing down, pretty soon you can buy one big ass server that will last potentially decades as it would be purpose built for ai. Things like 'context based home security' yeah thats just, automatic, free, part of the ai system. Everyone wi…

Based on our current trajectory, it seems more likely everyone will upload everything to the cloud and pay perpetual royalties to access their own data.

I really think this is a temporary scenario, there will be advancements in ai's building the next generation of ais, where the scale of the model continually shrinks and maybe there will be some break through that allows us to double the use of existing hardware/memory etc.

10 years ago I couldn't do alexa at my house, now I'm pretty close with a Qwen3:8b / Ollamma LLM (I mean I never really wanted alexa to do anything other then play music, automate stuff, etc. zero interest in it teaching me how to code).

I'm even thinking at some point we'll consider ai to be a fundamental human right to have access too as otherwise you are inherently in a disadvantaged position in terms of wealth prospects to those who do have access.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#95

Post by psyclobe »

nateb2022 wrote: post #55
psyclobe wrote: post #28

I have always envisioned a ai server being part of a family's major purchases e.g. when they buy a house, appliance, etc. they also buy a 'ai system'. Machine hardware evolution is slowing down, pretty soon you can buy one big ass server that will last potentially decades as it would be purpose built for ai. Things like 'context based home security' yeah thats just, automatic, free, part of the ai system. Everyone wi…

I disagree. Let's take the M1 vs the M5 ( https://www.macrumors.com/2025/11/10/apple-silicon-m1-to-m5-... ): - 6× faster CPU/GPU performance - 6× faster AI performance - 7.7× faster AI video processing - 6.8× faster 3D rendering - 2.6× faster gaming performance - 2.1× faster code compiling Over the span of 5 years. Plus, realistically what makes an "ai" server different from a computer? This "lineage info of the fami…

Today, not much differentiates them. But as time passes our only option will be to further specialize the hardware to get realistic gains; at some point perhaps a 'purpose built analog' computer kinda thing will get to the point where it is so useful, that it would be like the 'Standard Template Constructs' concept in Warhammer 30k. So what you can make a faster ai but, the current one can 'teach everyone, basically anything'.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#96

Post by aegis_camera »

tristor wrote: post #82

I'd like to recreate this benchmark using Qwopus on my M5 Max. I am curious if the theoretically improved reasoning capabilities from distillation improve its scoring. Adding this one to my to-do list for some point in the next few weeks.

M5 MAX should be very capable, you have a great brand new MBP.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#97

Post by aegis_camera »

antiterra wrote: post #87
bigyabai wrote: post #5

Earlier quoted context omitted.

Context is your limitation, on the M5. The larger your model is, the longer you'll be waiting on token prefill. TFTT with 0 tokens of context isn't a real-world benchmark. That's why most professional inference solutions reach for GPU-heavy hardware like the Jetson. Apple Silicon seems like a strange and overly expensive fit for this use cae.

I'm not a hardware expert here but this strikes me as inaccurate, though the actual performance can be scenario dependent. The Jetson hardware is targeted to low power robotics implementations. The Jetson Orin is currently marketed as prototyping platform, and I believe it does not generally challenge recent Apple Silicon for inference performance, even considering prefill. In the latest Blackwell based Jetson Thor,…

You do have a deep understanding of AI hardware landscape. Thanks for your analysis.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#98

Post by icedchai »

psyclobe wrote: post #93
zamadatix wrote: post #72

Earlier quoted context omitted.

If you bought a big ass server for your home 10 years ago it probably wouldn't have even have had a GPU/AI accelerator at all. If it did, it would have been something with wimpy compute and VRAM because you needed the video encoder/decoder for security cameras or the like. I'm not sure that really gives confidence hardware has really slowed down enough to invest in it for decades. Single core CPU performance has but…

Yeah but, how long do mainframes last? Think of the COBOL systems used in government. No reason to update them, they worked forever; their job is discrete and they performed it well enough where intense updating wasn't a requirement.

You also need to ask: How much do mainframes cost? They were engineered for backwards compatibility and reliability, with built in redundancy you don't find in consumer hardware.

AI models are changing every other day. I have to rebuild llama.cpp from source regularly. We are no where close to a personal "AI mainframe."

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#99

Post by tristor »

aegis_camera wrote: post #96
tristor wrote: post #82

I'd like to recreate this benchmark using Qwopus on my M5 Max. I am curious if the theoretically improved reasoning capabilities from distillation improve its scoring. Adding this one to my to-do list for some point in the next few weeks.

M5 MAX should be very capable, you have a great brand new MBP.

I've been doing a lot of experimentation with Qwen3.5 models locally, and I've found for other tasks that the Opus 4.6 distilled versions of the model ("Qwopus") tend to perform better for other tasks. But this is mostly based on the quality of output, not necessarily from a performance perspective. I'll report back once I get around to running the benchmark. I'm also interested in applying local AI tools onto my local security setup (built on UniFi).

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#100

Post by infecto »

shmoogy wrote: post #37
infecto wrote: post #22

Can someone share how this stacks up to a Frigate? What I am struggling with this is how it sits in the security stack. Is it recording things of interest with motion or is it only a layer on top of the existing nvr

Buy a coral TPU for frigate - it can handle a ton of inference and is very cheap for what it offloads off the cpu

I already run frigate. I am asking how this stacks up to it.
Post reply on HN