Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MacBook M5 Pro and Qwen3.5 = Local AI Security System

sharpai.org

Post reply on HN
Print view View original

41–50 of 162 posts

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#41

Post by jagged-chisel »

psyclobe wrote: post #28

I have always envisioned a ai server being part of a family's major purchases e.g. when they buy a house, appliance, etc. they also buy a 'ai system'. Machine hardware evolution is slowing down, pretty soon you can buy one big ass server that will last potentially decades as it would be purpose built for ai. Things like 'context based home security' yeah thats just, automatic, free, part of the ai system. Everyone wi…

And it's not going to happen any time soon because there's no recurring revenue to be gained from users/homeowners for such a thing.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#42

Post by hparadiz »

BoredPositron wrote: post #27
hparadiz wrote: post #6

Currently the barrier to entry for local models is about $2500. Funny thing is $2500 is about the amount my parents paid for a 166 MHZ machine in 1995.

The used model is 9B even with a big context you can easily run it on 16GB. You don't need a $2500 machine for it.

For coding and personal assistance the context window on 16GB is not good enough. Ideally I want a context window of 100k.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#43

Post by Octoth0rpe »

psyclobe wrote: post #28

I have always envisioned a ai server being part of a family's major purchases e.g. when they buy a house, appliance, etc. they also buy a 'ai system'. Machine hardware evolution is slowing down, pretty soon you can buy one big ass server that will last potentially decades as it would be purpose built for ai. Things like 'context based home security' yeah thats just, automatic, free, part of the ai system. Everyone wi…

> pretty soon you can buy one big ass server that will last potentially decades as it would be purpose built for ai.

This feels like a very, very weak prediction (though certainly possible).

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#44

Post by aegis_camera »

psyclobe wrote: post #28

I have always envisioned a ai server being part of a family's major purchases e.g. when they buy a house, appliance, etc. they also buy a 'ai system'. Machine hardware evolution is slowing down, pretty soon you can buy one big ass server that will last potentially decades as it would be purpose built for ai. Things like 'context based home security' yeah thats just, automatic, free, part of the ai system. Everyone wi…

Thanks for your insight, hardware of AI will be cheaper and memory of footage would be always saved locally.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#46

Post by jmalicki »

Octoth0rpe wrote: post #43
psyclobe wrote: post #28

I have always envisioned a ai server being part of a family's major purchases e.g. when they buy a house, appliance, etc. they also buy a 'ai system'. Machine hardware evolution is slowing down, pretty soon you can buy one big ass server that will last potentially decades as it would be purpose built for ai. Things like 'context based home security' yeah thats just, automatic, free, part of the ai system. Everyone wi…

> pretty soon you can buy one big ass server that will last potentially decades as it would be purpose built for ai. This feels like a very, very weak prediction (though certainly possible).

Perhaps if we truly run out of steam on the process node front?

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#47

Post by 0xbadcafebee »

hparadiz wrote: post #24
johndough wrote: post #18

Earlier quoted context omitted.

Perhaps OP was referring to a usable agentic system, for which $2500 sounds about right. I've got a 3060 myself, which is nice to play around with the smaller models for free (minus electricity) and with 100% uptime, but I was not able to program anything with them yet that I didn't want to rewrite completely. A heavily quantized Qwen3.5-27B model is getting close though. Maybe in a few months.

I was actually thinking of the AMD Ryzen AI Max+ 395 which compiles the linux kernel in 62 seconds and is the first usable integrated graphics solution I've seen. Benchmarks: https://old.reddit.com/r/LocalLLaMA/comments/1rpw17y/ryzen_a...

What does usable mean? There have been laptops and desktops with AI-capable iGPUs and 96-12GB RAM for 2 years.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#48

Post by anoopengineer »

jagged-chisel wrote: post #41
psyclobe wrote: post #28

I have always envisioned a ai server being part of a family's major purchases e.g. when they buy a house, appliance, etc. they also buy a 'ai system'. Machine hardware evolution is slowing down, pretty soon you can buy one big ass server that will last potentially decades as it would be purpose built for ai. Things like 'context based home security' yeah thats just, automatic, free, part of the ai system. Everyone wi…

And it's not going to happen any time soon because there's no recurring revenue to be gained from users/homeowners for such a thing.

With that logic, there wouldn't be anyone selling refrigerators or dishwashers.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#49

Post by bithive123 »

shmoogy wrote: post #37
infecto wrote: post #22

Can someone share how this stacks up to a Frigate? What I am struggling with this is how it sits in the security stack. Is it recording things of interest with motion or is it only a layer on top of the existing nvr

Buy a coral TPU for frigate - it can handle a ton of inference and is very cheap for what it offloads off the cpu

Before anyone buys a TPU for Frigate, try OpenVino on a cheap Intel N100 CPU. My mini PC frigate installation can handle 5 cameras easily.

Re: MacBook M5 Pro and Qwen3.5 = Local AI Security System

#50

Post by BoredPositron »

hparadiz wrote: post #42
BoredPositron wrote: post #27

Earlier quoted context omitted.

The used model is 9B even with a big context you can easily run it on 16GB. You don't need a $2500 machine for it.

For coding and personal assistance the context window on 16GB is not good enough. Ideally I want a context window of 100k.

In the other reply you said 50k. 16GB vram provides 40-70k on the 9b depending on the implementation and quant. Which is more than enough for the tool we are discussing in this thread but it looks like you are just changing your story instead of admitting that your initial comment was made in a hunch. Adding ever changing context in responses "to be right" is just bad manner.
Post reply on HN