121–130 of 162 posts
Why is there so much interest in local AI systems, am I missing something? Cloud providers have scale and expertise that would allow for much bigger throughput at lower costs. The small latency gains will be nice, but ChatGPT and Claude already come through blazingly fast via their API.
Why is there so much interest in local AI systems, am I missing something? Cloud providers have scale and expertise that would allow for much bigger throughput at lower costs. The small latency gains will be nice, but ChatGPT and Claude already come through blazingly fast via their API.
I don't know if that's why other people are interested. I'm probably weird. But that's what drives my interest.
Earlier quoted context omitted.
My first 'real' machine was a Price Club (now Costco) 386sx for $3800 in late '89, which would be nearly $10k adjusted for inflation. 16 MHz, 1 MB RAM, 40 MB hard disk. That was bargain basement for that era. IBMs, Compaqs and the like were ~$5k similarly configured, and the first 486s were in the $7-9k area.
This picture of the Ryzen AI Max+ blew my mind. https://images.prismic.io/frameworkmarketplace/Z7aVJZ7c43Q3f... Look this isn't an ad. I've been building my own desktops since I was 14. It's always been a CPU and motherboard and memory separate type of deal but this thing has it all integrated. Look how small it is. I use Gentoo. I compile all the things. I know exactly how long it takes to compile gcc because I do i…
How is Qwen3.5 with 9B anywhere close to GPT-5.4 with xxxB?
It's a subset task. ..
It is still incredibly impressive of course! I just wish it was jailbroken
Earlier quoted context omitted.
M5 MAX should be very capable, you have a great brand new MBP.
I've been doing a lot of experimentation with Qwen3.5 models locally, and I've found for other tasks that the Opus 4.6 distilled versions of the model ("Qwopus") tend to perform better for other tasks. But this is mostly based on the quality of output, not necessarily from a performance perspective. I'll report back once I get around to running the benchmark. I'm also interested in applying local AI tools onto my loc…
Earlier quoted context omitted.
Yes you can get an RTSPS stream, but looks like Aegis is doing some validation that won't accept them. They look like - rtsps://192.168.1.1:7441/uOndh6hJd3Bti4kd?enableSrtp
Oh, sorry about that. I didn't test RTSPS stream, what model is it? I'll go by one and test. Before then, I'll check the flow to loosen the validation. Let's prepare a release for this ...
Currently the barrier to entry for local models is about $2500. Funny thing is $2500 is about the amount my parents paid for a 166 MHZ machine in 1995.
9B = 9 billion parameters. Q4_K_M is the quantization which will come in somewhere around 4.5 bits per weight.
It will run well on a $500 Mac Mini.
Earlier quoted context omitted.
Thanks for pointing out Unifi Protect, as long as the camera supports ONVIF(RTSP), then it could be connected, please let me know more, I'm not familiar with Unifi Protect, will do more research...
Yes you can get an RTSPS stream, but looks like Aegis is doing some validation that won't accept them. They look like - rtsps://192.168.1.1:7441/uOndh6hJd3Bti4kd?enableSrtp
Earlier quoted context omitted.
Oh, sorry about that. I didn't test RTSPS stream, what model is it? I'll go by one and test. Before then, I'll check the flow to loosen the validation. Let's prepare a release for this ...
There are many different models, but all should come up with similar RTSPS stream from Protect. Let me know when you cut a new release and i'll try it!
I have always envisioned a ai server being part of a family's major purchases e.g. when they buy a house, appliance, etc. they also buy a 'ai system'. Machine hardware evolution is slowing down, pretty soon you can buy one big ass server that will last potentially decades as it would be purpose built for ai. Things like 'context based home security' yeah thats just, automatic, free, part of the ai system. Everyone wi…
I disagree. Let's take the M1 vs the M5 ( https://www.macrumors.com/2025/11/10/apple-silicon-m1-to-m5-... ): - 6× faster CPU/GPU performance - 6× faster AI performance - 7.7× faster AI video processing - 6.8× faster 3D rendering - 2.6× faster gaming performance - 2.1× faster code compiling Over the span of 5 years. Plus, realistically what makes an "ai" server different from a computer? This "lineage info of the fami…
Overall system performance is better at about 2x improvement thanks to extra cores/other improvements/changes. I could see other more specialized benchmarks improving more thanks to different improvements/core/power/size improvements in other components (GPU/NPU/etc...).