Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

blog.simbastack.com

Post reply on HN
Print view View original

21–30 of 151 posts

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#21

Post by asenna »

ahknight wrote: post #13
asenna wrote: post #5

Earlier quoted context omitted.

Thanks! I was honestly casually trying it out on the side with Claude's help. And I was actually pleasantly surprised to see how good the result was. Still blows my mind I can do all this from my 2021 MBP. I'll try to do a post once I have the next steps working (helping with planning and editing videos with Davinci Resolve).

I also have a 64GB M1 Max and am similarly impressed with what that workhorse can do. The M5 tempted me -- a lot -- but then I looked at what I was already getting done on that machine and just couldn't justify it ... yet. Someday, surely, but not yet. Gemma4 gave all my local projects new life, just like what you did here. Great job. Long live the M1 Max!

100%

Although knowing how good these local models are getting, I am now eyeing the upcoming M5 Ultra Mac Studio (256gigs perhaps). But knowing how crazy the market is, it might be a year before I get the chance to get my hands on it. If it even launches by WWDC.

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#22

Post by embedding-shape »

desro wrote: post #7

> The skill is open at ~/.claude/skills/video-index/. If you're working on something similar (indexing personal archives, getting a local model to do real archival work, building agents that drive editing tools), I'd be glad to compare notes. When your Claude wrote this post they might not have selected the right URL to share, unless your home folder is exposed. Care to share the skill files?

We just got a modern example of the classic message from a friend who just picked up programming, containing: "I just created my own web app, wanna check it out? It's here: http://localhost:8080"

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#23

Post by juancn »

busfahrer wrote: post #10
egorfine wrote: post #9

Thanks for the article! I have a beefy M5 Pro and I'm eagerly looking around for ways to use local models (specifically Gemma4 & Qwen3.6). This is an excellent thing to do. Especially that LLMs excel at batching thus you can index multiple photos and videos in parallel for no performance penalty.

I have been contemplating a M5 Pro MBP, but for the life for me I wasn't able to find benchmarks for real-world models, do you happen to know how many tokens per second roughly you get with MoE models like Qwen 3.6 35B/A3B or Gemma 4 26B?

I'm running unsloth/Qwen3.6-35B-A3B-UD-Q8_K_XL on an M3 Max, 64GB at ~57 t/s with llama-server

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#24

Post by embedding-shape »

busfahrer wrote: post #10
egorfine wrote: post #9

Thanks for the article! I have a beefy M5 Pro and I'm eagerly looking around for ways to use local models (specifically Gemma4 & Qwen3.6). This is an excellent thing to do. Especially that LLMs excel at batching thus you can index multiple photos and videos in parallel for no performance penalty.

I have been contemplating a M5 Pro MBP, but for the life for me I wasn't able to find benchmarks for real-world models, do you happen to know how many tokens per second roughly you get with MoE models like Qwen 3.6 35B/A3B or Gemma 4 26B?

You need to ask macOS people for their prefill speed as well, there are two numbers you care about here, and current MacBooks have generally terrible numbers when it comes to prefill performance. Surely it'll get better with time, but if you already have a desktop, I'd go the "beefy GPU" route first.

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#25

Post by mft_ »

satvikpendem wrote: post #12
egorfine wrote: post #9

Thanks for the article! I have a beefy M5 Pro and I'm eagerly looking around for ways to use local models (specifically Gemma4 & Qwen3.6). This is an excellent thing to do. Especially that LLMs excel at batching thus you can index multiple photos and videos in parallel for no performance penalty.

Unsloth Studio [0] is what I recommend these days, open source alternative to the more widely known LM Studio, and also built by the people who make good quantizations of released models. With MTP support not merged in you should get 2x token generation speed with no accuracy difference. They also have MLX quants if you scroll down a bit, which is a format specifically for macOS' Metal GPU acceleration but that's not…

I tried Unsloth Studio recently and was disappointed - in particular the downloading functionality is half-baked and didn’t cope with resuming downloads. As it seemed to just be a simple wrapper over llama.cpp, I found that huggingface hub, llama.cpp, and a couple of simple scripts actually offered better functionality once it was set up.

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#26

Post by brcmthrowaway »

juancn wrote: post #23
busfahrer wrote: post #10

Earlier quoted context omitted.

I have been contemplating a M5 Pro MBP, but for the life for me I wasn't able to find benchmarks for real-world models, do you happen to know how many tokens per second roughly you get with MoE models like Qwen 3.6 35B/A3B or Gemma 4 26B?

I'm running unsloth/Qwen3.6-35B-A3B-UD-Q8_K_XL on an M3 Max, 64GB at ~57 t/s with llama-server

Prefill speed and 27B number?

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#28

Post by jaggederest »

asenna wrote: post #14
asenna wrote: post #8

Earlier quoted context omitted.

Oops! My bad. Fixing it now. And yeah, I can share the Skill file. Give me 5 mins.

Ok I scrambled to finalize a name for it and create a new repo for it - https://github.com/Simbastack-hq/framedex PS - I just put this together in the last few mins, removed my personal files and references. So it's not tested properly, please let me know if any issues. It's still an early hack, but I have thousands of still images as well from my camera which I've not processed and I need to do the same analysis for…

Hey friend, try something in this ballpark, your post has a bunch of painful AI tropes:

https://github.com/blader/humanizer

You get a pass here because you're doing really cool stuff but it's kinda tough to read past the AI nonsense, and it's relatively easy to screen out "it's not x it's y" kind of things and the bolded bullet points.

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#29

Post by asenna »

jaggederest wrote: post #28
asenna wrote: post #14

Earlier quoted context omitted.

Ok I scrambled to finalize a name for it and create a new repo for it - https://github.com/Simbastack-hq/framedex PS - I just put this together in the last few mins, removed my personal files and references. So it's not tested properly, please let me know if any issues. It's still an early hack, but I have thousands of still images as well from my camera which I've not processed and I need to do the same analysis for…

Hey friend, try something in this ballpark, your post has a bunch of painful AI tropes: https://github.com/blader/humanizer You get a pass here because you're doing really cool stuff but it's kinda tough to read past the AI nonsense, and it's relatively easy to screen out "it's not x it's y" kind of things and the bolded bullet points.

Thanks for this! This is exactly what I was looking for.

Tbh, I have a lot of thoughts and ideas and things to share and I do spend time and effort trying to de-AI-ing it but this should help a lot.

I'll try it out.

In fact, I was expecting getting shit on by HN readers for this but was pleasantly surprised that readers moved past it.

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#30

Post by regexorcist »

egorfine wrote: post #18
satvikpendem wrote: post #12

Earlier quoted context omitted.

Unsloth Studio [0] is what I recommend these days, open source alternative to the more widely known LM Studio, and also built by the people who make good quantizations of released models. With MTP support not merged in you should get 2x token generation speed with no accuracy difference. They also have MLX quants if you scroll down a bit, which is a format specifically for macOS' Metal GPU acceleration but that's not…

I have researched for quite a bit and so far the fastest runtime is the oMLX one. But there's a caveat: ttft on MLX on M4 Pro is enormous. On M5 Pro it has been greatly sped up.

Curious if you tested llama.cpp and still found oMLX faster? I haven't tried the latter myself, might give it a go.
Post reply on HN