Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

blog.simbastack.com

Post reply on HN
Print view View original

51–60 of 151 posts

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#51

Post by asenna »

enos_feedler wrote: post #33
theodorewiles wrote: post #31

My take is that B2C AI applications are kind of structurally limited by how hard it is to build personalized context. The idea of capable local models could be a huge unlock here if they are able to do the bottom-up context collection research / tagging / etc. at scale.

Is it really local models that unlock this? Surely stateless model APIs would yield the same benefits? I get that local can be “cheaper” depending on usage, but we’ve been renting storage and compute from clouds at a premium for ages..

A huge thing here was the massive amount of data that was just processed - I went through about 1TB of files over 24 hours.

Using API to analyze even a subset of this would've been painful imo.

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#52

Post by jaggederest »

refulgentis wrote: post #43
jaggederest wrote: post #42

Earlier quoted context omitted.

I don't think "if you iterate on this, try using some tools, and ultimately demand that the output meet or exceed your demonstrated taste in other domains" is a hot take, honestly.

It's not a hot take, you're right, I gravely misunderstood the timing in your post, i.e. you were clearly framing it as after and being polite and encouraging. I'm more hot about it because it's frustrating having so many HN posts be a place for people to work out first drafts, especially when the first piece of feedback is "hey, uh, you clearly used AI and it's horrible to read as a result." So easy to avoid...good…

No worries, text is hard whether there's AI involved or not - I, in turn, mistook your clarification as a snarky "ah well of course if they try harder it'll be fine", my apologies for that. I share your frustration, but the best way I think is to educate not remonstrate unless they're someone who should clearly know better[1]

[1] https://news.ycombinator.com/item?id=48172536

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#53

Post by repparw »

asenna wrote: post #29
jaggederest wrote: post #28

Earlier quoted context omitted.

Hey friend, try something in this ballpark, your post has a bunch of painful AI tropes: https://github.com/blader/humanizer You get a pass here because you're doing really cool stuff but it's kinda tough to read past the AI nonsense, and it's relatively easy to screen out "it's not x it's y" kind of things and the bolded bullet points.

Thanks for this! This is exactly what I was looking for. Tbh, I have a lot of thoughts and ideas and things to share and I do spend time and effort trying to de-AI-ing it but this should help a lot. I'll try it out. In fact, I was expecting getting shit on by HN readers for this but was pleasantly surprised that readers moved past it.

if you care for some feedback about the writing, dropping the link and saying "PR's are open!" would land probably equal or better, and would reduce noise on the message. as sibling said, substance and noise

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#55

Post by 0xbadcafebee »

iMerNibor wrote: post #35
throwa356262 wrote: post #3

I ran Gemma on a 2015 thinkpad to do something similar. Fortunately, I could upgrade the memory otherwise it would have been a painful exercise. Not gonna lie, llama.cpp had the fans spinning at max speed. But it worked and I got the job done.

> the fans spinning at max speed This always confuses me - don't people want their computations to run as fast as possible and thus inevitably produce more heat that needs to be vented? I suppose sometimes it is just an analogy for "its utilizing 100% of my resources" (which I'm guessing it is here), but I've definitely had people say it as an actual complaint in different contexts

Fans shouldn't be running at max speed if the model fits in RAM with room to spare for context. Usually fans max out when the model doesn't fit and the CPU is chugging to make up the difference (or the user didn't tune LLM settings)

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#57

Post by enos_feedler »

asenna wrote: post #51
enos_feedler wrote: post #33

Earlier quoted context omitted.

Is it really local models that unlock this? Surely stateless model APIs would yield the same benefits? I get that local can be “cheaper” depending on usage, but we’ve been renting storage and compute from clouds at a premium for ages..

A huge thing here was the massive amount of data that was just processed - I went through about 1TB of files over 24 hours. Using API to analyze even a subset of this would've been painful imo.

I thought about that in this video case and it's true. I thought the parent comment was making a broader statement about local models in general. But even with video, if it was stored in private cloud storage near the LLM could this still have worked efficiently? What are the most painful elements of this whole setup / work environment if everything was cloud?

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#58

Post by michaelbuckbee »

theodorewiles wrote: post #31

My take is that B2C AI applications are kind of structurally limited by how hard it is to build personalized context. The idea of capable local models could be a huge unlock here if they are able to do the bottom-up context collection research / tagging / etc. at scale.

I made a B2C AI app that's fully local (and free) to do AI based contextual file renaming.

So if you give it a bunch of screenshots it will try and intelligently name them based upon what is in the screenshot. Same for videos, PDFs, etc.

But to your point I haven't even tried charging money as it feels like something Apple is just going to bake in as a feature.

https://finalfinalreallyfinaluntitleddocumentv3.com/

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#59

Post by ngai_aku »

I’d like to do something like this for the collection of home videos I have piling up, but I’m still on 16GB M1. Any hope of getting decent results with smaller models? If not, does anyone have tips on GPU rental?

I have a Claude max sub and plenty of OpenRouter credit, but I don’t feel good about uploading my family’s private videos

Re: Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

#60

Post by AlecSchueler »

refulgentis wrote: post #41
jaggederest wrote: post #39

Earlier quoted context omitted.

It didn't, it went from "this reeks from AI after edits, here's a tool that can help" to "people can read past it but there are better ways, you must demand quality". I don't think those two things are inconsistent.

Ah, I see, after he uses the tool it'll be great because he has taste.

I think you missed an important distinction being made:

> I also use a lot of AI but you really have to demand quality from it, whether it's writing, media, or code. It's clear you've got the taste from your media work, and we're all still learning as we go...

Their use of AI for "media work" has shown a taste but their writing usage still needs to equal that.

Post reply on HN