Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I indexed 669 GB of my GoPro videos using my M1 Max computer and local ML models

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 129 posts

Re: I indexed 669 GB of my GoPro videos using my M1 Max computer and local ML models

#51

Post by iliashad »

asenna wrote: post #28

Funny this is almost EXACTLY what I did a few days ago on the same machine using very similar techniques and was on the front-page of HN as well: https://news.ycombinator.com/item?id=48222733 https://blog.simbastack.com/indexed-a-year-of-video-locally/ I wasn't familiar with your project though, interesting stuff. I'm trying to add more photography related features to Framedex but yeah there's so much we can do local…

That's great, I checked your article when it was in front page because someone mentioned my project in the comments.

Good job for the article and the project. That's great, yes local models are getting better and better

Re: I indexed 669 GB of my GoPro videos using my M1 Max computer and local ML models

#52

Post by iliashad »

esjeon wrote: post #48

> Then, run the frame analysis pipeline, which will divide the video into separate video scenes (1s each, or 1fps) > (…) > Frames analyzed 57,537 Aha, it makes total sense. This number sounds much more reasonable than “669 GB”, since the actual total size of processed frames would be like 10-30 GB. (Not downplaying anything. Doing-at-home always requires some math on practicality) > Total compute time 67h 40m 42s I’m…

> Aha, it makes total sense. This number sounds much more reasonable than “669 GB”, since the actual total size of processed frames would be like 10-30 GB.

The reason why is “669 GB” is the total raw footage size when I'm doing the video processing, I downscaled each frame to 720p to make the video processing much faster and I don't need full original quality in order to get accurate results (as far as I know and experiment with).

> I’m just curious tho — is there any paying options that can accelerate this kind of process? Just spin up GPU instances?

For now, I found that NVIDIA GPU for example RTX 3060 with 12GB Vram was much faster than my M1 Max. (still working on optimizing for speed and accuracy).

Re: I indexed 669 GB of my GoPro videos using my M1 Max computer and local ML models

#53

Post by iliashad »

WhitneyLand wrote: post #34

I’d like to see embedding of actual video clips become practical in this type of workflow. Frame level embedding it covering a lot, but can miss out on a lot of action related searches.

Sure, I'm using (https://huggingface.co/collections/Qwen/qwen25-vl) which can help me understand action like falling down, because I can provide for example 5 frames (down scaled to 720p) to understand what is happening in this part of the video

Re: I indexed 669 GB of my GoPro videos using my M1 Max computer and local ML models

#55

Post by iliashad »

ingvay7 wrote: post #40
iliashad wrote: post #30

Earlier quoted context omitted.

Not really. Grab frames, lower res, classify, combine metadata, transcribe the audio, convert those data (text, visual and audio) to embedding, save them over a vector DB and SQL DB. Which helped me to do semantic search, RAG, search using a screenshot of the video to find the exact the moment in the video plus search using an audio file as well. And other features unlocked with vector DB

Really cool work and workflow. strongly prefer this kind of local, open pipeline that i control over a dependency on Adobe tools and lock ins.

I agree with that, thank you for your feedback. Also, maybe you're not a video editor and you just wanna search your videos. The video editing integrations are optional and you have full control. You can switch between Adobe Premiere Pro, Final cut Pro or Davinci Resolve

Re: I indexed 669 GB of my GoPro videos using my M1 Max computer and local ML models

#56

Post by JMiao »

justinram11 wrote: post #47

Something I've enjoyed more than I expected is Google and Apple photos sending me photo memories and compilations of various things in my life and my kids lives over the last decade. I'm really bullish on taking more video of my kids, with the thought that it will become easier and easier for AI to put them into little compilations I can enjoy later.

do you use android and ios, or is there another benefit to having personal media with both?

Re: I indexed 669 GB of my GoPro videos using my M1 Max computer and local ML models

#57

Post by iliashad »

JMiao wrote: post #56
justinram11 wrote: post #47

Something I've enjoyed more than I expected is Google and Apple photos sending me photo memories and compilations of various things in my life and my kids lives over the last decade. I'm really bullish on taking more video of my kids, with the thought that it will become easier and easier for AI to put them into little compilations I can enjoy later.

do you use android and ios, or is there another benefit to having personal media with both?

Can you please elaborate more?

Re: I indexed 669 GB of my GoPro videos using my M1 Max computer and local ML models

#59

Post by asdfasgasdgasdg »

Cool build but the example videos you provide at the end are . . . not what I would hope for when thinking about the highlights of 2000+ videos of biking? For example the dog barking video only has one scene repeated two or three times and it's five seconds long?

Re: I indexed 669 GB of my GoPro videos using my M1 Max computer and local ML models

#60

Post by ingvay7 »

iliashad wrote: post #55
ingvay7 wrote: post #40

Earlier quoted context omitted.

Really cool work and workflow. strongly prefer this kind of local, open pipeline that i control over a dependency on Adobe tools and lock ins.

I agree with that, thank you for your feedback. Also, maybe you're not a video editor and you just wanna search your videos. The video editing integrations are optional and you have full control. You can switch between Adobe Premiere Pro, Final cut Pro or Davinci Resolve

cannot wait to incorporate this to my workflow. thanks
Post reply on HN