Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Gemini can now natively embed video, so I built sub-second video search

github.com

Post reply on HN
Print view View original

61–70 of 118 posts

Re: Show HN: Gemini can now natively embed video, so I built sub-second video search

#61

Post by macNchz »

greggsy wrote: post #52
macNchz wrote: post #29

This is a really cool implementation—embeddings still often feel like magic to me. That said, this exact use case is sort of also my biggest point of concern with where AI takes us, much more so than most of the common AI risks you hear lots of chatter about. We live in a world absolutely loaded with cameras now but ultimately retain some semblance of semi-anonymity/privacy in public by virtue of the fact that nobody…

All the major cloud providers offer some form of face detection and numberplate reading, with many supporting object detection (ie package, vehicle, person) out of the camera itself.

It's definitely creeping into things, though most of the features I've seen are fairly simplistic compared to what would be possible if the video was being reviewed + indexed by current SoTA multimodal LLMs.

Re: Show HN: Gemini can now natively embed video, so I built sub-second video search

#62

Post by jimmySixDOF »

sohamrj wrote: post #35
macNchz wrote: post #29

This is a really cool implementation—embeddings still often feel like magic to me. That said, this exact use case is sort of also my biggest point of concern with where AI takes us, much more so than most of the common AI risks you hear lots of chatter about. We live in a world absolutely loaded with cameras now but ultimately retain some semblance of semi-anonymity/privacy in public by virtue of the fact that nobody…

Totally valid concern. Right now the cost ($2.50/hr) and latency make continuous real-time indexing impractical, but that won't always be the case. This is one of the reasons I'd want to see open-weight local models for this, keeps the indexing on your own hardware with no footage leaving your machine. But you're right that the broader trajectory here is worth thinking carefully about.

How are you getting to $2.50/hr ? The price sheet says its 0.00079 per frame.

https://ai.google.dev/gemini-api/docs/pricing#gemini-embeddi...

Re: Show HN: Gemini can now natively embed video, so I built sub-second video search

#64

Post by sohamrj »

WatchDog wrote: post #55

I don't quite understand the 5 second overlap. I assume it's so that events that occur over the chunk boundary don't get missed, but is there any examples or benchmarking to examine how useful this is?

yea, it's so events on a chunk boundary still get captured in at least one chunk. i haven't had the chance to do formal benchmarks on overlap vs. no-overlap yet. the 5s default is a pragmatic choice, long enough to catch most events that would otherwise be split, short enough to not add much cost (120 chunks/hr to ~138). also it's configurable via the --overlap flag.

Re: Show HN: Gemini can now natively embed video, so I built sub-second video search

#67

Post by jjwiseman »

jimmySixDOF wrote: post #62
sohamrj wrote: post #35

Earlier quoted context omitted.

Totally valid concern. Right now the cost ($2.50/hr) and latency make continuous real-time indexing impractical, but that won't always be the case. This is one of the reasons I'd want to see open-weight local models for this, keeps the indexing on your own hardware with no footage leaving your machine. But you're right that the broader trajectory here is worth thinking carefully about.

How are you getting to $2.50/hr ? The price sheet says its 0.00079 per frame. https://ai.google.dev/gemini-api/docs/pricing#gemini-embeddi...

From what I see the code downsamples video to 5 fps, so 1 hour of video is 3600 seconds * 5 fps = 18,000 frames. 18,000 frames * $0.00079/frame = $14.22. A couple dollars more with the overlap.

(The code also tries to skip "still" frames, but if your video is dynamic you're looking at the cost above.)

Re: Show HN: Gemini can now natively embed video, so I built sub-second video search

#68

Post by whattheheckheck »

cake_robot wrote: post #41
macNchz wrote: post #29

This is a really cool implementation—embeddings still often feel like magic to me. That said, this exact use case is sort of also my biggest point of concern with where AI takes us, much more so than most of the common AI risks you hear lots of chatter about. We live in a world absolutely loaded with cameras now but ultimately retain some semblance of semi-anonymity/privacy in public by virtue of the fact that nobody…

Yeah, the panopticon is now technically very feasible it's just expensive to implement (for now).

Its very cheap to target an individual though so they dont need to look everywhere

Re: Show HN: Gemini can now natively embed video, so I built sub-second video search

#69

Post by FuckButtons »

macNchz wrote: post #29

This is a really cool implementation—embeddings still often feel like magic to me. That said, this exact use case is sort of also my biggest point of concern with where AI takes us, much more so than most of the common AI risks you hear lots of chatter about. We live in a world absolutely loaded with cameras now but ultimately retain some semblance of semi-anonymity/privacy in public by virtue of the fact that nobody…

Once the hardware to run inference for something like the vision understanding module of this can be run on a low / medium power asic drones are going to be absolutely horrifying weapons.

Re: Show HN: Gemini can now natively embed video, so I built sub-second video search

#70

Post by zahlman »

macNchz wrote: post #29

This is a really cool implementation—embeddings still often feel like magic to me. That said, this exact use case is sort of also my biggest point of concern with where AI takes us, much more so than most of the common AI risks you hear lots of chatter about. We live in a world absolutely loaded with cameras now but ultimately retain some semblance of semi-anonymity/privacy in public by virtue of the fact that nobody…

> this exact use case is sort of also my biggest point of concern with where AI takes us, much more so than most of the common AI risks you hear lots of chatter about.

I've been hearing warnings that AI would be used for this since well before it seemed feasible.

Post reply on HN