so, a great politician once said: "the internet is for p0rn" - and if we accept this, what will this thing pinpoint in p0rn videos? :-D LOL
There it is! Oh, it disappeared. There it is again! Oh, it disappeared. It’s back! Wait, no.
Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
81–86 of 86 posts
Re: Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
#82> The test works by inserting a semantically important "needle" frame at random positions in long videos, which the system must then find and analyze. This seems to be somewhat unwise. Such an insertion would qualify as an anomaly. And if it's also trained that way, would you not train the model to find artificial frames where they don't belong? Would it not have been better to find a set of videos where something sp…
Yeah the needle in a haystack tests are so stupid. It seems clear with LLMs that performance degrades massively with context size, yet those tests claim the model performs perfectly.
Re: Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
#83My favorite AI feature is to put a YouTube link in Gemini and ask it to summarize. Or even better: put the link of a 20min video "5 ways to" and ask "what are the 5 ways?" I think Gemini analyzes the transcription. Can I do the same for free with Qwen3?
Same - I see a lot of "vaguely interesting but no way I'm spending 40 minutes on that" kind of videos, and it usually works. However, I have noticed it occasionally will just summarize the wrong video for me. It might be if the video is very new, or something? I'm not sure.
Re: Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
#84Earlier quoted context omitted.
It’s not difficult to hack this together with CLIP. I did this with about a tenth of my movie collection last week with a GTX 1080 - though it lacks temporal understanding so you have to do the scene analysis yourself
Would you be willing to share more details of what you did?
Sure. I had a lot of help from Claude Opus 4.5, but it was roughly:
- Using pyscenedetect to split each video on a per scene level
- Using the decord library https://github.com/dmlc/decord to pull frames from each scene at a particular sample rate (specific rate I don't have handy right now, but it was 1-2 per scene)
- Aggregating frames in batches of around 256 frames to be normalized for CLIP embedding on GPU (had to re-write the normalization process for this because the default library does it on CPU)
- Uploading the frames along with metadata (timestamp, etc) into a vector DB, in my case Qdrant running locally along with a screenclip of the frame itself for debugging.
I'm bottlenecked by GPU compute so I also started experimenting with using Modal for the embedding work too, but then vacation ended :) Might pick it up again in a few weeks. I'd like to be able to have a temporal-aware and potentially enriched search so that I can say "Seek to the scene in Oppenheimer where Rami Malek testifies" and be able to get a timestamped clip from the movie.
Re: Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
#85For anyone using Qwen3-VL: where are you running it? I had tons of reliability problems with Qwen3-VL inference providers on OpenRouter — based on uptime graphs I wasn’t alone. But when it worked, Qwen3-VL was pack-leading good at AI Vision stuff.
My company's GPU cluster
Re: Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
#86Hope this on day will be used for auto-tagging all video assets with time codes. The dream of being able to search for running horse and find a clip containing a running horse at 4m42s in one of thousands of clips.
Gemini already does this (and has for awhile): https://ai.google.dev/gemini-api/docs/video-understanding