"Where the video goes: stays on your machine" - No, the frames (that this tool extracts) obviously get sent to Anthropic if you use Claude.
Claude-real-video － any LLM can watch a video
11–20 of 86 posts
Re: Claude-real-video － any LLM can watch a video
#12Hi HN! I built this because I was frustrated that no LLM actually "sees" a video — Claude won't accept video files, ChatGPT reads the transcript only, and Gemini samples at a fixed 1fps (missing fast cuts, over-sampling static slides). claude-real-video takes a URL or local file and: 1. Extracts frames at every scene change (not fixed intervals) + a density floor 2. Deduplicates with a sliding-window pixel-diff algor…
I gave Claude a video provided by a county attorney for a speeding ticket I got. It was spot on in its analysis, even though I don’t like what the video showed. What does it mean that Claude can’t view video; it did it just fine. Or do you mean tool less?
yeah im pretty sure claude code can handle videos. its been doing frame by frame analysis for me with generated video to iterate on pipelines
Re: Claude-real-video － any LLM can watch a video
#13Cool idea, but keyframes are not videos. Motion, object permanence, are not things Claude can infer from a set of images. Nice demo though!
Re: Claude-real-video － any LLM can watch a video
#14This looks cool but this should be renamed without having Claude in the name.
Re: Claude-real-video － any LLM can watch a video
#15This looks cool but this should be renamed without having Claude in the name.
llm-real-video would be a much better name
Re: Claude-real-video － any LLM can watch a video
#16I was just thinking about this exact use case yesterday: And it's for me measuring different charged speeds at different starting battery capacities and different temperatures and I was like well. What if I just had a video camera pointing at the voltage going in and out and then I could see the battery percentage increase and I can have a temperature gun pointed at the phone as well. And I couldn't know what tempera…
It's kind of wild how much we are abandoning basic problem solving skills in favor of just pointing an enormous stack of GPUs at it
Re: Claude-real-video － any LLM can watch a video
#17I was just thinking about this exact use case yesterday: And it's for me measuring different charged speeds at different starting battery capacities and different temperatures and I was like well. What if I just had a video camera pointing at the voltage going in and out and then I could see the battery percentage increase and I can have a temperature gun pointed at the phone as well. And I couldn't know what tempera…
It's kind of wild how much we are abandoning basic problem solving skills in favor of just pointing an enormous stack of GPUs at it
Identifying objects in pictures was considered an insurmountable task only a few years ago, like in the xckd comic https://xkcd.com/1425/
Re: Claude-real-video － any LLM can watch a video
#18Pretty terribly expensive way to watch a video with Claude.
Use Gemini or some local VLM to do this way more efficiently. We spent quite a bit of time on video understanding, and Claude will just burn tokens.
Check out this library: https://vlm-run.github.io/mm/
You can swap models and try out different encoding methods for videos (https://vlm-run.github.io/mm/encoders/#video)
Re: Claude-real-video － any LLM can watch a video
#19Cool idea, but keyframes are not videos. Motion, object permanence, are not things Claude can infer from a set of images. Nice demo though!
Exactly! We experimented with a whole bunch of video encoding techniques for LLMs here: https://vlm-run.github.io/mm/encoders/#video
Re: Claude-real-video － any LLM can watch a video
#20Cool idea, but keyframes are not videos. Motion, object permanence, are not things Claude can infer from a set of images. Nice demo though!
I have been going through this with claude and qwenvl3:8b this week. Both are pretty decent at inferring context and analyzing contact sheets. Finding high visual interest moments with a mixture of coarse and fine keyframes.