71–80 of 86 posts
Earlier quoted context omitted.
Seems cool from the docs page, I was about to give it a shot but https://github.com/vlm-run/mm goes 404 …
It’s unclear if that’s intentional since it’s listed also under open source on the main company site: https://www.vlm.run/open-source/mm
In either case, here you go, it's public now: https://github.com/vlm-run/mm.
"Where the video goes: stays on your machine" - No, the frames (that this tool extracts) obviously get sent to Anthropic if you use Claude.
Pretty terribly expensive way to watch a video with Claude. Use Gemini or some local VLM to do this way more efficiently. We spent quite a bit of time on video understanding, and Claude will just burn tokens. Check out this library: https://vlm-run.github.io/mm/ You can swap models and try out different encoding methods for videos ( https://vlm-run.github.io/mm/encoders/#video )
Assuming that's your project, the GitHub link from the PyPi page is a 404.
Pretty terribly expensive way to watch a video with Claude. Use Gemini or some local VLM to do this way more efficiently. We spent quite a bit of time on video understanding, and Claude will just burn tokens. Check out this library: https://vlm-run.github.io/mm/ You can swap models and try out different encoding methods for videos ( https://vlm-run.github.io/mm/encoders/#video )
Assuming that's your project, the GitHub link from the PyPi page is a 404.
Cool idea, but keyframes are not videos. Motion, object permanence, are not things Claude can infer from a set of images. Nice demo though!
Exactly! We experimented with a whole bunch of video encoding techniques for LLMs here: https://vlm-run.github.io/mm/encoders/#video
Cool idea, but keyframes are not videos. Motion, object permanence, are not things Claude can infer from a set of images. Nice demo though!
How do you handle things like scrolling quickly in a video?
Hi HN! I built this because I was frustrated that no LLM actually "sees" a video — Claude won't accept video files, ChatGPT reads the transcript only, and Gemini samples at a fixed 1fps (missing fast cuts, over-sampling static slides). claude-real-video takes a URL or local file and: 1. Extracts frames at every scene change (not fixed intervals) + a density floor 2. Deduplicates with a sliding-window pixel-diff algor…
I gave Claude a video provided by a county attorney for a speeding ticket I got. It was spot on in its analysis, even though I don’t like what the video showed. What does it mean that Claude can’t view video; it did it just fine. Or do you mean tool less?
Hi HN! I built this because I was frustrated that no LLM actually "sees" a video — Claude won't accept video files, ChatGPT reads the transcript only, and Gemini samples at a fixed 1fps (missing fast cuts, over-sampling static slides). claude-real-video takes a URL or local file and: 1. Extracts frames at every scene change (not fixed intervals) + a density floor 2. Deduplicates with a sliding-window pixel-diff algor…
Very cool I have something that does this as well along these lines. I’ll dig into yours over the next few days and contribute where and if I can too, awesome to see!
Are models any good at descerning motion from multiple frames? For instance if I gave models multiple animations of a bouncing ball as individual frames. Would they be able to tell which bounce was the more realistic motion. (Is this a potential new benchmark? maybe also variations of stair dismount)