Ive used qwen3-VL on deepwalker lately. All I can stay is that this model is so underrated.
Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
51–60 of 86 posts
Re: Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
#52Not so relevant to the thread but ive been uploading screenshots from citrix guis and asking qwen3-vl for the appropriate next action eg Mouseclick, and while it knows what to click it struggles to accurately return which pixel coordinates to click. Anyone know a way to get accurate pixel coordinates returned?
How do you prompt the model? In my experience, Qwen3-VL models have very accurate grounding capabilities (I’ve tested Qwen3-VL-30B-A3B-Instruct, Qwen3-VL-30B-A3B-Thinking, and Qwen3-VL-235B-A22B-Thinking-FP8).
Note that the returned values are not direct pixel coordinates. Instead, they are normalized to a 0–1000 range. For example, if you ask for a bounding box, the model might output:
```json [ {"bbox_2d": [217, 112, 920, 956], "label": "cat"} ] ```
Here, the values represent [x_min, y_min, x_max, y_max]. To convert these to pixel coordinates, use:
[x_min / 1000 * image_width, y_min / 1000 * image_height, x_max / 1000 * image_width, y_max / 1000 * image_height]
Also, if you’re running the model with vLLM > 0.11.0, you might be hitting this bug: https://github.com/vllm-project/vllm/issues/29595
Re: Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
#53For anyone using Qwen3-VL: where are you running it? I had tons of reliability problems with Qwen3-VL inference providers on OpenRouter — based on uptime graphs I wasn’t alone. But when it worked, Qwen3-VL was pack-leading good at AI Vision stuff.
I run it on ollama
the big boy model?
Re: Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
#54Re: Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
#55Not so relevant to the thread but ive been uploading screenshots from citrix guis and asking qwen3-vl for the appropriate next action eg Mouseclick, and while it knows what to click it struggles to accurately return which pixel coordinates to click. Anyone know a way to get accurate pixel coordinates returned?
It’s been about a year since I looked into this sort of thing, but molmo will give you x,y coordinates. I hacked together a project about it. I also think Microsoft’s omniparser is good at finding coordinates too. https://huggingface.co/allenai/Molmo-7B-D-0924 https://github.com/logankeenan/george https://github.com/microsoft/OmniParser
Thanks ill try this!
Re: Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
#56Not so relevant to the thread but ive been uploading screenshots from citrix guis and asking qwen3-vl for the appropriate next action eg Mouseclick, and while it knows what to click it struggles to accurately return which pixel coordinates to click. Anyone know a way to get accurate pixel coordinates returned?
How do you prompt the model? In my experience, Qwen3-VL models have very accurate grounding capabilities (I’ve tested Qwen3-VL-30B-A3B-Instruct, Qwen3-VL-30B-A3B-Thinking, and Qwen3-VL-235B-A22B-Thinking-FP8). Note that the returned values are not direct pixel coordinates. Instead, they are normalized to a 0–1000 range. For example, if you ask for a bounding box, the model might output: ```json [ {"bbox_2d": [217, 11…
Will give this a go, cheers :)
Re: Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
#57Not so relevant to the thread but ive been uploading screenshots from citrix guis and asking qwen3-vl for the appropriate next action eg Mouseclick, and while it knows what to click it struggles to accurately return which pixel coordinates to click. Anyone know a way to get accurate pixel coordinates returned?
It's very not accurate, but sometimes instructing to return pyautogui code works. prompt: I attach a screenshot (1920x1080). Write code to click the submit button using pyautogui. attachment: reply: import pyautogui pyautogui.click(100, 200)
Ive been asking for pyautogui output already but it is still very hit and miss
Re: Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
#58Hope this on day will be used for auto-tagging all video assets with time codes. The dream of being able to search for running horse and find a clip containing a running horse at 4m42s in one of thousands of clips.
It’s not difficult to hack this together with CLIP. I did this with about a tenth of my movie collection last week with a GTX 1080 - though it lacks temporal understanding so you have to do the scene analysis yourself
I'm guessing you're not storing the CLIP for every single frame, instead of every second or so? Also, are you using the cosine similarity? How are you finding the nearest vector?
Re: Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
#59Re: Qwen3-VL can scan two-hour videos and pinpoint nearly every detail
#60Does anyone else worry about this technology used for Big Brother type surveillance?
How do you think this tech was developed in the first place? It's probably trained and used in the surveillance bid for a decade before it comes to consumers, and this probably isn't the SoA stuff that governments have access to, we're probably 5-10 years behind what's on the cutting edge.
I wouldn’t bet. IT innovation used to be lead by the defence industry, but that has changed and now consumer technology is driving the innovation from what I have been told.
I’m sure they have some cool secret stuff, but they are perhaps not 10 years ahead. Also, I find unlikely that those secrets wouldn’t make it to the public society now, as we are probably close the top of the AI bubble.