Another day another Qwen model
Qwen3-VL
151–160 of 166 posts
Re: Qwen3-VL
#152The Chinese are doing what they have been doing to the manufacturing industry as well. Take the core technology and just optimize, optimize, optimize for 10x the cost/efficiency. As simple as that. Super impressive. These models might be bechmaxxed but as another comment said, i see so many that it might as well be the most impressive benchmaxxing today, if not just a genuinely SOTA open source model. They even relea…
They still suck at explaining which model they serve is which, though. They also released today Qwen3-VL Plus [1] today alongside Qwen3-VL 235B [2] and they don't tell us which one is better. Note that Qwen3-VL-Plus is a very different model compared to Qwen-VL-Plus. Also, qwen-plus-2025-09-11 [3] vs qwen3-235b-a22b-instruct-2507 [4]. What's the difference? Which one is better? Who knows. You know it's bad when OpenA…
it's sometimes not really a matter of which one is better but which one fits best.
For example many have switched to qwen3 models but some still vastly prefer the reasoning and output of QwQ (a qwen2.5 model).
And the difference between them: those with "plus" are closed weight, you can only access them through their api. The others are open-weight, so if they fit your use case, and if ever the want or need arise, you can download them, use them, even fine-tune them locally, even if qwen don't offer access to them any more.
Re: Qwen3-VL
#153As I mentioned yesterday - I recently needed to process hundreds of low quality images of invoices (for a construction project). I had a script that had used pil/opencv, pytesseract, and open ai as a fallback. It still has a staggering number of failures. Today I tried a handful of the really poor quality invoices and Qwen spat out all the information I needed without an issue. What's crazier is it gave me the boundi…
Interesting. I have in the past tried to get bounding boxes of property boundaries on satellite maps estimated by VLLM models but had no success. Do you have any tips on how to improve the results?
shameless plug here for AMD's AI Dev Day - registration is open and they want feedback on what to focus on: https://www.amd.com/en/corporate/events/amd-ai-dev-day.html
Re: Qwen3-VL
#154Earlier quoted context omitted.
I’ve tried that too, trying to detect the scan layout to get better OCR, but it didn’t really beat a fine-tuned Qwen 2.5 VLM 7B. I’d say fine-tuning is the way to go
What's the cost of the fine-tuned model? If you were attempting to optimize for cost, would it be worth it to detect scan layouts to get better OCR? Honestly, I'm such a noob in this space. I had 1 project I needed to do, didn't want to do it by hand which would have taken 2 days so I spent 5 trying to get a script to do it for me.
the model runs on H200 in ~20s, costing about $2.4/hr. on L4 it’s cheaper at ~$0.3/hr but takes ~85s to finish. overall, H200 ends up cheaper at volume. my scan has a separate issue though: each page has two columns, so text from the right side sometimes overflows into the left. OCR can’t really tell where sentences start and end unless the layout is split by column.
Re: Qwen3-VL
#155Earlier quoted context omitted.
I’ve tried that too, trying to detect the scan layout to get better OCR, but it didn’t really beat a fine-tuned Qwen 2.5 VLM 7B. I’d say fine-tuning is the way to go
what fine tuning approach did you use?
just unsloth on colab using A100 and dataset on google drive.
Re: Qwen3-VL
#156Earlier quoted context omitted.
There seems to be a EU shop as well, but I can't see it's without VAT, not even on checkout page. There's a 50 EUR discount code though.
Loooking closely, the shop does not seem to be located within the EU. And the 50€ discount does not apply to the 128GB config. Also, if you are interested, it might help to have a look into the user forum: https://de.gmktec.com/community/xenforum
ooof. thanks for that
Re: Qwen3-VL
#157Sadly it still fails the "extra limb" test. I have a few images of animals with an extra limb photoshopped onto them. A dog with an leg coming out of it's stomach, or a cat with two front right legs. Like every other model I have tested, it insists that the animals have their anatomically correct amount of limbs. Even pointing out there is a leg coming from the dogs stomach, it will push back and insist I am confused…
I wonder if you used their image editing feature if it would insist on “correcting” the number of limbs even if you asked for unrelated changes.
It will. I actually made a test when NanoBanana first went GA which featured a photo of a one-legged man and asked the model to change the clothing into pants. It added the pants as requested and then proceeded to "heal" his missing leg in the process.
Very difficult for even SOTA to go against data that is as well-represented as bipedal humanoids.
Re: Qwen3-VL
#158Earlier quoted context omitted.
They still suck at explaining which model they serve is which, though. They also released today Qwen3-VL Plus [1] today alongside Qwen3-VL 235B [2] and they don't tell us which one is better. Note that Qwen3-VL-Plus is a very different model compared to Qwen-VL-Plus. Also, qwen-plus-2025-09-11 [3] vs qwen3-235b-a22b-instruct-2507 [4]. What's the difference? Which one is better? Who knows. You know it's bad when OpenA…
it's sometimes not really a matter of which one is better but which one fits best. For example many have switched to qwen3 models but some still vastly prefer the reasoning and output of QwQ (a qwen2.5 model). And the difference between them: those with "plus" are closed weight, you can only access them through their api. The others are open-weight, so if they fit your use case, and if ever the want or need arise, yo…
If the naming is so clear to you, then why don't you explain: for a user who wants to use Qwen3-VL through an API, which one has better performance? Qwen3-VL Plus or Qwen3-VL 235b?
Re: Qwen3-VL
#159Earlier quoted context omitted.
Surprising this is the first time I’ve seen anyone say this out loud.
Because it doesn’t make sense. The reason there’s a bubble is investor belief that AI will unlock tons of value. The reason the bubble is concentrated in silicon and model providers is because investors believe they have the most leverage to monetize this new value in the short term. If all of that stuff becomes free, the money will just move a few layers up to all of the companies whose cost structure has suddenly b…
How does it not make sense for Chinese AI companies to undermine American proprietary AI using open source AI? That’s already happened somewhat openly and dramatically with DeepSeek. In fact it seems to me that that was the moment when public perception about the inevitability of OpenAI dominance began to falter.
Re: Qwen3-VL
#160Earlier quoted context omitted.
These don't run 200B models at all, results show it can run 13B at best. 70B is ~3 tk / s according to someone on Reddit.
I don't know where you've got those numbers, but they're wrong. https://www.reddit.com/r/LocalLLaMA/comments/1n79udw/inferen... seems comparable to the Framework Desktop and reputable - they didn't just quote a number, they showed benchmark output. I get far more than 3 t/s for a 70B model on normal non-unified RAM, so that's completely unfeasible performance for a unified memory architecture like Halo.
It depends on the model.
It's typically ok for MoE models but if you try to run something non-MoE the speed will plummet. In that same thread there are people getting 50 tok/s on MoE models and 5 on non MoE. (https://www.reddit.com/r/LocalLLaMA/comments/1n79udw/comment...)
And while it has unified memory the memory is quite slow. 250GB/s compared to 500+ for M4 Max or 1800 GB/s for a 5090. So it's fast for a CPU, but pretty slow for a GPU.
(That said, there are not a lot of cheap options for running large models locally. They all have significant compromises.)