Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3-VL

qwen.ai

Post reply on HN
Print view View original

111–120 of 166 posts

Re: Qwen3-VL

#111

Post by vessenes »

ralusek wrote: post #63
vessenes wrote: post #53

I spent a little time with the thinking model today. It's good. It's not better than GPT5 Pro. It might be better than the smallest GPT 5, though. My current go-to test is to ask the LLM to construct a charging solution for my macbook pro with the model on it, but sadly, I and the pro have been sent to 15th century Florence with no money and no charger. I explain I only have two to three hours of inference time, whic…

I JUST had a very intense dream that there was a catastrophic event that set humanity back massively, to the point that the internet was nonexistent and our laptops suddenly became priceless. The first thought I had was absolutely hating myself for not bothering to download a local LLM. A local LLM at the level of qwen is enough to massively jump start civilization.

Yeah bring Qwen and OSS-120b for sure. You’re going to want some solar panels with usb-c output tho

Re: Qwen3-VL

#112

Post by vessenes »

nl wrote: post #59
vessenes wrote: post #53

I spent a little time with the thinking model today. It's good. It's not better than GPT5 Pro. It might be better than the smallest GPT 5, though. My current go-to test is to ask the LLM to construct a charging solution for my macbook pro with the model on it, but sadly, I and the pro have been sent to 15th century Florence with no money and no charger. I explain I only have two to three hours of inference time, whic…

> predicting Halley's comet, and in exchange demanding a workshop and 20 copper pennies from the Medicis I love this! Simple and probably effective (or would get you killed for witchcraft)

Hard in that you might have to starve for a few decades though. I’d prefer interest rate arb based on competing city state aggression

Re: Qwen3-VL

#113

Post by Keyframe »

ricardobeat wrote: post #109
Keyframe wrote: post #93

Earlier quoted context omitted.

That's AMD Ryzen AI Max+ 395, right? Lots of those boxes popping up recently, but isn't that dog slow? And I can't believe I'm saying this - but maybe RAM filled-up mac might be a better option?

Yes, but the mac costs 3-4x more. You can get one of these 395 systems with 96GB for ~1k.

When I was looking it was more like 1.6k euros, but still great price. Mac studio with M4 Max 16/40/16 with 128GB is double that. That's all within a range of "affordable". Now, if it's at least twice the speed, I don't see a reason not to. Even though my religion is against buying a mac as well.

edit, just took a look at amazon. GMKtec EVO-X2 AI, which is the AMD Ryzen AI Max+ 395 with 128GB of RAM is 3k euros. Mac M4 Max with 16 cores and 128 gigs is 4.4k euros. Damn, Europe. If you go with M4 Max with 14 cores, but still 16 cores of "Neural engine"... ah, you can't get 128 GB of RAM then. Classic Apple :)

edit2: look at gmktec site itself. machine is 2k euros there. Damn, amazon.

Re: Qwen3-VL

#114

Post by benterix »

richardlblair wrote: post #23

As I mentioned yesterday - I recently needed to process hundreds of low quality images of invoices (for a construction project). I had a script that had used pil/opencv, pytesseract, and open ai as a fallback. It still has a staggering number of failures. Today I tried a handful of the really poor quality invoices and Qwen spat out all the information I needed without an issue. What's crazier is it gave me the boundi…

I wonder why you chose Qwen specifically - Mistral has a specialized model just for OCR that they advertised heavily (I tested it and it works surprisingly well, at least on English-language books from 80s and 90s).

Re: Qwen3-VL

#115

Post by captainregex »

unixhero wrote: post #47
richardlblair wrote: post #23

As I mentioned yesterday - I recently needed to process hundreds of low quality images of invoices (for a construction project). I had a script that had used pil/opencv, pytesseract, and open ai as a fallback. It still has a staggering number of failures. Today I tried a handful of the really poor quality invoices and Qwen spat out all the information I needed without an issue. What's crazier is it gave me the boundi…

So where did you load up Qwen and how did you supply the pdf or photo files? I don't know how to use these models, but want to learn

AnythingLLM also good for that GUI experience!

Re: Qwen3-VL

#116

Post by captainregex »

captainregex wrote: post #115
unixhero wrote: post #47

Earlier quoted context omitted.

So where did you load up Qwen and how did you supply the pdf or photo files? I don't know how to use these models, but want to learn

AnythingLLM also good for that GUI experience!

I should add that sometimes LM Studio just feels better for the use case, same model same purpose seemingly different output usually when involving RAG, but Anything is definitely a very intuitive visual experience

Re: Qwen3-VL

#117

Post by indigoabstract »

VladVladikoff wrote: post #33
richardlblair wrote: post #28

Earlier quoted context omitted.

With Qwen I went as stupid as I could: please provide the bounding box metadata for pytesseract for the above image. And it spat it out.

It’s funny that many of us say please. I don’t think it impacts the output, but it also feels wrong without it sometimes.

The way I think of it, talking to an LLM is a bit like talking to myself or listening to an echo, since what I get back depends only on what I put in. If it senses that I'm frustrated, it will be inclined to make even more stuff up in an attempt to appease me, so that gets me nowhere.

I've found it more useful to keep it polite and "professional" and restart the conversation if we've begun going around in circles.

And besides, if I make a habit of behaving badly with LLMs, there's a good chance that I'll do it without thinking at some point and get in trouble.

Re: Qwen3-VL

#118

Post by bitflourjikg »

vardump wrote: post #79

So 235B parameter Qwen3-VL is FP16, so practically it requires at least 512 GB RAM to run? Possibly even more for a reasonable context window? Assuming I don’t want to run it on a CPU, what are my options to run it at home under $10k? Or if my only option is to run the model with CPU (vs GPU or other specialized HW), what would be the best way to use that 10k? vLLM + Multiple networked (10/25/100Gbit) systems?

A non-CPU setup will very likely require an electrical service upgrade or tactical positioning of different systems on different circuits for you to be able to run models that large. Several kW setups also cost non-trivial sums of money to run usually

Re: Qwen3-VL

#119

Post by Borealid »

ramon156 wrote: post #101
Borealid wrote: post #96

Earlier quoted context omitted.

Did you casually glance at how the hardware in the Framework Desktop (Strix Halo) works before commenting?

These don't run 200B models at all, results show it can run 13B at best. 70B is ~3 tk / s according to someone on Reddit.

I don't know where you've got those numbers, but they're wrong.

https://www.reddit.com/r/LocalLLaMA/comments/1n79udw/inferen... seems comparable to the Framework Desktop and reputable - they didn't just quote a number, they showed benchmark output.

I get far more than 3 t/s for a 70B model on normal non-unified RAM, so that's completely unfeasible performance for a unified memory architecture like Halo.

Re: Qwen3-VL

#120

Post by te0006 »

Keyframe wrote: post #113
ricardobeat wrote: post #109

Earlier quoted context omitted.

Yes, but the mac costs 3-4x more. You can get one of these 395 systems with 96GB for ~1k.

When I was looking it was more like 1.6k euros, but still great price. Mac studio with M4 Max 16/40/16 with 128GB is double that. That's all within a range of "affordable". Now, if it's at least twice the speed, I don't see a reason not to. Even though my religion is against buying a mac as well. edit, just took a look at amazon. GMKtec EVO-X2 AI, which is the AMD Ryzen AI Max+ 395 with 128GB of RAM is 3k euros. Mac…

Note that the small print on the GMKtec site says that prices do not include customs and VAT. Which seem to amount to 19% in the EU. So, almost 2,4K€.
Post reply on HN