Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen2.5-VL-32B: Smarter and Lighter

qwenlm.github.io

Post reply on HN
Print view View original

41–50 of 303 posts

Re: Qwen2.5-VL-32B: Smarter and Lighter

#41

Post by hmottestad »

jampa wrote: post #33
hmottestad wrote: post #24

Just don’t ask it about the tiananmen square massacre or you’ll get a security warning. Even if you rephrase it. It’ll happily talk about Bloody Sunday. Probably a great model, but it worries me that it has such restrictions. Sure OpenAI also has lots of restrictions, but this feels more like straight up censorship since it’ll happily go on about bad things the governments of the west have done.

DeepSeek's website seems to be using two models. The one that censors only does so in the online version. Are you saying that censoring happens with this model, even in the offline version?

I tried the R1 distill of llama 8B, which did refuse direct questions about the massacre.

Haven’t tried this new model locally, but I agree with you that it looks like there is a secondary censorship going on. If I ask it to list the 10 worst catastrophes of recent Chinese history with Thinking enabled then it’ll actually think about the massacre. Gets blocked very quickly, but it doesn’t look like the thinking is particularly censored.

Re: Qwen2.5-VL-32B: Smarter and Lighter

#42

Post by zamadatix »

jsheard wrote: post #22
echelon wrote: post #17

Earlier quoted context omitted.

Pretty soon I won't be using any American models. It'll be a 100% Chinese open source stack. The foundation model companies are screwed. Only shovel makers (Nvidia, infra companies) and product companies are going to win.

I still don't get where the money for new open source models is going to come from once setting investor dollars on fire is no longer a viable business model. Does anyone seriously expect companies to keep buying and running thousands of ungodly expensive GPUs, plus whatever they spend on human workers to do labelling/tuning, and then giving away the spoils for free, forever?

Once setting investment dollars on fire is no longer viable it'll probably be because scaling died anyways so what's the rush to have a dozen new frontier models each year.

Re: Qwen2.5-VL-32B: Smarter and Lighter

#43

Post by Imustaskforhelp »

api wrote: post #31
finnjohnsen2 wrote: post #26

Earlier quoted context omitted.

ads again. somehow. its like a law of nature.

If nationalist propaganda counts as ads, that might already be supporting Chinese models. Ask them about Tiananmen Square. Any kind of media with zero or near zero copying/distribution costs becomes a deflationary race to the bottom. Someone will eventually release something that's free, and at that point nothing can compete with free unless it's some kind of very specialized offering. Then you run into a the problem…

These ads can also have ads blockers though.

Perplexity released the deepseek r1 1331? ( I am not sure I forgot) It basically removes chinese censorships / yes you can ask it about the tiananmen square.

I think the next iteration of these ai model ads would be sneaky which might be hard to remove

Though it's funny you comment about chinese censorship yet american censorship is fine lol

Re: Qwen2.5-VL-32B: Smarter and Lighter

#44

Post by hmottestad »

generalizations wrote: post #28
hmottestad wrote: post #24

Just don’t ask it about the tiananmen square massacre or you’ll get a security warning. Even if you rephrase it. It’ll happily talk about Bloody Sunday. Probably a great model, but it worries me that it has such restrictions. Sure OpenAI also has lots of restrictions, but this feels more like straight up censorship since it’ll happily go on about bad things the governments of the west have done.

Nah, it's great for things that Western models are censored on. The True Hacker will keep an Eastern and Western model available, depending on what they need information on.

I tried to ask it about Java exploits that would allow me to gain RCE, but it refused just as most western models do.

That was the only thing I could think to ask really. Do you have a better example maybe?

Re: Qwen2.5-VL-32B: Smarter and Lighter

#45

Post by asadm »

hmottestad wrote: post #24

Just don’t ask it about the tiananmen square massacre or you’ll get a security warning. Even if you rephrase it. It’ll happily talk about Bloody Sunday. Probably a great model, but it worries me that it has such restrictions. Sure OpenAI also has lots of restrictions, but this feels more like straight up censorship since it’ll happily go on about bad things the governments of the west have done.

a) nobody, in production, asks those questions b) chatgpt is similarly biased on israel/palestine issue. Try making it agree that there is a genocide ongoing or on Palestinians right to defend themselves.

Re: Qwen2.5-VL-32B: Smarter and Lighter

#46

Post by Imustaskforhelp »

ada1981 wrote: post #30
jsheard wrote: post #22

Earlier quoted context omitted.

I still don't get where the money for new open source models is going to come from once setting investor dollars on fire is no longer a viable business model. Does anyone seriously expect companies to keep buying and running thousands of ungodly expensive GPUs, plus whatever they spend on human workers to do labelling/tuning, and then giving away the spoils for free, forever?

Money from the Chinese defense budget? Everyone using these models undercuts US companies. Eventually China wins.

And wez the end user get open source models.

Also china doesn't have access to that many gpus because of the chips act.

And i hate it , i hate it when america sounds more communist than china who open sources their stuff because free markets.

I actually think that more countries need to invest into AI and not companies wanting profit.

This could be the decision that can impact the next century.

Re: Qwen2.5-VL-32B: Smarter and Lighter

#47

Post by akra »

mitthrowaway2 wrote: post #32
jsheard wrote: post #22

Earlier quoted context omitted.

I still don't get where the money for new open source models is going to come from once setting investor dollars on fire is no longer a viable business model. Does anyone seriously expect companies to keep buying and running thousands of ungodly expensive GPUs, plus whatever they spend on human workers to do labelling/tuning, and then giving away the spoils for free, forever?

Maybe from NVIDIA? "Commoditize your product's complement". https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/

This is the reason IMO. Fundamentally China right now is better at manufacturing (e.g. robotics). AI is the complement to this - AI increases the demand for tech manufactured goods. Whereas America is in the opposite position w.r.t which side is their advantage (i.e. the software). AI for China is an enabler into a potentially bigger market which is robots/manufacturing/etc.

Commoditizing the AI/intelligence part means that the main advantage isn't the bits - its the atoms. Physical dexterity, social skills and manufacturing skills will gain more of a comparative advantage vs intelligence work in the future as a result - AI makes the old economy new again in the long term. It also lowers the value of AI investments in that they no longer can command first mover/monopoly like pricing for what is a very large capex cost undermining US investment in what is their advantage. As long as it is strategic, it doesn't necessarily need to be economic on its own.

Re: Qwen2.5-VL-32B: Smarter and Lighter

#48

Post by disgruntledphd2 »

wetwater wrote: post #38
faizshah wrote: post #35

Earlier quoted context omitted.

I just started self hosting as well on my local machine, been using https://lmstudio.ai/ Locally for now. I think the 32b models are actually good enough that I might stop paying for ChatGPT plus and Claude. I get around 20 tok/second on my m3 and I can get 100 tok/second on smaller models or quantized. 80-100 tok/second is the best for interactive usage if you go above that you basically can’t read as fast as it gen…

Are there any good sources that I can read up on estimiating what would be hardware specs required for 7B, 13B, 32B .. etc size If I need to run them locally? I am grad student on budget but I want to host one locally and trying to build a PC that could run one of these models.

MacBook with 64gb RAM will probably be the easiest. As a bonus, you can train pytorch models on the built in GPU.

It's really frustrating that I can't just write off Apple as evil monopolists when they put out hardware like this.

Re: Qwen2.5-VL-32B: Smarter and Lighter

#49

Post by refulgentis »

echelon wrote: post #17
simonw wrote: post #4

Big day for open source Chinese model releases - DeepSeek-v3-0324 came out today too, an updated version of DeepSeek v3 now under an MIT license (previously it was a custom DeepSeek license). https://simonwillison.net/2025/Mar/24/deepseek/

Pretty soon I won't be using any American models. It'll be a 100% Chinese open source stack. The foundation model companies are screwed. Only shovel makers (Nvidia, infra companies) and product companies are going to win.

I've been waiting since November for 1, just 1*, model other than Claude than can reliably do agentic tool call loops. As long as the Chinese open models are chasing reasoning and benchmark maxxing vs. mid-2024 US private models, I'm very comfortable with somewhat ignoring these models.

(this isn't idle prognostication hinging on my personal hobby horse. I got skin in the game, I'm virtually certain I have the only AI client that is able to reliably do tool calls with open models in an agentic setting. llama.cpp got a massive contribution to make this happen and the big boys who bother, like ollama, are still using a dated json-schema-forcing method that doesn't comport with recent local model releases that can do tool calls. IMHO we're comfortably past a point where products using these models can afford to focus on conversational chatbots, thats cute but a commodity to give away per standard 2010s SV thinking)

* OpenAI's can but are a little less...grounded?...situated? i.e. it can't handle "read this file and edit it to do $X". Same-ish for Gemini, though, sometimes I feel like the only person in the world who actually waits for the experimental models to go GA, as per letter of the law, I shouldn't deploy them until then

Re: Qwen2.5-VL-32B: Smarter and Lighter

#50

Post by theturtletalks »

generalizations wrote: post #28
hmottestad wrote: post #24

Just don’t ask it about the tiananmen square massacre or you’ll get a security warning. Even if you rephrase it. It’ll happily talk about Bloody Sunday. Probably a great model, but it worries me that it has such restrictions. Sure OpenAI also has lots of restrictions, but this feels more like straight up censorship since it’ll happily go on about bad things the governments of the west have done.

Nah, it's great for things that Western models are censored on. The True Hacker will keep an Eastern and Western model available, depending on what they need information on.

Wouldn’t they just run R1 locally and not have any censorship at all? The model isn’t censored at its core, it’s censored through the system prompt. Perplexity and Huggingface have their own versions of R1 that is not censored.
Post reply on HN