Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen2.5-VL-32B: Smarter and Lighter

qwenlm.github.io

Post reply on HN
Print view View original

291–300 of 303 posts

Re: Qwen2.5-VL-32B: Smarter and Lighter

#291

Post by refulgentis »

cess11 wrote: post #283
refulgentis wrote: post #49

Earlier quoted context omitted.

I've been waiting since November for 1, just 1*, model other than Claude than can reliably do agentic tool call loops. As long as the Chinese open models are chasing reasoning and benchmark maxxing vs. mid-2024 US private models, I'm very comfortable with somewhat ignoring these models. (this isn't idle prognostication hinging on my personal hobby horse. I got skin in the game, I'm virtually certain I have the only A…

You mean like https://manusai.ai/ is supposed to function?

Yes, exactly, and no trivially: Manus is Sonnet with tools

Re: Qwen2.5-VL-32B: Smarter and Lighter

#292

Post by maleldil »

mistrial9 wrote: post #279
simonw wrote: post #274

Earlier quoted context omitted.

Right: I could give you a recipe that tells you to first create a Python virtual environment, then install mlx-vlm, then make sure to downgrade to numpy 1.0 because some of the underlying libraries don't work with numpy 2.0 yet... ... or I can give you a one-liner that does all of that with uv.

python-specific side question -- is there some indication in the python ecosystems that Numpy 2x is not getting adoption? numpy-1.26 looks like 'stable' from here

I have a project on torch 2.6 and numpy 2.2. I never had any issues with that combination.

Re: Qwen2.5-VL-32B: Smarter and Lighter

#293

Post by _xtrimsky »

epolanski wrote: post #117
elorant wrote: post #113

Earlier quoted context omitted.

The average user won't self-host a model.

...yet

I've tried self hosting. It is quite difficult, and either you are limited to low models, either you need a very expensive setup. I couldn't run this model on my gaming computer.

If I try other models, I basically end up with a very bad version of AI. Even if I'm someone who uses Anthropic APIs a lot, it's absolutely not worth it to try and self host it. The APIs are much better and you get much cheaper results.

Self hosting for AI might be useful for 0.001% of people honestly.

Re: Qwen2.5-VL-32B: Smarter and Lighter

#294

Post by Imustaskforhelp »

bee_rider wrote: post #269
Imustaskforhelp wrote: post #220

Earlier quoted context omitted.

My reasoning for proprietary AI to be associated with communism is that they aren't competing in a free market way where everyone does one thing and do its best. They are simultaneously trying to do all things internally. For example , Chatgpt etc. self hosts them on their own gpu and they can generate 10tk/s or something. Now there exists groq , cerebras who can do token generation of 4000 tk/s but they kind of requ…

It seems to me like they are acting like true capitalists; they seem very happy with the idea that capital (rather than labor) gives them the right to profit. But, they don’t seem to be too attached to free-market-ism.

I mean how is a free and open source model not a free market schism atleast in the ai world.

Re: Qwen2.5-VL-32B: Smarter and Lighter

#295

Post by dragonwriter »

Imustaskforhelp wrote: post #220
bee_rider wrote: post #77

Earlier quoted context omitted.

China has allowed quite a bit of market liberalism, so it isn’t that surprising if their AI stuff is responding to the market. But, I don’t really see the connection on the flip side. Why should proprietary AI be associated with communism? If anything I guess a communist handling of AI would also be to share the model.

My reasoning for proprietary AI to be associated with communism is that they aren't competing in a free market way where everyone does one thing and do its best. They are simultaneously trying to do all things internally. For example , Chatgpt etc. self hosts them on their own gpu and they can generate 10tk/s or something. Now there exists groq , cerebras who can do token generation of 4000 tk/s but they kind of requ…

> My reasoning for proprietary AI to be associated with communism is that they aren't competing in a free market way where everyone does one thing and do its best.

That seems based on a very weird idea of what capitalism and communism are; idealized free markets have very little to do with the real-world economic system for which the name “capitalism” was coined, and dis-integration where “everyone does one thing” has little to do with either capitalism or free markets, though it might be a convenient assumption for 101-level discussions of market competition where you want to avoid dealing with real-world issues like partially-overlapping markets and imperfect substitutes to assume every good exists in an isolated market of goods which compete only and exactly with the other groups in that same market in a simple way.

Re: Qwen2.5-VL-32B: Smarter and Lighter

#296

Post by simonw »

mistrial9 wrote: post #279
simonw wrote: post #274

Earlier quoted context omitted.

Right: I could give you a recipe that tells you to first create a Python virtual environment, then install mlx-vlm, then make sure to downgrade to numpy 1.0 because some of the underlying libraries don't work with numpy 2.0 yet... ... or I can give you a one-liner that does all of that with uv.

python-specific side question -- is there some indication in the python ecosystems that Numpy 2x is not getting adoption? numpy-1.26 looks like 'stable' from here

I think it's just that it's a breaking change to a fundamental library, so it will take many months for the ecosystem to upgrade.

Similar thing happened when Pydantic upgraded from 1 to 2.

Re: Qwen2.5-VL-32B: Smarter and Lighter

#297

Post by rfoo »

tw1984 wrote: post #264
rfoo wrote: post #241

Earlier quoted context omitted.

> Right now DeepSeek models hosted in china are having very high latency. If you are talking about DeepSeek's own hosted API service. It's because they deliberately decided to run the service in heavily overloaded conditions and have very aggressive batching policy to extract more out of their (limited) H800s. Yes, for some reason (the reason I heard is "our boss don't want to run such a business" which sounds absurd…

> the reason I heard is "our boss don't want to run such a business" which sounds absurd Liang gave up the No.1 Chinese hedge fund position to create AGI, he has very good chance to short the entire US share market and pocket some stupid amount of $ when R2 is released, he has pretty much unlimited support from local and central Chinese government. Trying to make some pennies from hosting models is not going to susta…

tbh the "short the stock market" story is pretty silly, it wasn't predictable at all. but yeah, the guy got to do whatever he want to do now.

Re: Qwen2.5-VL-32B: Smarter and Lighter

#298

Post by mirekrusin »

tempoponet wrote: post #272
mirekrusin wrote: post #208

Earlier quoted context omitted.

For coding and other integrations people pay per token on api key, not subscription. Claude code costs few $ per task on your code - it gets expensive quite quickly.

But something comparable to a local hosted model in the 32-70b range costs pennies on the dollar compared to Claude, will be 50x faster than your gpu, and with a much larger context window. Local hosting on GPU only really makes sense if you're doing many hours of training/inference daily.

...or working for company which forbids sending IP over wire somewhere.

Also "many hours of inference daily" may mean you're doing your usual stuff daily while running some processing in the background that takes hours/days or you've put together some reactive automation that runs often all the time.

ps. local training rarely makes sense.

ps. 2. not sure where you got 50x slower from; 4090 is actually faster than A100 for example and 5090 is ~75% faster than 4090

Re: Qwen2.5-VL-32B: Smarter and Lighter

#299

Post by cess11 »

refulgentis wrote: post #291
cess11 wrote: post #283

Earlier quoted context omitted.

You mean like https://manusai.ai/ is supposed to function?

Yes, exactly, and no trivially: Manus is Sonnet with tools

Right. Apparently they also claim it's more than that:

https://xcancel.com/peakji/status/1898997311646437487

Re: Qwen2.5-VL-32B: Smarter and Lighter

#300

Post by Zambyte »

bigfudge wrote: post #268
Zambyte wrote: post #129

Earlier quoted context omitted.

There are lots of "alliterated" versions of models too, which is where people will essentially remove the models ability to reject responding to a prompt. The huihui r1 14b alliterated had some trouble telling me about tiananmen square, basically dodging the question by telling me about itself, but after some coaxing I was able to get the info out of it. I say this because I think that the Perplexity model is tuned o…

Abliterated? Alliterated LLMs might be fun though…

Oops, yeah I don't know how that got autocorrected three times without my noticing. Abliterated.
Post reply on HN