Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3-4B-Thinking-2507

huggingface.co

Post reply on HN
Print view View original

61–64 of 64 posts

Re: Qwen3-4B-Thinking-2507

#61

Post by wkat4242 »

Imustaskforhelp wrote: post #30
thatwasunusual wrote: post #16

Earlier quoted context omitted.

Let's say any country create the most powerful - and thus best - LLMs. They over time infiltrate it with their political will. Over 20-30 years, I'd imagine people asking those LLMs will have their minds' shifted. But. That's just me, my pessimism-sci-fi scenario.

I think that just as how perplexity had actually created a deepseek(fine-tune?)[1], then there is more and more incentive towards making uncensored models though I am gonna be honest, Kimi K2 isn't that censored but I tried the gguf variant of this model on local pc and it definitely is censored / biased towards china (like taiwan is part of them and so on) But still, the most recent version of american foss model gp…

The problem I've had with Chinese models is that when they get confused they revert to Chinese which is just gibberish to me of course. American models revert to English in those situations but that's fine for me.

And models can be abliterated to remove the censorship. I use llama3 that way.

Re: Qwen3-4B-Thinking-2507

#62

Post by wkat4242 »

cowpig wrote: post #7
esafak wrote: post #6

Earlier quoted context omitted.

https://artificialanalysis.ai/leaderboards/models?open_weigh...

Compare these rankings to actual usage: https://openrouter.ai/rankings Claude is not cheap, why is it far and away the most popular if it's not top 10 in performance? Qwen3 235b ranks highest on these benchmarks among open models, but I have never met someone who prefers its output over Deepseek R1. It's extremely wordy and often gets caught in thought loops. My interpretation is that the models at the top of Artific…

> But Grok is not very good compared to the anthropic, openai, or google models despite ranking so highly in benchmarks.

That's political I think. I know several alt right types that swear by grok because "Elon doesn't give it any of that woke crap". They don't care that there's better, for them it's the only viable option.

Re: Qwen3-4B-Thinking-2507

#63

Post by iamnotagenius »

hnuser123456 wrote: post #53
magnat wrote: post #45

Earlier quoted context omitted.

> if you run it at the full 262144 tokens of context youll need ~65gb of ram What is the relationship between context size and RAM required? Isn't the size of RAM related only to number of parameters and quantization?

A 24GB GPU can run a ~30b parameter model at 4bit quantization at about 8k-12k context length before every GB of VRAM is occupied.

Not quite true. Depends on number of KV heads. GLM4 32b at IQ4 quant and Q8 context can run full context with only 20GiB VRAM.

Re: Qwen3-4B-Thinking-2507

#64

Post by aitchnyu »

magnat wrote: post #45
nisten wrote: post #36

If you want to have an opinion on it, just install lmstudio and run the q8_0 version of it i.e. here https://huggingface.co/bartowski/Qwen_Qwen3-4B-Instruct-2507... . you can even run it on a 4gb raspberry pi Qwen_Qwen3-4B-Instruct-2507-Q4_K_L.gguf https://lmstudio.ai/ Keep in mind if you run it at the full 262144 tokens of context youll need ~65gb of ram. Anyway if you're on mac you can search for "qwen3 4b 2507 mlx…

> if you run it at the full 262144 tokens of context youll need ~65gb of ram What is the relationship between context size and RAM required? Isn't the size of RAM related only to number of parameters and quantization?

Whats the space complexity for context size? And who is trying to drop it into linear complexity?
Post reply on HN