Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

231–240 of 848 posts

Re: Qwen 3.8 27B

#231

Post by kccqzy »

bmitc wrote: post #138
hypfer wrote: post #109

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…

Lol at that command. Why is this stuff so hard to run locally? I've spent a few days trying to figure it all out and haven't been able to. LM Studio doesn't work behind proxies. Ollama is confusing and doesn't seem to support Qwen3? And Llama.cpp is your command. I just want to run ` ` with some default parameters set and for it to run locally.

There are easier ways to run it. OP seemed to enjoy tinkering and customizing the command to run it exactly the way they want. When I don’t want to tinker Unsloth Studio is probably closest to pick a model and voila.

Re: Qwen 3.8 27B

#232

Post by Casteil »

satvikpendem wrote: post #179
Casteil wrote: post #82

One thing a lot of people don't seem to factor when hyping Qwen is how much models like this tend to 'overthink' with seemingly endless 'second guessing'. 3.8 seems no different from what I've tried thus far. As capable as it is, it's hard to justify using it when a competing model (e.g. Gemma4:26b-a3b) can consistently achieve the same or similar response with only 1/10th as many 'thinking' tokens, achieve much high…

Reduce or turn off thinking: https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

Given that it apparently defaults to 'xhigh', this is probably the answer.

Granted, it's still much lower tokens/s than you'll get out of many MoE models.

Edit: Even set to medium or low there's still a lot of second guessing, less consistency, lower 'acceptable response' rate, and slower/more token churn vs gemma4:26b-a3b. I think gemma4 is just a better 'general purpose' model.

Re: Qwen 3.8 27B

#234

Post by dannyw »

Casteil wrote: post #219
satvikpendem wrote: post #158

Earlier quoted context omitted.

Reduce or turn down thinking: https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

Yeah, that's probably the answer given that it apparently defaults to 'xhigh'.

Probably helps it score a little bit better in benchmarks :) `medium` seems like a nice balance so far; along with some light steering to vary think effort as needed for task and being pragmatic.

Re: Qwen 3.8 27B

#235

Post by altruios »

satvikpendem wrote: post #177
ramon156 wrote: post #64

People will claim it's not comparable to Opus despite it beating the score. I'm not sure I disagree, but I'm also unsure whether I care. Most new models nowadays are "good enough". I cannot complain because I'd rather spend that time improving my prompts and docs. Opus might be a _slight bit better_ at picking up vague hints, but it's also extremely expensive, and I hit the 5 hour limit way too quick. I care a lot ab…

You should check out Grok, it's quite a good deal from the Cursor subscription side but it's cheap even by API prices.

[flagged]

Re: Qwen 3.8 27B

#236

Post by mixologic »

tinfoilhatter wrote: post #196
nubg wrote: post #165

Earlier quoted context omitted.

just to clarify. yes YOU built it. just because you used some tool doesn't mean the idea, prompting, reprompting, babysitting was not your creative input and effort. put differently, if you put a random person infront of whatever model you used (say, a 50yo receptionist at a pharmacy in india), they would not have been able to create that, because they would have lacked the motivation, idea, background knowledge, tas…

So if I hire an artist and am a motivated individual, have an idea for a painting, have background knowledge about paintings and have taste in paintings and can offer a critique of the painting as the artist paints it, then somehow I created the painting? Absurd logic. The AI built the website.

Nothing absurd about that. What do you think an "Executive producer" is? A "Director" ? Does Peter Jackson get credit for creating the Lord of the Rings Trilogy films? Christopher Nolan for his films? But did he make them ? No, it was the collective effort of thousands of individuals all working under their direction.

Just like if somebody creates software today, and the end result is generated by the collective effort of thousands of agents, the "Director" still gets credit.

Re: Qwen 3.8 27B

#237

Post by redox99 »

ycui7 wrote: post #199
redox99 wrote: post #143

Earlier quoted context omitted.

Yep. These small models are actually worse than GPT 3.5 at some tasks (like recalling facts). You can definitely make models smarter at specific tasks (like tool calling, coding) but you can't compress the entire human knowledge into a 30GB file. It's just not enough bits.

that is why we enable web search for the agent. the memory can come from the internet. deepseek-v4-flash needs web search to return true facts.

Yeah but results are worse, see https://news.ycombinator.com/item?id=49301574

Re: Qwen 3.8 27B

#238

Post by nubg »

tinfoilhatter wrote: post #196
nubg wrote: post #165

Earlier quoted context omitted.

just to clarify. yes YOU built it. just because you used some tool doesn't mean the idea, prompting, reprompting, babysitting was not your creative input and effort. put differently, if you put a random person infront of whatever model you used (say, a 50yo receptionist at a pharmacy in india), they would not have been able to create that, because they would have lacked the motivation, idea, background knowledge, tas…

So if I hire an artist and am a motivated individual, have an idea for a painting, have background knowledge about paintings and have taste in paintings and can offer a critique of the painting as the artist paints it, then somehow I created the painting? Absurd logic. The AI built the website.

no, because there is another human involved.

llms are not human.

Re: Qwen 3.8 27B

#239

Post by dannyw »

jrflo wrote: post #148
scrlk wrote: post #8

Beats Opus 4.7 Max (w/ Claude Code) on DeepSWE (42.2 vs 40). Looks like Qwen's 27B models continue to pack some punch. Unsloth's GGUF quants are up: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

That kind of result makes me suspicious of benchmaxxing. Qwen 27B is 100x smaller than Opus 4.7. Is it really 100x more parameter-efficient? Two orders of magnitude is hard to believe. I don't have the hardware to run a 27B, but I'm curious what real world use is like. Maybe I'll have to buy some usage on a cloud provider to run my own tests, but this seems fishy to me.

It's very agentic coding focused; and I'd say a good executor but certainly not Opus in scale; overall knowledge; long-horizon work and recovery; etc.

e.g. If you try to chat to it about something philosophical for example, or maybe a debate / creative writing, then you'll very quickly see how it is still a much smaller model at the end of the day.

Still, it's such a relatively accessible model to run, and I find a big part of leveraging smaller models is to give it well-scoped tasks; not too high level or ambitious ones. Very impressive for its size and the ability to run locally :)

Re: Qwen 3.8 27B

#240

Post by hypfer »

manquer wrote: post #210
hypfer wrote: post #203

Earlier quoted context omitted.

> Why would they wish to handcraft this ? Because this is kinda the one new thing that arrived in the technology scene, so getting at least some amount of understanding of its "inner" workings might prove useful in the future. Beside that, it is also just.. interesting? It's fun tuning the machine to see it improve. For some, anyway.

The people OP mentioned about "just want AI" . The pain point they raised is this is too complicated for people who just want to get started, that is not true anymore. It is certainly fun to fine-tune and setup if you like do something like that, however the need to do it hardly is a barrier for those who don't want complexity as OP imagines. Lower level API/interfaces should not be a barrier for people if they are a…

> More and more people are thinking agent native so this is not really a issue.

Why this headache inducing lingo tho? What does that even mean, and why should I sign up for your webinar about that?

Post reply on HN