Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

721–730 of 848 posts

Re: Qwen 3.8 27B

#721

Post by macwhisperer »

yassa9 wrote: post #50

Can anyone who has that specific personal test he tries on different models , and tries this model , to tell us here if possible , how good or bad is this new model ? compared to others ? I only trust those users genuine personal tests

yeah this model is chefs kiss..

running an untouched, vanilla 4-bit version (Q4_0) I baked myself today (benched it against Q4_K_M (16gb) and IQ3_M (12gb), Q4_0 (15gb) is king)...

this model--

1: over 60% faster than qwen 3.6 version of the same dense 27b model, same engine setup (don't ask how, im not sure either)

2: has better reasoning quality, less "loopy" with its thinking patterns.. most certainly the smartest model on my roster currently

3: has the longest task horizon ive ever experienced (locally or otherwise)...I sent it a bunch of compressed ideas for an app, it sent me back the largest python app ive ever seen in one single ai response pass (80kb text file)

thanks qwen!! hoping to see the full model range get released...

Re: Qwen 3.8 27B

#722

Post by ixaxaar »

pulse7 wrote: post #682
kimsey0 wrote: post #571

If anyone else is running this on an RTX 5090, https://github.com/Neroued/ninfer as inference engine gets me ~138 tokens/second, roughly double what I get with a naive llama.cpp setup.

It there anything similar for RTX 3090 and RTX 4090?

I'm running it using a 4090 on using llama.cpp with Q5_K_S and its running at ~33 t/s

Re: Qwen 3.8 27B

#724

Post by Yajirobe »

cjbprime wrote: post #714
dexterlagan wrote: post #704

Tested the model briefly with my usual eval: a couple of questions on general knowledge most small models often get wrong, then write a fully-featured todo list app in JS, then rewrite the same app in Rust with Tauri. Granted, most models are well trained on basic todo apps, but it gives me an idea of the basic SWE capabilities I can build on. As far as I'm concerned, if it can successfully setup a local git repo, wr…

> General knowledge: I usually ask 2 questions many small models get wrong: summarize Operation Trojan Horse by John Keel, give publication year. Summarize the Ariel school incident of 1994. Qwen3.8 got the first question right, along correct publication year, gave glorious detail on Keel's theory, but got the second wrong. It thought that school was located in the USA. Ah well. I'm not sure that this means anything.…

Did the model refuse to answer? Did it say that it doesn't know? If not, then it's a fair game in my opinion.

Re: Qwen 3.8 27B

#725

Post by dexterlagan »

cjbprime wrote: post #714
dexterlagan wrote: post #704

Tested the model briefly with my usual eval: a couple of questions on general knowledge most small models often get wrong, then write a fully-featured todo list app in JS, then rewrite the same app in Rust with Tauri. Granted, most models are well trained on basic todo apps, but it gives me an idea of the basic SWE capabilities I can build on. As far as I'm concerned, if it can successfully setup a local git repo, wr…

> General knowledge: I usually ask 2 questions many small models get wrong: summarize Operation Trojan Horse by John Keel, give publication year. Summarize the Ariel school incident of 1994. Qwen3.8 got the first question right, along correct publication year, gave glorious detail on Keel's theory, but got the second wrong. It thought that school was located in the USA. Ah well. I'm not sure that this means anything.…

Agreed. It's just to see how much knowledge the model still has for its size. For comparison, no other model of this size has ever given me the correct answer to the Keel question. Even GPT-OSS-120B got it wrong, but it's expected for its relative old age.

Re: Qwen 3.8 27B

#726

Post by irthomasthomas »

lacoolj wrote: post #512
simonw wrote: post #421

Earlier quoted context omitted.

For anyone who followed yesterday's Gemini 3.7 Flash pelican which rendered in Safari but not in Firefox or Chrome... https://news.ycombinator.com/item?id=49289112#49290012 . - that turned out to be my fault, not the model. My SVG rendering software was stripping some attributes. Here's the Gemini 3.7 Flash pelicans in the fixed renderer: https://tools.simonwillison.net/markdown-svg-renderer.html#u...

Holy crap that is above and beyond But, not animated like the Qwen 2.4T one you showed, so minus points for that

Animation was not requested.

Re: Qwen 3.8 27B

#727

Post by dexterlagan »

ra wrote: post #717
cjbprime wrote: post #714

Earlier quoted context omitted.

> General knowledge: I usually ask 2 questions many small models get wrong: summarize Operation Trojan Horse by John Keel, give publication year. Summarize the Ariel school incident of 1994. Qwen3.8 got the first question right, along correct publication year, gave glorious detail on Keel's theory, but got the second wrong. It thought that school was located in the USA. Ah well. I'm not sure that this means anything.…

I concur, treating models as question and answer machines and judging them on recall is meaningless, unless you're measuring quantisation impact on a foundation model maybe.

100% agree, it's really just for kicks&giggles. The fact that the model answered correctly, unlike every other model of its size before it, still pleasantly surprised me.

Re: Qwen 3.8 27B

#728

Post by dexterlagan »

terhechte wrote: post #708
dexterlagan wrote: post #704

Tested the model briefly with my usual eval: a couple of questions on general knowledge most small models often get wrong, then write a fully-featured todo list app in JS, then rewrite the same app in Rust with Tauri. Granted, most models are well trained on basic todo apps, but it gives me an idea of the basic SWE capabilities I can build on. As far as I'm concerned, if it can successfully setup a local git repo, wr…

Which setup did you use? MLX/GGUF, Quant, Engine (e.g. llama.cpp or MTPLX, etc)? There’s so much variety these days.

It was in LMStudio (llama.cpp), Q4 by Unsloth. Applied the recommended defaults published by Unsloth.

Re: Qwen 3.8 27B

#729

Post by qeternity »

WhyNotHugo wrote: post #718
dofm wrote: post #417

There's a real change (compared to 3.6) in the way it writes in thinking — it drops words like "to" and "we" in "We need to", talks generally in note form, drops the/and all over the place, avoids "for". "Need be helpful concise", "Need maybe not overdo", "Need ask!" Almost caveman. I have an (unsourced, vague) suspicion that this rather unique thinking trace pattern is actually hobbling the MTP predictions, which se…

This is pretty much Chinese grammar with English words. In Chinese you'd say "need ask" or "I need ask" if you want to be unusually verbose, but there's no "to". All your examples align with this too. It just sounds to me like its optimising by removing the superfluous English words and thinking more Chinesely.

Has nothing to do with Chinese.

Frontier labs have already been doing this for a while, verified in smuggled traces from OAT/Ant.

Simply a way to reduce tokens.

Re: Qwen 3.8 27B

#730

Post by dexterlagan »

dexterlagan wrote: post #704

Tested the model briefly with my usual eval: a couple of questions on general knowledge most small models often get wrong, then write a fully-featured todo list app in JS, then rewrite the same app in Rust with Tauri. Granted, most models are well trained on basic todo apps, but it gives me an idea of the basic SWE capabilities I can build on. As far as I'm concerned, if it can successfully setup a local git repo, wr…

Meaningless yet fun fact: DeepSeek V4 Pro 0813 made a much worse icon for the same app, and only produced an SVG I had to convert manually to .png. Qwen3.8 made a perfect icon in .png. I don't yet know how it did it, but it did it.

Qwen3.8 also seems to know French quite a bit better than Copilot, at least on common expressions. I have yet to run more tests for languages, but I'm baffled by its finer accuracy on the few sentences I asked it to translate, which were botched by Copilot. Go figure.

Post reply on HN