Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

841–848 of 848 posts

Re: Qwen 3.8 27B

#841

Post by lwansbrough »

oarsinsync wrote: post #761
lwansbrough wrote: post #515

Earlier quoted context omitted.

Perhaps this isn’t a new observation but the problem with LLMs is very clear with these. It’s a nice microcosm. The LLM will draw a fish companion (unprompted!) with a nice gradient but won’t get the pelican’s feet right. It’s obviously a problem of fundamental understanding and demonstrates that reasoning is more “directionless rigour”.

Have you seen what happens when you ask 500 humans to draw a bicycle? No pelican, nobody riding it, just the bike. https://foerstel.com/memory-bikes/

I suspect if you showed one human a million photos of a bike they might get it right.

Re: Qwen 3.8 27B

#842

Post by hadlock »

eblanshey wrote: post #564
hadlock wrote: post #406

Earlier quoted context omitted.

For high concurrency, using the blackwell's native native W4A4 MLP compute path, nvfp4 is something like a 1.2-1.5x performance increase over FP8. We're doing data enrichment (so, tasks completed successfully + tokens/second) so the performance bump shows up in the tasks/month number. I am just now getting the benchmarks running against 3.8 27b but I expect similar results from benching 3.6 27b at the same quant.

I see. Did you see any intelligence degradation between FP8 and NVFP4 for 3.6 27B? You're using vLLM, right?

I didn't have time to run this as well, but we're getting a 99% agent completion rate across all tasks and 98% task decision that matches the human selected option(s) on Qwen 3.8 27B @ NVFP4. There may be a difference between FP8 and NVFP4 but it's inconsequential for our data enrichment purposes.

Re: Qwen 3.8 27B

#843

Post by anotherCodder »

anotherCodder wrote: post #817

vllm on 4x 5090 is getting ~20 tok/s with mtp on (their own thread on the hf card). i had qwen3.8-27b up the day after release, one rtx pro 6000, 140 tok/s spec, 0.156s first token, full 262k. image and video on the same api. numbers: https://github.com/avifenesh/memra try it: https://inference.tiyuvta.ai/app $0.38 in / $0.20 cache / $2.60 out. openrouter's only host right now is 23 tok/s at $0.45 / $3.20.

price update: $0.40 in / $0.10 cache / $2.90 out

Re: Qwen 3.8 27B

#844

Post by focxle »

ramon156 wrote: post #64

People will claim it's not comparable to Opus despite it beating the score. I'm not sure I disagree, but I'm also unsure whether I care. Most new models nowadays are "good enough". I cannot complain because I'd rather spend that time improving my prompts and docs. Opus might be a _slight bit better_ at picking up vague hints, but it's also extremely expensive, and I hit the 5 hour limit way too quick. I care a lot ab…

[flagged]

Re: Qwen 3.8 27B

#845

Post by ChucklsTheBeard »

Yajirobe wrote: post #724
cjbprime wrote: post #714

Earlier quoted context omitted.

> General knowledge: I usually ask 2 questions many small models get wrong: summarize Operation Trojan Horse by John Keel, give publication year. Summarize the Ariel school incident of 1994. Qwen3.8 got the first question right, along correct publication year, gave glorious detail on Keel's theory, but got the second wrong. It thought that school was located in the USA. Ah well. I'm not sure that this means anything.…

Did the model refuse to answer? Did it say that it doesn't know? If not, then it's a fair game in my opinion.

Ideally it would run a web search to check questionable knowledge. Google-fu has been one of the most important SWE skills for a long time!

Re: Qwen 3.8 27B

#846

Post by verdverm »

alfiedotwtf wrote: post #407
skrebbel wrote: post #178

Earlier quoted context omitted.

I'd love to understand this more. Are you saying the Qwen team spends their very impressive human and compute resources on publishing these amazing models and then botches the chat template with mundane bugs? Like maybe I just misunderstand what's the hard part but wouldn't you assume that people who can put together an impressive model can also write a proper jinja chat template for it?

The chat templates are usually the first thing that every major release bork on, and all new model architectures end up having a ~2 week initial window of small fixes before they’re not DoA

Laguna wwa standout in that they borked the quants released with the main model and had to update the next day

Re: Qwen 3.8 27B

#847

Post by Zambyte »

jychang wrote: post #833
Zambyte wrote: post #830

Earlier quoted context omitted.

Honestly it seems like a job for the harness, rather than the model. Sample the model with the same question, perhaps with varying temperature (?), and use that to establish a degree of confidence in the answer. If the model provides very different answers every time, respond that it doesn't know. If it responds with the same answer usually but a different answer sometimes, respond with moderate confidence. If the mo…

that could work sometimes but that's terribly hacky engineering

I... really don't agree. Being confident in using documentation to program in a variety of environmnets is not "terribly hacky engineering". Good engineering involves leveraging documentation well.

Re: Qwen 3.8 27B

#848

Post by kpatucha »

CMay wrote: post #699
pulse7 wrote: post #680

Earlier quoted context omitted.

Can you please tell which Gemma 4 variant managed to correctly reason through your private benchmarks? Was is Gemma 4 31B? What quantizations and context lengths did you use for Gemma 4 and Qwen 3.8 27B? I am asking because I can't even load Gemma 4 31B on my GPU with any reasonable quantization (even with small context), while I can run Qwen 3.8 27B with large context and good quantization...

Gemma 4 12B, Gemma 4 12B QAT, Gemma 4 31B, Gemma 4 31B QAT Gemma 4 26BA4B would get close, but not quite and sometimes even get stuck in loops despite a repeat penalty. Do not use any newer updated templates or Unsloth fixes. Use older official templates that released with the models on the huggingface repo. The template here worked: https://huggingface.co/google/gemma-4-12B-it/tree/657684fef0... llama-server --model…

Gemma 4 12B? This sounds really interesting with Q_4 (preferably QAT) this fits comfortably in 12 or 16 GB VRAM.

Could you elaborate on Gemma 4 12B capabilities from your experience and benchmarks?

Post reply on HN