Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

791–800 of 848 posts

Re: Qwen 3.8 27B

#791

Post by mixermachine »

scirob wrote: post #769

We maintain German Langauge index as no one publishes or reruns these sepeartly. Qwen 3.8 27B is a small improvement with some regressions in our benchmarks not a huge jump like benchmarks listed. https://dach.peerbench.ai/compare?models=qwen%2Fqwen3.8-27b,... German language has never been a big focus for asian models but they still outperform Gemma models https://dach.peerbench.ai/compare?models=openai%2FQwen%2FQwe…

Did you already test TranslateGemma? I use this model for my Android Studio Translation Plugin (https://plugins.jetbrains.com/plugin/30265-localizepipe) and so far it produces great results for its size.

If there are other models (of similar size) out there, that are better at this, please let me know.

Re: Qwen 3.8 27B

#792

Post by prabhanjana_c »

On my RTX3060 - 12GB VRAM + 24GB RAM , with below command

       ollama run qwen3.8:27b --verbose "explain mmap”, 

 I got  2.41 Tokens/s, Not sure if that can be improved considering VRAM doesn’t fit the entire, model.
Additional details:

total duration: 8m18.2870918s load duration: 612.105ms prompt eval count: 12 token(s) prompt eval duration: 2.900965s prompt eval rate: 4.14 tokens/s eval count: 1193 token(s) eval duration: 8m14.660618s eval rate: 2.41 tokens/s

System spec: NVIDIA GeForce RTX3060 AMD Ryzen 5 1600 Six-Core B450 AORUS M Mother board. NVIDIA-SMI 620.02 Driver:620.02, CUDA Version: 13.2

Re: Qwen 3.8 27B

#793

Post by CMay »

monster_truck wrote: post #774
CMay wrote: post #518

Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it. Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks. The VRAM usage see…

In my experience MTP's speed increase doesn't seem to justify the apparent loss of success at the edge, it would have to be at least 4x faster to meaningfully churn through the first 3 failures in the time it would have taken to do it once without What exactly are you doing that the prompt is eating an entire 65536 window? Surely it would be better to let it use any number of approaches that call tools to access that…

> In my experience MTP's speed increase doesn't seem to justify the apparent loss of success at the edge

If you set manual MTP settings, you'll override dynamic adjustments the inference engine will try to do. Sometimes the dynamic adjustments aren't optimal. With the settings I use, MTP is always a net win.

> What exactly are you doing that the prompt is eating an entire 65536 window?

I'm not using the full context window.

> Surely it would be better to let it use any number of approaches that call tools to access that in parts and reason/summarize to an output file as it works through the whole thing.

Tools would not help.

Re: Qwen 3.8 27B

#794

Post by redox99 »

anon373839 wrote: post #763
redox99 wrote: post #655

Earlier quoted context omitted.

The human genome is 800MB, (and 500MB with compression according to GPT) so that part is easy. I think the problem is that the conversation drifted from "all useful human knowledge" to "enough knowledge to do my job". Both a human and a current large model will bring up unprompted some tax regulation that applies in your district that could matter to your task. A small model won't know the tax code of every city in t…

> A small model won't know the tax code of every city in the world, as it's probably impossible to fit, and it's the kind of thing that unless you already know about, it's very hard to search for What makes it hard to search for? If you tell me you want to open a lemonade stand in East Bumbling Falls, I won't know that city’s municipal ordinances. But I will know that cities tend to have ordinances. So I’d search bro…

Usually at least here it's tens of thousands of laws like "replace article 3 paragraph 2 with:" and it's not like you can ctrl f "lemonade stand" and get something useful.

A strong llm without that knowledge can probably spend millions of tokens parsing every single ordinance and give you an answer if you directly ask about that.

But both a local human or an "all knowing" llm when you just tell it about the stand will be like "hmm wasn't there a law about low calorie beverages being tax exempt? Let me check" and after a very precise search come back with a tip that if you sell sugar free lemonade that's tax exempt.

I mean would you get an accountant from another country just because he can use search?

Re: Qwen 3.8 27B

#795

Post by D4Ha »

bilekas wrote: post #189
D4Ha wrote: post #133

Earlier quoted context omitted.

Do you find it useful or worthwhile to split a large LLM across two GPUs on a desktop? If you've tried it, what worked well and what didn't? I'm especially interested in mismatched VRAM setups, e.g. a 16 GB GPU + a 24 GB GPU. How much overhead did you see from inter-GPU transfers, and did the extra usable VRAM outweigh the performance hit?

I haven't tried this either but I'm guessing if you could pool the GPU memory over whatever the kids are using these days, I think it was SLI back in my day. The GPU memory should still be faster than the RAM?

Unfortunately I checked, SLI doesnt work for this situation. Because the program loading the LLM uses CUDA library, which doesnt account for/takes advantage of SLI for this purpose at least.

Re: Qwen 3.8 27B

#796

Post by airbreather »

dofm wrote: post #417

There's a real change (compared to 3.6) in the way it writes in thinking — it drops words like "to" and "we" in "We need to", talks generally in note form, drops the/and all over the place, avoids "for". "Need be helpful concise", "Need maybe not overdo", "Need ask!" Almost caveman. I have an (unsourced, vague) suspicion that this rather unique thinking trace pattern is actually hobbling the MTP predictions, which se…

Yeah well for whatever reason I often tend to drop a lot of those small words when typing something on keyboard...

Re: Qwen 3.8 27B

#797

Post by cogman10 »

c16 wrote: post #756

Big thank you to the Qwen team. 3.6 A3B was shocking good, and now I'm hoping they release an 3.8 A3B model too. Edit: Having used qwen3.8:27b-mlx on MBP M4 64GB, I get around ~45 tok/s. A3B would be great for smaller devices, but it's definitely usable. As I understand it it's a mixture of MLX and MTP.

Looks like their A3B model is on the way

[1] https://www.reddit.com/r/LocalLLaMA/comments/1voxppd/qwen_38...

Re: Qwen 3.8 27B

#798

Post by latentsea »

simonw wrote: post #419

Absolutely the best pelican I've seen from a model that runs on my laptop: https://tools.simonwillison.net/markdown-svg-renderer#url=ht... Bicycle is the right shape. Pelican beak is excellent. Nice background. Most importantly, the pelican has one leg on each side of the bicycle - that's very rare. (No chain on this bicycle though - in the reasoning trace it says "already chainstay... skip chain detail; maybe a smal…

I asked it to make an animated svg of a pelican doing tricks on a bmx bike and it did. I then asked it to turn that into a browser game. This model fucks.

Re: Qwen 3.8 27B

#800

Post by graceful6800 »

nojs wrote: post #705
jjcm wrote: post #495

Earlier quoted context omitted.

27 t/s. I suspect there will be significant speed ups in the coming weeks.

Any idea why it’s so slow? the entire model should fit in the vram of one card.

The NVFP4 quant is completely broken, so I'm not shocked that other quants aren't fully there yet. Give it some time to cook.
Post reply on HN