Played with this on my Mac a bit today. Not bad!
Qwen 3.8 27B
631–640 of 848 posts
Re: Qwen 3.8 27B
#632Absolutely the best pelican I've seen from a model that runs on my laptop: https://tools.simonwillison.net/markdown-svg-renderer#url=ht... Bicycle is the right shape. Pelican beak is excellent. Nice background. Most importantly, the pelican has one leg on each side of the bicycle - that's very rare. (No chain on this bicycle though - in the reasoning trace it says "already chainstay... skip chain detail; maybe a smal…
Totally agree. I actually didn't know you already got to it and I ran it on mine. It even added a scarf lol https://imgur.com/a/IgYKnmk Took almost 90 minutes on my Threadripper Pro, and feels well worth it. 28k tokens from the Q8 model
And yours got the chain!
Re: Qwen 3.8 27B
#633Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it. Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks. The VRAM usage see…
> Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. I don't expect you to blab publicly about your private benchmark, but what sorts of reasoning does it require?
It writes turing complete Beauty and the Beast fanfic.
Re: Qwen 3.8 27B
#634Absolutely the best pelican I've seen from a model that runs on my laptop: https://tools.simonwillison.net/markdown-svg-renderer#url=ht... Bicycle is the right shape. Pelican beak is excellent. Nice background. Most importantly, the pelican has one leg on each side of the bicycle - that's very rare. (No chain on this bicycle though - in the reasoning trace it says "already chainstay... skip chain detail; maybe a smal…
It's definitely trained on generating SVGs. Not that they trained on the pelican prompt itself, but probably a more general RL step that has it generate SVG and a stronger vision model judges it. It's up to you if that counts as benchmaxxed.
Generation of SVGs is (obviously) a generally useful task.
Re: Qwen 3.8 27B
#635Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it. Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks. The VRAM usage see…
I was quite impressed by Muse Glimmer, and while I am sure people will observe that it is less good on benchmarks, my first experiences with this new 27B have been somewhat exasperating, whereas testing Muse Glimmer was rather fun. I have not tested either in an agentic context, mind you.
Yeah, Glimmer is excellent. You don't really test Glimmer with one-shots, because it's explicitly designed for multi-turn solution finding. The way I see it, if I've got a task that could be done either agentic or requires a lot of context (for example, dumping 600KB of API documentation and another 300KB of codebase for a project) then I would reach for Glimmer easy and it seems like it could get there most of the time.
Qwen might be useful to bring out for a second opinion on some more focused details that are largely information complete. Like, use Glimmer to bring together all the relevant critical data and evaluate what the actual problems are, then maybe prototype a solution. If it's still acting up, maybe throw the resulting context at Qwen and let it meditate on it.
I think there was some study done where ideally you would want to throw a bunch of different models at a problem since they don't all have the same perspective or diagnosis on what the problems or the solutions are.
Re: Qwen 3.8 27B
#636Earlier quoted context omitted.
(mumbles) Shannon entropy... Kolmogorov complexity... something, something... On a more serious note, it depends on your cutoff for "entire human knowledge". It's easy to prove for a generous interpretations of "entire human knowledge" that it can't be done, but hard for something like "all useful human knowledge".
IDK, 30GB is a lot of data when we're talking about text! Moby Dick, uncompressed, is ~1MB. Compressed, it's around 500KB. I feel fairly certain that one could fit all of the textual knowledge required to cultivate a world-class in <60,000 Moby Dicks. (Arguably in <5,000 Moby Dicks with intense effort/pruning).
I think a specialized model could squeeze all you need to know about a certain profession in 30GB. But not all professions at once, which is what these models try to do.
Or maybe not, maybe there's a world model needed for human level at any profession that is very hard to quantify and requires more than 30GB by itself.
Re: Qwen 3.8 27B
#637Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…
To add for me (and claude assisted but tuned myself), here is for 3060 12gb + 5950x 128 gb ram without image support: https://pastebin.com/HsAUBGSY Here is the chat template I used (and renamed to qwen.jinja): https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/r... For image support, drop to NGL 24 and add: --mmproj-url https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/main/mmproj-F16.gguf \ --no-mmproj…
How many tokens/second you getting? I have the same CPU but a 5070 and only 64 GB of ram. I just got llama.cpp built and am now hitting a whopping 5-6 tokens/s.
Re: Qwen 3.8 27B
#638Earlier quoted context omitted.
Reduce or turn off thinking: https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates
In my experience with Qwen3.6 35B-A3B, disabling thinking made the model generate inaccurate replies. Ask it for the recipe of egg salad and it gives you the recipe of an omelette. Did I miss something, is it possible to have that model be reliable without thinking?
Turning off thinking mode you need a different tokenizer and a whole host of settings. It's more like switching your vehicle from gas to diesel, than turning your headlights from lo to hi.
Re: Qwen 3.8 27B
#639Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it. Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks. The VRAM usage see…
> The VRAM usage seems way less efficient than Gemma 4 or Glimmer though Maybe it's implicit that you're using llama.cpp (although you don't mention GGUF), but it's hard to reach concrete conclusions about the model architecture based on one implementation in one runtime.
Aren't things like KV size inherent to the model?
Re: Qwen 3.8 27B
#640Wow. This model is so good, and we have GLM 5.3 (seems great voor security related work) and Deepseek. In a few months we'll have Fable/Sol-like capabilities that are not coming from the big US companies. I feel as a programmer that that is more than enough. How wil OpenAI and Anthropic survive when frontier model intelligence becomes commoditized?
The demand curve for speed and intelligence seems pretty steep to me. If you look at the hiring marketplace, being just marginally better than your peers can be very lucrative. If you’re competing on speed or capability as a company (or as an employee), you’re probably going to be willing to pay for the frontier.
Most companies have a limited budget.
Good enough with a lower price will win the masses