461–470 of 848 posts
Absolutely the best pelican I've seen from a model that runs on my laptop: https://tools.simonwillison.net/markdown-svg-renderer#url=ht... Bicycle is the right shape. Pelican beak is excellent. Nice background. Most importantly, the pelican has one leg on each side of the bicycle - that's very rare. (No chain on this bicycle though - in the reasoning trace it says "already chainstay... skip chain detail; maybe a smal…
I'm struggling to figure out what to use this for. From the intelligence benchmarks in OMLX. If only they would release another MoE model. Intelligence Benchmark Comparison --- Detail --- Model: scottlowry--Qwen3.8-27B-oQ4e-mtp Benchmark Accuracy Correct Total Time(s) Think -------------------------------------------------------------- GSM8K 93.3% 28 30 282 No MATHQA 46.7% 14 30 26.3 No HUMANEVAL 96.7% 29 30 156.5 No…
Try manually asking both more discrete esoteric knowledge questions. Or use benchmarks which are less coding focused. The 3.6-35B-A3B with post-training may do well in coding type benchmarks and math but the density of its knowledge falls off in my experience (vs 3.6 27B dense Q8-K-XL unsloth GGUF) when you need to use it for less commonly used domains of knowledge.
Earlier quoted context omitted.
I'm generally extremely skeptical about a lot of the model hype that show up in comments. Except when there is an extreme mismatch the performance, quirks and quality of these things are difficult to nail down. You wouldn't know that from the comment section of every single release . I think some of these are excited, eager users always ready to hype up the new thing. The same crowd that previously would constantly p…
> I think most of it is bot driven spam by the various labs. It's hard not to notice 3 month old accounts with very strong opinions about various frontier labs and little else. I've assumed the same as well. I also assume that many of the companies developing these models engage in benchmaxxing. At my company we've developed our own internal benchmarks for evaluating LLM models as they become available. The benchmark…
I've got an RX 7900 XT (20GB of VRAM) and I can run glimmer with a full 128k context window with the draft model at 65-80 tok/s.
This model, on the other hand, I get about 30 tok/s with a 30k context. Raising the context or loading the draft layers for MTP drops performance to 9-15 tok/s.
So I wonder how big the "real world" delta between Glimmer and Qwen is here. I can already run 3-bit DSv4-flash at 9-15 tok/s with 100k~ context, and I suspect it would outperform 4-bit Qwen 3.8 27B here.
I'll have to experiment and see if I just made a stupid mistake somewhere, but it looks like Glimmer might make more sense for the comically specific niche of "20GB VRAM".
I did notice if you go beyond Medium he starts overthinking like hell as per usual for a Qwen model.
that's using llama.cpp
llama-server \ -m ~/somePath/Qwen3.8-27B-UD-Q8_K_XL.gguf \ -np 1 --kv-unified \ -fa on --no-cache-idle-slots --reasoning-preserve \ --temp 0.2 \ --spec-type draft-mtp,ngram-mod --spec-draft-n-max 3 --spec-draft-n-min 1 \ --cache-type-k f16 --cache-type-v f16 \ --chat-template-kwargs '{"preserve_thinking": true}' \
I tried playing with all the recommended parameters from the unsloth page with no luck...
in one of the high fever ramblings it ended with amen... lol
Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…
There's a real change (compared to 3.6) in the way it writes in thinking — it drops words like "to" and "we" in "We need to", talks generally in note form, drops the/and all over the place, avoids "for". "Need be helpful concise", "Need maybe not overdo", "Need ask!" Almost caveman. I have an (unsourced, vague) suspicion that this rather unique thinking trace pattern is actually hobbling the MTP predictions, which se…
Oh dear. I need to try to understand what is going on here.
Earlier quoted context omitted.
Try manually asking both more discrete esoteric knowledge questions. Or use benchmarks which are less coding focused. The 3.6-35B-A3B with post-training may do well in coding type benchmarks and math but the density of its knowledge falls off in my experience (vs 3.6 27B dense Q8-K-XL unsloth GGUF) when you need to use it for less commonly used domains of knowledge.
My use cases try to avoid accessing world knowledge in the model (I give it access to web search for some adhoc RAG), and ya, I'm just focused on coding so that's the only place I'm looking at right now.