no FIM though :(, imo most slept on usecase for local models
[deleted]
Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
451–460 of 482 posts
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#452Earlier quoted context omitted.
I just create the nopelican user to avoid seeing the same type of comments for scoring new models. Why doesn't someone create a pelican by month thread, like who is hiring, so that all who want to talk about their prefered mode and pelican can post with leisure at full extend. Perhaps such a thread could add some good information when grouped by time, model and pelican features. But I, honestly, think that the pelica…
The point of a benchmark is that it allows a relative comparison. The Pelican is one such benchmark. Feel free to create a "how does it compare to Claude 3.5 Sonnet" benchmark. If people find it useful, it will be run against new LLMs to generate additional points of comparison. I will also say; it's really easy to just skim past comments. I suspect your ROI time-wise in creating this account to complain will never b…
Usually I read the top comments in posts, they usually have the best information. I don't think the pelican test deserve to be at top position. HN top posts should reflect the best of our community, not by karma but by the value and insight that they provide.
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#453Earlier quoted context omitted.
For coding often quality at the margin is crucial even at a premium. It’s not the same as cranking out spam emails or HN posts at scale. This is why the marginal difference between your median engineer and your P99 engineer is comp is substantial, while the marginal comp difference between your median pick and packer vs your P99 pick and packer isn’t. I’d also say it keeps the frontier shops competitive while costing…
Given the very limited experience I have where I've been trying out a few different models, the quality of the context I can build seems to be much more of an issue than the model itself. If I build a super high quality context for something I'm really good at, I can get great results. If I'm trying to learn something new and have it help me, it's very hit and miss. I can see where the frontier models would be useful…
Apologies for my ignorance but how can you know the quality of the context?
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#454Earlier quoted context omitted.
I get ~5 tokens/s on an M4 with 32G of RAM, using: llama-server \ -hf unsloth/Qwen3.6-27B-GGUF:Q4_K_M \ --no-mmproj \ --fit on \ -np 1 \ -c 65536 \ --cache-ram 4096 -ctxcp 2 \ --jinja \ --temp 0.6 \ --top-p 0.95 \ --top-k 20 \ --min-p 0.0 \ --presence-penalty 0.0 \ --repeat-penalty 1.0 \ --reasoning on \ --chat-template-kwargs '{"preserve_thinking": true}' 35B-A3B model is at ~25 t/s. For comparison, on an A100 (~RTX…
Why use --fit on on an M4? My understanding was that given the unified memory, you should push all layers to the GPU with --n-gpu-layers all. Setting --flash-attn on and --no-mmap may also get you better results.
Meaningless question, fit will put everything on the gpu if it fits. Fa is default on. No-mmap is not an inference tradeoff and if you do turn it off you need to turn on direct io via -dio
What he should actually do is enable speculative decoding
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#455Earlier quoted context omitted.
I can run Qwen3.5-27B-Q4_K_M on my weird PC with 32 GB of system memory and 6 GB of VRAM. It's just a bit slow, is all. I get around 1.7 tokens per second. IMO, everyone in this space is too impatient. (Intel Core i7 4790K @ 4 Ghz, nVidia GTX Titan Black, 32 GB 2400 MHz DDR3 memory) Edit: Just tested the new Qwen3.6-27B-Q5_K_M. Got 1.4 tokens per second on "Create an SVG of a pellican riding a bicycle." https://gist.…
Don't forget that you're also spending much more electricity because it takes so long to run inference.
Given current hardware prices I wouldn't expect this to tip the scales.
Mind you, for me local models are a fun experiment, for anything serious I would use a frontier model. So I'm not using this for hours a day.
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#456Earlier quoted context omitted.
[flagged]
I've been using it in a few harnesses (FP8 quant, max context length) and it does seem to get tripped up by tool use, often repeating the same tool when it failed previously - that's usually not a great sign for long-term context and multi-step reasoning. It is excellent at one-shotting though and might be most useful as a sub-agent for a stronger frontier coordinator.
yeah that tracks, tool repetition on failure is a classic sign the model isn't really reading its own context. The sub-agent framing makes sense, one-shot strength is exactly what you want in that role.
(Also somehow got flagged for my original comment, which, classic HN lol)
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#457A bit skeptical about a 27B model comparable to opus...
You should try it out. I'm incredibly impressed with Qwen 3.5 27B for systems programming work. I use Opus and Sonnet at work and Qwen 3.x at home for fun and barely notice a difference given that systems programming work needs careful guidance for any model currently. I don't try to one shot landing pages or whatever.
Is it available for API use? I don't have a laptop capable of running it.
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#458Earlier quoted context omitted.
You are right, which is why I do not intend to use a GGUF file but a set of files with a different layout, and this is why I need to make changes in llama.cpp.
If you have to come up with a custom format anyway, why not just make it a draft extension to GGUF layout definitions (something like "coalesced expert fetch" or the like) and submit it for inclusion in the standard? Then future models could be autoconverted to such a format.
This is a good suggestion.
I will consider to do this after I gather enough experience to determine which is the best layout and when I will have enough benchmark data to support that.
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#459Earlier quoted context omitted.
I find it hard to believe anyone who has ever done business inside China doesn’t know that the structure of Chinese business is built around massive IP theft and repurposing on a state wide systematic level. It’s not a nationalism point, it’s an objective and easily verified truth. Most code is not P99, but companies pay a premium to produce code that is. That’s my point.
Chinese companies are built on IP theft, and Anthropic/Open AI are not? And in world where code generation costs are trending to zero, goodluck commanding a premium to produce any kind of code. There is a whole bunch of P99 code that is open-source. What makes code P99 is not the model that produces it, but the people who verify/validate/direct it.
Didn’t the major American labs pirate a whole bunch of their training data?
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#460Earlier quoted context omitted.
In every one of these threads for a new Chinese open weights model, it's always the same tired discussion of how this is all actually a psyop by the Chinese government to undermine US interests and it can't answer questions about Tienanmen Square. Meanwhile I'm over here solving real world business problems with a model that I can securely run on-prem and not pay out the nose for cloud GPU inference. And then after w…
> The last thread for Qwen's new hosted model was full of folks talking about how "China" is no longer releasing open weights models, when the next day Moonshot AI releases Kimi 2.6. A few days later and here's Qwen again with another open release. looks like you declared win argument, because you now see that 2.6 was released, but at that time your opponents argument stand. Also, you can't predict if Chinese labs wi…
> looks like you declared win argument, because you now see that 2.6 was released, but at that time your opponents argument stand.
Their argument was based entirely on speculation, but stated as a matter of fact, despite Alibaba making very clear statements that they were going to continue releasing open models.
And the core of my argument is that they were conflating a single company with the motivations of multiple companies in a country. Nobody talks about US companies by saying "The Americans are going to do X", they say "OpenAI/Anthropic/Google is going to do X".