Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

qwen.ai

Post reply on HN
Print view View original

471–480 of 482 posts

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#471

Post by danielhanchen »

hypercube33 wrote: post #168
danielhanchen wrote: post #73

Earlier quoted context omitted.

We made Unsloth Studio which should help :) 1. Auto best official parameters set for all models 2. Auto determines the largest quant that can fit on your PC / Mac etc 3. Auto determines max context length 4. Auto heals tool calls, provides python & bash + web search :)

Sadly doesn't support fine tuning on AMD yet which gave me a sad since I wanted to cut one of these down to be specific domain experts. Also running the studio is a bit of a nightmare when it calls diskpart during its install (why?)

Apologies as well didn't reply sooner - Studio supports AMD out of the box now! We worked with AMD to make it work! One thing that is still missing is pre-compiled AMD ROCM binaries, which we're trying to see if we can integrate that.

Interesting on diskpart - let me check and get back to you [EDIT] - visual studio build tools, python 3.13, git, cmake, node.js are all msi-based installers - so these are likely the culprits on using diskpart - essentially MSI installers check if there's enough disk space before installing items

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#472

Post by danielhanchen »

Zopieux wrote: post #250
danielhanchen wrote: post #73

Earlier quoted context omitted.

We made Unsloth Studio which should help :) 1. Auto best official parameters set for all models 2. Auto determines the largest quant that can fit on your PC / Mac etc 3. Auto determines max context length 4. Auto heals tool calls, provides python & bash + web search :)

Thanks for that. Did you notice that the unsloth/unsloth docker image is 12GB? Does it embed CUDA libraries or some default models that justifies the heavy footprint?

Hey so sorry didn't reply sooner - yes the docker used to be I think 4-8GB ish since CUDA sadly itself is 4GB I think, and PyTorch takes the rest. So unfortunately the Unsloth Docker image has ballooned due to this. We tried reducing it as much as possible, but it's hard :( https://hub.docker.com/r/vllm/vllm-openai/tags for eg is around 11GB ish, ad we're 13.6GB ish.

We'll try our best to compress it more, but it's tough

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#473

Post by danielhanchen »

WanderPanda wrote: post #206
danielhanchen wrote: post #73

Earlier quoted context omitted.

We made Unsloth Studio which should help :) 1. Auto best official parameters set for all models 2. Auto determines the largest quant that can fit on your PC / Mac etc 3. Auto determines max context length 4. Auto heals tool calls, provides python & bash + web search :)

I applaud that you recently started providing the KL divergence plots that really help understand how different quantizations compare. But how well does this correlate with closed loop performance? How difficult/expensive would it be to run the quantizations on e.g. some agentic coding benchmarks?

Hey! Sorry for not replying sooner - yes we'll keep publishing more KLD - sadly some are saying we are "optimizing" for KLD now since we posted so many haha - but the whole purpose of quantization is to match the BF16 logits as much as possible whilst reducing disk space (ie reduce KLD).

In general so this is funny and a quirk of quantization - sometimes 8bit, 4bit models do BETTER on downstream benchmarks (SWE Bench for eg), since sometimes rounding can actually somehow act as a "regularization" method (this is just my hunch).

So KLD isn't that expensive, since we leverage the trick of causal attention - since causal attention is lower triangular, we can do 1 forward pass on the enter text (say 2048 tokens), and you attain logits for the prediction for every token's position - so this is O(N^2).

However coding benchmarking require actual inference, and cannot use the causal attention trick, and it's best to run them 10 times since temperature = 1.0 is not deterministic - and take an average. We plan to maybe do something like https://marginlab.ai/trackers/claude-code/, which takes a random sample and does it over time.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#474

Post by danielhanchen »

cyanydeez wrote: post #192
danielhanchen wrote: post #73

Earlier quoted context omitted.

We made Unsloth Studio which should help :) 1. Auto best official parameters set for all models 2. Auto determines the largest quant that can fit on your PC / Mac etc 3. Auto determines max context length 4. Auto heals tool calls, provides python & bash + web search :)

Is unsloth working on managing remote servers, like how vscode integrates with a remote server via ssh?

Hey sorry on the delay - we just added API support, so you can access a remote server - it includes optional python, tool call, bash and web search support if you enable them.

For SSH - we haven't yet done that - for now we have a SHA256 encryption approach, but it's not SSH yet. HTTPS will also sadly have to be the end user's setup process as well - we plan to make it better soon!

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#475

Post by danielhanchen »

kristjansson wrote: post #201
cyanydeez wrote: post #192

Earlier quoted context omitted.

Is unsloth working on managing remote servers, like how vscode integrates with a remote server via ssh?

Lmstudio Link is GREAT for that right now

Oh yes LM Link is cool!

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#476

Post by danielhanchen »

wuschel wrote: post #101
danielhanchen wrote: post #73

Earlier quoted context omitted.

We made Unsloth Studio which should help :) 1. Auto best official parameters set for all models 2. Auto determines the largest quant that can fit on your PC / Mac etc 3. Auto determines max context length 4. Auto heals tool calls, provides python & bash + web search :)

Great project! Thank you for that!

Thank you and appreciate it! Sorry on the delayed reply as well

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#477

Post by danielhanchen »

jbellis wrote: post #180
danielhanchen wrote: post #73

Earlier quoted context omitted.

We made Unsloth Studio which should help :) 1. Auto best official parameters set for all models 2. Auto determines the largest quant that can fit on your PC / Mac etc 3. Auto determines max context length 4. Auto heals tool calls, provides python & bash + web search :)

what are you using for web search?

We use Duck Duck Go - sorry on the delayed response as well

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#479

Post by danielhanchen »

verdverm wrote: post #215
danielhanchen wrote: post #53

Earlier quoted context omitted.

Haha :) We had some issues with Kimi-2.6 since it was int4 and we were investigating how to handle it :)

Appreciate what y'all do! We were slacking about how many HGX-B300 it would take to run Kimi and it looks like we could actually fit 2-3 Kimis on a single HGX.

Sorry on the delay - oh haha that would be cool :) We did release 2bit dynamic ones, but unsure if they'll be helpful

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#480

Post by danielhanchen »

DarmokJalad1701 wrote: post #172
danielhanchen wrote: post #51

Earlier quoted context omitted.

We also made some dynamic MLX ones if they help - it might be faster for Macs, but llama-server definitely is improving at a fast pace. https://huggingface.co/unsloth/Qwen3.6-27B-UD-MLX-4bit

What exactly does the .sh file install? How does it compare to running the same model in, say, omlx?

Sorry on the delay - so it installs https://github.com/Blaizzy/mlx-vlm and other components and sets up the commands - you don't need to use it but we thought it might be easier for folks
Post reply on HN