Qwen 3 now supports ARM and MLX

The unfortunate thing with their Qwen3-next naming is that it doesn't reflect on the fact that the architecture is completely different from Qwen3. Much more different than the difference between Qwen2 and Qwen3 even.

So support is likely to take quite some time because it's not just regular transformer blocks stacked on each other, but a brand new hybrid architecture using SSM.

Re: Qwen 3 now supports ARM and MLX

#5

Post by NiekvdMaas » Sat, Sep 13, 2025, 7:58 AM UTC

littlestymaar wrote: post #4

ukuina wrote: post #2

[June 2025]

The unfortunate thing with their Qwen3-next naming is that it doesn't reflect on the fact that the architecture is completely different from Qwen3. Much more different than the difference between Qwen2 and Qwen3 even. So support is likely to take quite some time because it's not just regular transformer blocks stacked on each other, but a brand new hybrid architecture using SSM.

From https://github.com/ggml-org/llama.cpp/issues/15940#issuecomm...:

> This is a massive task, likely 2-3 months of full-time work for a highly specialized engineer. Until the Qwen team contributes the implementation, there are no quick fixes.

Re: Qwen 3 now supports ARM and MLX

#6

Post by p0w3n3d » Sat, Sep 13, 2025, 8:22 AM UTC

ukuina wrote: post #2

[June 2025]

Yeah I've been using qwen3 on mlx in July already

Re: Qwen 3 now supports ARM and MLX

#7

Post by veber-alex » Sat, Sep 13, 2025, 8:25 AM UTC

littlestymaar wrote: post #4

ukuina wrote: post #2

[June 2025]

The unfortunate thing with their Qwen3-next naming is that it doesn't reflect on the fact that the architecture is completely different from Qwen3. Much more different than the difference between Qwen2 and Qwen3 even. So support is likely to take quite some time because it's not just regular transformer blocks stacked on each other, but a brand new hybrid architecture using SSM.

It's already supported in vLLM, SGLang and MLX.

Re: Qwen 3 now supports ARM and MLX

#8

Post by littlestymaar » Sat, Sep 13, 2025, 10:44 AM UTC

veber-alex wrote: post #7

littlestymaar wrote: post #4

Earlier quoted context omitted.

The unfortunate thing with their Qwen3-next naming is that it doesn't reflect on the fact that the architecture is completely different from Qwen3. Much more different than the difference between Qwen2 and Qwen3 even. So support is likely to take quite some time because it's not just regular transformer blocks stacked on each other, but a brand new hybrid architecture using SSM.

It's already supported in vLLM, SGLang and MLX.

The Qwen team made sure to land PRs to vLLM and SGLang on the first day, which is nice.

Re: Qwen 3 now supports ARM and MLX

#9

Post by anemll » Sun, Sep 14, 2025, 11:55 PM UTC

It’s also supported in Apple Neural Engine https://github.com/Anemll/Anemll