Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3-Coder-Next

qwen.ai

Post reply on HN
Print view View original

291–300 of 443 posts

Re: Qwen3-Coder-Next

#291

Post by pocksuppet »

embedding-shape wrote: post #17
simonw wrote: post #3

This GGUF is 48.4GB - https://huggingface.co/Qwen/Qwen3-Coder-Next-GGUF/tree/main/... - which should be usable on higher end laptops. I still haven't experienced a local model that fits on my 64GB MacBook Pro and can run a coding agent like Codex CLI or Claude code well enough to be useful. Maybe this will be the one? This Unsloth guide from a sibling comment suggests it might be: https://unsloth.ai/docs/models/qwen3…

> I still haven't experienced a local model that fits on my 64GB MacBook Pro and can run a coding agent like Codex CLI or Claude code well enough to be useful I've had mild success with GPT-OSS-120b (MXFP4, ends up taking ~66GB of VRAM for me with llama.cpp) and Codex. I'm wondering if maybe one could crowdsource chat logs for GPT-OSS-120b running with Codex, then seed another post-training run to fine-tune the 20b v…

You are describing distillation, there are better ways to do it, and it was done in the past, Deepseek distilled onto Qwen.

Re: Qwen3-Coder-Next

#292

Post by christkv »

kristopolous wrote: post #255
simonw wrote: post #3

This GGUF is 48.4GB - https://huggingface.co/Qwen/Qwen3-Coder-Next-GGUF/tree/main/... - which should be usable on higher end laptops. I still haven't experienced a local model that fits on my 64GB MacBook Pro and can run a coding agent like Codex CLI or Claude code well enough to be useful. Maybe this will be the one? This Unsloth guide from a sibling comment suggests it might be: https://unsloth.ai/docs/models/qwen3…

We need a new word, not "local model" but "my own computers model" CapEx based This distinction is important because some "we support local model" tools have things like ollama orchestration or use the llama.cpp libraries to connect to models on the same physical machine. That's not my definition of local. Mine is "local network". so call it the "LAN model" until we come up with something better. "Self-host" exists b…

I won't need a heater with that running in my room.

Re: Qwen3-Coder-Next

#293

Post by djeastm »

bri3d wrote: post #113
Nemi wrote: post #94

Earlier quoted context omitted.

> The subscription services have assumptions baked in about the usage patterns; they're oversubscribed and subsidized. Selling dollars for $.50 does that. It sounds like they have a business model issue to me.

This is how every cloud service and every internet provider works. If you want to get really edgy you could also say it's how modern banking works. Without knowing the numbers it's hard to tell if the business model for these AI providers actually works, and I suspect it probably doesn't at the moment, but selling an oversubscribed product with baked in usage assumptions is a functional business model in a lot of spa…

>Without knowing the numbers it's hard to tell if the business model for these AI providers actually works

It'll be interesting to see what OpenAI and Anthropic will tell us about this when they go public (seems likely late this year--along with SpaceX, possibly)

Re: Qwen3-Coder-Next

#294

Post by bigyabai »

FrenchTouch42 wrote: post #290
bigyabai wrote: post #273

Earlier quoted context omitted.

OOM is a pretty terrible benchmark too, though. You can build a DDR4 machine that "technically" loads 256gb models for maybe $1000 used, but then you've got to account for the compute aspect and that's constrained by a number of different variables. A super-sparse model might run great on that DDR4 machine, whereas a 32b model would cause it to chug. There's just not a good way to visualize the compute needed, with a…

> time-to-first-token/token-per-second/memory-used/total-time-of-test Would it not help with the DDR4 example though if we had more "real world" tests?

Maybe, but even that fourth-order metric is missing key performance details like context length and model size/sparsity.

The bigger takeaway (IMO) is that there will never really be hardware that scales like Claude or ChatGPT does. I love local AI, but it stresses the fundamental limits of on-device compute.

Re: Qwen3-Coder-Next

#295

Post by cmrdporcupine »

skhameneh wrote: post #23

It’s hard to elaborate just how wild this model might be if it performs as claimed. The claims are this can perform close to Sonnet 4.5 for assisted coding (SWE bench) while using only 3B active parameters. This is obscenely small for the claimed performance.

It feels more like Haiku level than Sonnet 4.5 from my playing with it.

Re: Qwen3-Coder-Next

#296

Post by cmrdporcupine »

Der_Einzige wrote: post #245
cirrusfan wrote: post #28

Earlier quoted context omitted.

If it sounds too good to be true…

It literally always is. HN Thought DeepSeek and every version of Kimi would finally dethrone the bigger models from Anthropic, OpenAI, and Google. They're literally always wrong and average knowledge of LLMs here is shockingly low.

Nobody has been saying they'd be dethroned. We're saying they're often "good enough" for many use cases, and that they're doing a good job of stopping the Big Guys from creating a giant expensive moat around their businesses.

Chinese labs are acting as a disruption against Altman etcs attempt to create big tech monopolies, and that's why some of us cheer for them.

Re: Qwen3-Coder-Next

#297

Post by MzxgckZtNqX5i »

onion2k wrote: post #286
kybernetikos wrote: post #261

Earlier quoted context omitted.

Gpt3.5 as used in the first commercially available chat gpt is believed to be hundreds of billions of parameters. There are now models I can run on my phone that feel like they have similar levels of capability. Phones are never going to run the largest models locally because they just don't have the size, but we're seeing improvements in capability at small sizes over time that mean that you can run a model on your…

The G in GPT stands for Generalized. You don't need that for specialist models, so the size can be much smaller. Even coding models are quite general as they don't focus on a language or a domain. I imagine a model specifically for something like React could be very effective with a couple of billion parameters, especially if it was a distill of a more general model.

I'll be that guy: the "G" in GPT stands for "Generative".

Re: Qwen3-Coder-Next

#298

Post by zozbot234 »

kristopolous wrote: post #255
simonw wrote: post #3

This GGUF is 48.4GB - https://huggingface.co/Qwen/Qwen3-Coder-Next-GGUF/tree/main/... - which should be usable on higher end laptops. I still haven't experienced a local model that fits on my 64GB MacBook Pro and can run a coding agent like Codex CLI or Claude code well enough to be useful. Maybe this will be the one? This Unsloth guide from a sibling comment suggests it might be: https://unsloth.ai/docs/models/qwen3…

We need a new word, not "local model" but "my own computers model" CapEx based This distinction is important because some "we support local model" tools have things like ollama orchestration or use the llama.cpp libraries to connect to models on the same physical machine. That's not my definition of local. Mine is "local network". so call it the "LAN model" until we come up with something better. "Self-host" exists b…

You can run plenty of models on a $10K machine or even a lot less than that, it all depends how much you want to wait for results. Streaming weights from SSD storage using mmap() is already a reality when running the largest and sparsest models. You can save even more on memory by limiting KV caching at the cost of extra compute, and there may be ways to push RAM savings even higher simply by tweaking the extent to which model activations are recomputed as needed.

Re: Qwen3-Coder-Next

#299

Post by Vinnl »

johnsmith1840 wrote: post #203
dehrmann wrote: post #47

Earlier quoted context omitted.

I wonder if the future in ~5 years is almost all local models? High-end computers and GPUs can already do it for decent models, but not sota models. 5 years is enough time to ramp up memory production, consumers to level-up their hardware, and models to optimize down to lower-end hardware while still being really good.

Opensource or local models will always heavily lag frontier. Who pays for a free model? GPU training isn't free! I remember early on people saying 100B+ models will run on your phone like nowish. They were completely wrong and I don't think it's going to ever really change. People always will want the fastest, best, easiest setup method. "Good enough" massively changes when your marketing team is managing k8s cluster…

> People always will want the fastest, best, easiest setup method

When there are no other downsides, sure. But when the frontier companies start tightening the thumbscrews, price will influence what people consider good enough.

Re: Qwen3-Coder-Next

#300

Post by codazoda »

simonw wrote: post #3

This GGUF is 48.4GB - https://huggingface.co/Qwen/Qwen3-Coder-Next-GGUF/tree/main/... - which should be usable on higher end laptops. I still haven't experienced a local model that fits on my 64GB MacBook Pro and can run a coding agent like Codex CLI or Claude code well enough to be useful. Maybe this will be the one? This Unsloth guide from a sibling comment suggests it might be: https://unsloth.ai/docs/models/qwen3…

I can't get Codex CLI or Claude Code to use small local models and to use tools. This is because those tools use XML and the small local models have JSON tool use baked into them. No amount of prompting can fix it.

In a day or two I'll release my answer to this problem. But, I'm curious, have you had a different experience where tool use works in one of these CLIs with a small local model?

Post reply on HN