Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3-Coder-Next

qwen.ai

Post reply on HN
Print view View original

361–370 of 443 posts

Re: Qwen3-Coder-Next

#361

Post by giancarlostoro »

andai wrote: post #205
volkercraig wrote: post #119

Earlier quoted context omitted.

Absolutely. I installed clawdbot for just long enough to send a single message, and it burned through almost a quarter of my session allowance. That was enough for me. Meanwhile I can use CC comfortably for a few hours and I've only hit my token limit a few times. I've had a similar experience with opencode, but I find that works better with my local models anyway.

I used it for a few mins and it burned 7M tokens. Wish there was a way to see where it's going! (There probably is, but I found it very hard to make sense of the UI and how everything works. Hard to change models, no chat history etc.?)

I have a feeling the different harnesses create new context windows instead of using one. The more context windows you open up with Claude the quicker your usage goes poof.

Re: Qwen3-Coder-Next

#362

Post by pstuart »

openclawai wrote: post #332
kristopolous wrote: post #255

Earlier quoted context omitted.

We need a new word, not "local model" but "my own computers model" CapEx based This distinction is important because some "we support local model" tools have things like ollama orchestration or use the llama.cpp libraries to connect to models on the same physical machine. That's not my definition of local. Mine is "local network". so call it the "LAN model" until we come up with something better. "Self-host" exists b…

For context on what cloud API costs look like when running coding agents: With Claude Sonnet at $3/$15 per 1M tokens, a typical agent loop with ~2K input tokens and ~500 output per call, 5 LLM calls per task, and 20% retry overhead (common with tool use): you're looking at roughly $0.05-0.10 per agent task. At 1K tasks/day that's ~$1.5K-3K/month in API spend. The retry overhead is where the real costs hide. Most cost…

Might there be a way to leverage local models just to help minimize the retries -- doing the tool calling handling and giving the agent "perfect execution"?

I'm a noob and am asking as wishful thinking.

Re: Qwen3-Coder-Next

#364

Post by redundantly »

mark_l_watson wrote: post #327
simonw wrote: post #3

This GGUF is 48.4GB - https://huggingface.co/Qwen/Qwen3-Coder-Next-GGUF/tree/main/... - which should be usable on higher end laptops. I still haven't experienced a local model that fits on my 64GB MacBook Pro and can run a coding agent like Codex CLI or Claude code well enough to be useful. Maybe this will be the one? This Unsloth guide from a sibling comment suggests it might be: https://unsloth.ai/docs/models/qwen3…

I configured Claude Code to use a local model (ollama run glm-4.7-flash) that runs really well on a 32G M2Pro macmini. Maybe my standards are too low, but I was using that combination to clean up the code, make improvements, and add docs and tests to a bunch of old git repo experiment projects.

Did you have to do anything special to get it to work? I tried and it would just bug out, things like respond with JSON strings summarizing what I asked of it or just outright getting things wrong entirely. For example, I asked it to summarize what a specific .js file did and it provided me with new code it made up based on the file name...

Re: Qwen3-Coder-Next

#365

Post by EGreg »

taneq wrote: post #346
openclawai wrote: post #332

Earlier quoted context omitted.

For context on what cloud API costs look like when running coding agents: With Claude Sonnet at $3/$15 per 1M tokens, a typical agent loop with ~2K input tokens and ~500 output per call, 5 LLM calls per task, and 20% retry overhead (common with tool use): you're looking at roughly $0.05-0.10 per agent task. At 1K tasks/day that's ~$1.5K-3K/month in API spend. The retry overhead is where the real costs hide. Most cost…

At this point isn’t the marginal cost based on power consumption? At 30c/kWh and with a beefy desktop pc pulling up to half a kW, that’s 15c/hr. For true zero marginal cost, maybe get solar panels. :P

This is an interesting question actually!

Marginal cost includes energy usage but also I burned out a MacBook GPU with vanity-eth last year so wear-and-tear is also a cost.

Re: Qwen3-Coder-Next

#366

Post by bityard »

danielhanchen wrote: post #357
bityard wrote: post #190

Earlier quoted context omitted.

Hi Daniel, I've been using some of your models on my Framework Desktop at home. Thanks for all that you do. Asking from a place of pure ignorance here, because I don't see the answer on HF or in your docs: Why would I (or anyone) want to run this instead of Qwen3's own GGUFs?

Thanks! Oh Qwen3's own GGUFs also works, but ours are dynamically quantized and calibrated with a reasonably large diverse dataset, whilst Qwen's ones are not - see https://unsloth.ai/docs/basics/unsloth-dynamic-2.0-ggufs

I've read that page before and although it all certainly sounds very impressive, I'm not an AI researcher. What's the actual goal of dynamic quantization? Does it make the model more accurate? Faster? Smaller?

Re: Qwen3-Coder-Next

#367

Post by mark_l_watson »

redundantly wrote: post #364
mark_l_watson wrote: post #327

Earlier quoted context omitted.

I configured Claude Code to use a local model (ollama run glm-4.7-flash) that runs really well on a 32G M2Pro macmini. Maybe my standards are too low, but I was using that combination to clean up the code, make improvements, and add docs and tests to a bunch of old git repo experiment projects.

Did you have to do anything special to get it to work? I tried and it would just bug out, things like respond with JSON strings summarizing what I asked of it or just outright getting things wrong entirely. For example, I asked it to summarize what a specific .js file did and it provided me with new code it made up based on the file name...

Yes, I had to set the Ollama context size to 32K

Re: Qwen3-Coder-Next

#368

Post by Computer0 »

margalabargala wrote: post #262
Aurornis wrote: post #248

Earlier quoted context omitted.

I experimented with the Q2 and Q4 quants. First impression is that it's amazing we can run this locally, but it's definitely not at Sonnet 4.5 level at all. Even for my usual toy coding problems it would get simple things wrong and require some poking to get to it. A few times it got stuck in thinking loops and I had to cancel prompts. This was using the recommended settings from the unsloth repository. It's always p…

Wonder where it falls on the Sonnet 3.7/4.0/4.5 continuum. 3.7 was not all that great. 4 was decent for specific things, especially self contained stuff like tests, but couldn't do a good job with more complex work. 4.5 is now excellent at many things. If it's around the perf of 3.7, that's interesting but not amazing. If it's around 4, that's useful.

I still have yet to find a "Small" model that can use function calls consistently enough to not be frustrating. That is the most noticeable difference I consistently see between even older "SOTA" models and the best performing "SMALL" models (<70b).

Re: Qwen3-Coder-Next

#369

Post by redwood_ »

mittermayr wrote: post #223
organsnyder wrote: post #89

Earlier quoted context omitted.

They run fairly well for me on my 128GB Framework Desktop.

what do you run this on if I may ask? lmstudio, ollama, lama? which cli?

I run Qwen3-Coder-Next (Qwen3-Coder-Next-UD-Q4_K_XL) on the Framework ITX board (Max+ 395 - 128GB) custom build. Avg. eval at 200-300 t/s and output at 35-40 t/s running with llama.cpp using rocm. Prefer Claude Code for cli.

Re: Qwen3-Coder-Next

#370

Post by Aurornis »

airstrike wrote: post #325
Aurornis wrote: post #320

Earlier quoted context omitted.

> I do not want my career to become dependent upon Anthropic As someone who switches between Anthropic and ChatGPT depending on the month and has dabbled with other providers and some local LLMs, I think this fear is unfounded. It's really easy to switch between models. The different models have some differences that you notice over time but the techniques you learn in one place aren't going to lock you into a provid…

right, but ChatGPT might not exist at some point, and if we don't force feed the open inference ecosystem and infrastructure back into the mouths of the AI devourer that is this hype cycle, we'll simply be accepting our inevitable, painful death

> right, but ChatGPT might not exist at some point

There are multiple frontier models to choose from.

They’re not all going to disappear.

Post reply on HN