Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.7-Max: The Agent Frontier

qwen.ai

Post reply on HN
Print view View original

261–270 of 317 posts

Re: Qwen3.7-Max: The Agent Frontier

#261

Post by nl »

wuliwong wrote: post #191
briga wrote: post #109

I was getting dangerously close to my weekly Claude Code limit last night so I had Claude set up Qwen3.6 with llama.cpp and OpenCode. Honestly it's a great (free!) alternative to Claude Code--certainly more than good enough for a lot of smaller less complex tasks. I'm excited to try this new version. The fact that open-source models are so close to the frontier is very impressive.

Do you have a feel for how it Qwen 3.6 compares to Sonnet 4.6? B/C in reality, that's what we use a lot. If we just use Opus 4.7 for everything code related, we'd have a monthly bill 10-20 times higher than using Sonnet where we can.

I think you could well be surprised by the Sonnet vs Opus bill (assuming you are paying via the API)

In my experience Sonnet bills can be higher than Opus because it churns a lot more trying to get things right.

Example from my fairly simple but agentic benchmark:

Opus 4.7, 25/25, 81c: https://sql-benchmark.nicklothian.com/?highlight=anthropic_c...

Opus 4.6, 24/25, 61c: https://sql-benchmark.nicklothian.com/?highlight=anthropic_c...

Sonnet 4.6: 24/25, 41c: https://sql-benchmark.nicklothian.com/?highlight=anthropic_c...

I only tested the free OpenRouter version of Qwen 3.6 Plus, and it scored 23/25: https://sql-benchmark.nicklothian.com/?highlight=qwen_qwen3....

This doesn't quite show Opus cheaper, but it isn't the 10-20 times more either. Harder tasks close the gap even further.

Re: Qwen3.7-Max: The Agent Frontier

#265

Post by irishcoffee »

gcr wrote: post #153
plufz wrote: post #113

Earlier quoted context omitted.

Which exact model are you using? And with which parameters and quant? And on what hardware? Are you using any specific MCPs or other tools to optimize performance like context-mode or dynamic context pruning? I’ve used local models a reasonable amount before but I’m just starting out with opencode. Haven’t had great results yet but really want this to work for simpler tasks. My opencode newly installed is also having…

here's a simple setup to get you started on an Apple M1 Max from 2021 with 32GB VRAM. it will download 20GB of models to `~/.cache/huggingface/hub`, which you can delete when you're done. /Users/gcr/llama.cpp/build/bin/llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:Q4_K_M --no-mmproj-offload --fit on -c 65536 # edit to taste --reasoning on --chat-template-kwargs '{"preserve_thinking": true}' --sleep-idle-seconds 90 #…

I have a 5070TI (16gb VRAM) with 32GB system ram and a 16 core AMD cpu. I am considering buying a second used videocard, probably the same model, but not for months yet. This hardware setup is new-for-me in that a buddy gave me most of it and I bought the TI card.

Are there any resources to help me figure out how to best optimize my runtime paramaters for a given model, based on a given task, similar to what you've shown?

I've been a little... irritated? that hooking vscode up to my company LLM subscription seems so much more out-of-the-box capiable than what I can get to work. My assumption at the moment is that I need to create a lot of... I think they're called harnesses? agents? workflows? integrations? (not sure) by hand. Is that accurate?

Right now I have ollama running an nvidia nano model and I can poke it with a stick over a web interface I installed. It works, initial token response is slow, after that it seems fine enough.

I can't seem to get a good handle on how much context I've used, when context usage starts to degrade response accuracy, or in general how to mirror the results I get (not in terms of accuracy or speed, just features) from the company github copilot + vscode integration.

I was also trying to get a plugin called qodeassist working via qtcreator, mixed results there as well.

I've been keeping up with this space since the jump, never paid for a sub, work gave me a sub a handful of weeks ago, so the actual useage is all new to me.

I can't say I'm super impressed with any of it relative to the hype, but I found it neat to be able to point vscode at a c++ codebase and say "enable wextra, build the code, tell me if there is any low-hanging fruit I can clean up" and get a useful response.

I also asked my local model to turn a picture of my dog into a picture of an otter, got a blank picture back, which the thinking bit told me it would do. The whole thing was actually kind of funny. "I am allowed to edit pictures, I can't edit pictures, I am allowed to edit pictures, I'll tell the user I did and send a blank picture back because I can't edit pictures, but I am allowed to."

Re: Qwen3.7-Max: The Agent Frontier

#266

Post by notatoad »

flakiness wrote: post #124

I'm using pi agent and love to try qwen models (hosted). What are the good options? The official provider doesn't include Alibaba. Is OpenRouter etc. fast enough? (As a reference, DeepSeek v4 is severely throttled on these proxy services.)

i use opencode zen as a convenient pay-as-you-go way to try out all these new models. it doesn't have 3.7 yet, but at the rate they usually update it probably will tomorrow.

I couldn’t say how throttled it is, but it seems fine?

Re: Qwen3.7-Max: The Agent Frontier

#267

Post by chengyongru »

miki123211 wrote: post #258
jorvi wrote: post #237

Earlier quoted context omitted.

Its very annoying this has been in the capability of models since the very beginning. It could check how probable its token values are and if those fall below a certain threshold either say "I don't know", or output the most probable (well, more like least improbable) tokens but give a very clear, very strong warning that it is a shot in the dark and likely to contain hallucinations. But no, Google and OpenAI would r…

It can't, because top n isn't always reliable. Hallucination detection is an open problem. If it were that simple, people would indeed "just" do it. Basically the problem is that LLMs aren't trained on things they don't know; an alternative way of saying this is that they're not trained on things they're not trained on, which is obviously true. When you RL a model and it answers incorrectly, you don't teach it to ans…

Models tend to default to their training data even when they lack sufficient context, they've never been trained to recognize their own uncertainty, so they hallucinate confidently instead.

Re: Qwen3.7-Max: The Agent Frontier

#268

Post by akoboldfrying »

ffsm8 wrote: post #214
jampekka wrote: post #193

Earlier quoted context omitted.

This counts only incorrect answers though. A model can get 0% hallucination rate just by refusing to answer all questions.

Isn't that precisely the reason why we introduced the term hallucination? Because llms have historically always made up bullshit of they cannot answer directly... If they now nailed this to maybe the model not respond instead of responding incorrectly, then a lot of previously unusable usecases would become feasible. So I feel like that's exactly the right metric and the way to track it wrt hallucinations.

The point is that it's not a useful metric on its own. For example, redirecting from /dev/null also achieves a zero hallucination rate.

We want the hallucination rate to decrease while the overall answer rate of queries remains sufficiently high. For more specifics, look into ROC and AUC.

Re: Qwen3.7-Max: The Agent Frontier

#269

Post by root_axis »

tantaman wrote: post #197
rlt wrote: post #128

Earlier quoted context omitted.

Well, yes, garbage in garbage out. That's a given and not what's meant by "hallucination" in this context.

the observation goes beyond garbage in garbage out. Mainly that we're always operating from some prior and limited understanding. That what may look like a hallucination could be closer to the truth than our current frameworks of understanding allow us to admit. The hermeneutic circle.

A properly designed benchmark won't use tests that leave room for ambiguous interpretation.

Re: Qwen3.7-Max: The Agent Frontier

#270

Post by anderber »

par wrote: post #250
sunaookami wrote: post #223

Earlier quoted context omitted.

Aider is still around? That is pre-tool-calling era stuff. Better compare against Pi.

I just started running coding agents locally. So you recommend Pi over opencode? (And obviously aider is out?)

I personally found better results with Opencode. But Pi is really nice too.
Post reply on HN