Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.7-Max: The Agent Frontier

qwen.ai

Post reply on HN
Print view View original

271–280 of 317 posts

Re: Qwen3.7-Max: The Agent Frontier

#271

Post by CamperBob2 »

tandr wrote: post #230
gcr wrote: post #11

Earlier quoted context omitted.

What’s the price point for getting into that sweet spot? I’m on an M1 Max with 32GB VRAM, so I’m looking forward to the 27B or 35B-A3B models. Is dropping $5k for an RTX 6000 or a DGX Spark really the best option?

Don't mind me asking, but where did you find $5k RTX 6000? Even 48GB model (previous gen) shows minimum at 7k, and 96GB one (Blackwell) is ~10k on Amazon...

$5K is presumably what it costs to pay some local gangsters to break into an nVidia warehouse. That's the only you will pay $5K for an RTX 6000 for the next couple of years.

The server edition has gone up $2K in the last couple of weeks alone, at the outlet where I bought one previously.

Re: Qwen3.7-Max: The Agent Frontier

#272

Post by Balinares »

jampekka wrote: post #193
throawayonthe wrote: post #81

Earlier quoted context omitted.

referencing this: https://artificialanalysis.ai/evaluations/omniscience?models... (had to add it to the chart, wasn't displayed by default. is it the lowest rate in the datasetor no?)

This counts only incorrect answers though. A model can get 0% hallucination rate just by refusing to answer all questions.

Yes, that's in fact precisely the desired behavior when a model doesn't know the answer.

Re: Qwen3.7-Max: The Agent Frontier

#273

Post by rrgok »

doublescoop wrote: post #240
ffsm8 wrote: post #214

Earlier quoted context omitted.

Isn't that precisely the reason why we introduced the term hallucination? Because llms have historically always made up bullshit of they cannot answer directly... If they now nailed this to maybe the model not respond instead of responding incorrectly, then a lot of previously unusable usecases would become feasible. So I feel like that's exactly the right metric and the way to track it wrt hallucinations.

I had a buddy in high school that was notorious for doing the same thing. (He's now a senior director at a Big 4 consultancy. :) )

Do you mind expanding a little more?

Re: Qwen3.7-Max: The Agent Frontier

#274

Post by Ey7NFZ3P0nzAe »

Eridrus wrote: post #225
goyozi wrote: post #2

These are very good numbers. I still don’t get why they don’t compare against latest competitor versions in these posts, it’s not like we’re all not going to notice.

Nobody releases numbers that show them to be worse than competitors lol. This even applies to OpenAI & Anthropic who don't even eval on the same datasets a lot of the time.

I do recall mistral doing this. It's not always about being the best, but also fastest or smallest. The user should have all the information for its own use case.

Re: Qwen3.7-Max: The Agent Frontier

#275

Post by khimaros »

gcr wrote: post #160
pixelesque wrote: post #150

Earlier quoted context omitted.

Out of interest, what machine and model are you running it on? I tried the qwen3.6-27b Q6_k GUFF in llama.cpp and LM Studio on my M2 MacBook Pro 32GB machine last week, and I barely get a token a second with either. What sort of speed should I be expecting? I tried some of the Llama 3 34b (nous-capybara?) models two years ago with llama.cpp, and I seem to remember getting a few tokens a second then, so not sure if I'…

There are two flavors of Qwen 3.6: - A 27B "dense" model - A 35B "Mixture of Experts" model, which activates only 3B parameters for each token. For your hardware, I strongly recommend `unsloth/Qwen3.6-35B-A3B-GGUF:Q4_K_M`. I have an M1 Max with 32GB VRAM from 2021 that can read at ~300-500 tokens/sec and write at ~30 tokens/sec with llama-cpp's default settings, which is plenty fast. The 27B model can read ~70tok/sec…

MTP recommended

Re: Qwen3.7-Max: The Agent Frontier

#276

Post by joelanman »

gcr wrote: post #160
pixelesque wrote: post #150

Earlier quoted context omitted.

Out of interest, what machine and model are you running it on? I tried the qwen3.6-27b Q6_k GUFF in llama.cpp and LM Studio on my M2 MacBook Pro 32GB machine last week, and I barely get a token a second with either. What sort of speed should I be expecting? I tried some of the Llama 3 34b (nous-capybara?) models two years ago with llama.cpp, and I seem to remember getting a few tokens a second then, so not sure if I'…

There are two flavors of Qwen 3.6: - A 27B "dense" model - A 35B "Mixture of Experts" model, which activates only 3B parameters for each token. For your hardware, I strongly recommend `unsloth/Qwen3.6-35B-A3B-GGUF:Q4_K_M`. I have an M1 Max with 32GB VRAM from 2021 that can read at ~300-500 tokens/sec and write at ~30 tokens/sec with llama-cpp's default settings, which is plenty fast. The 27B model can read ~70tok/sec…

Using omlx on the M1 max I get about 15tps from 27b

Re: Qwen3.7-Max: The Agent Frontier

#277

Post by Mashimo »

dfansteel wrote: post #213
Mashimo wrote: post #36

Earlier quoted context omitted.

> I’m honestly not able to run it What do you mean? This is not self hosted, it's closed source. And any website that targets China or is hosted in China will probably censor Tiananmen Square.

My computer lacks the ram.

Well if it had all the ram, you still could not run it because you can't download it. It's hosted by them, just like chatgpt or claude.

Re: Qwen3.7-Max: The Agent Frontier

#278

Post by tarruda »

smcleod wrote: post #218
tarruda wrote: post #12

Earlier quoted context omitted.

I have a 128G mac studio and even 397B was a happy surprise to me due to its high quantization resilience. I've created a 2.54BPW quant that fit on my hardware with 128k context, 20 tps tg and 200tps pp, while maintaining high scores on many benchmarks: https://huggingface.co/tarruda/Qwen3.5-397B-A17B-GGUF/discus...

That's impressive getting a 397B down to <110GB~. HF link is broken though!

> That's impressive getting a 397B down to It is higher than 110GB. MacOS allows up to 125G of the RAM to be shared with GPU, so it is certainly less than that!

> HF link is broken though!

Doesn't seem broken to me, but you should be able to search for tarruda/Qwen3.5-397B-A17B-GGUF on huggingface.

Re: Qwen3.7-Max: The Agent Frontier

#279

Post by jampekka »

speed_spread wrote: post #215
jampekka wrote: post #193

Earlier quoted context omitted.

This counts only incorrect answers though. A model can get 0% hallucination rate just by refusing to answer all questions.

Yes. A model that can answer "I don't know" would be much more trustable than the current used car salesman we have now.

Models can answer "I don't know". Hallucination benchmarks, including this, give the models the option to "not attempt". It's just that the metric linked doesn't take into account the rate of correct answers at all. It has its uses in analyzing incorrect vs not attempted answers, but gives a very partial picture.
Post reply on HN