Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.7 Preview

twitter.com

Post reply on HN
Print view View original

91–100 of 100 posts

Re: Qwen 3.7 Preview

#91

Post by benterix »

jwr wrote: post #89
argee wrote: post #62

Earlier quoted context omitted.

> qwen models (qwen3.6-35b-a3b and earlier 3.5) are WAY better with vision than gemma4-26b-a4b Can you give an example? And/or is there a benchmark specifically for this?

I have my own benchmark: a photo of filament boxes in my drawer. The task is to list (inventory) the filaments. It's not an easy task: the letters are small, the boxes are in various orientations, the labels have lots of writing on them, etc. This is what qwen produces (basically perfect): Fiberlogy (Blue Boxes - Left Side) ASA Black: 750 g (0.75 kg) Easy PLA Graphite: 850 g (0.85 kg) Ref Easy PLA Gray: 850 g (0.85 k…

Thanks, that's very useful. I find people's small individual tests more important than the usual benchmarks that tend to be gamed by every single lab.

Re: Qwen 3.7 Preview

#92

Post by cyanydeez »

jononor wrote: post #86
cyanydeez wrote: post #80

Earlier quoted context omitted.

there are benchmarks that have nothing to do with the training material, but with how the models are capable of things like reading code: https://needle-bench.cc/ Generally, you give them a document and you ask them to retrieve some subsection of the document then rate them on what they retrieved. You can always find enough random documents, or create your own, to always run these and you can make it arbitrarily long…

This seems like a viable eval strategy. Presumably finding a bug requires some degree of understanding of the code, beyond just information retrieval. However it probably does not measure things like prompt adherence or ability to create code that implements a specification?

you can extend the test pretty easily. run through design turns and ask it for it again and again. effectively measure context length.

ask it to modify lines 120-130 and add more context, etc.

we have rudimentry preLLM algoritms that can measure hamming distance and hashing.

you could even go all https://en.wikipedia.org/wiki/Jabberwocky to see if its sense of context is easily polluted.

the point though is there are benchmarks beyong pelican on a bike that cant be tokenmaxx and prove real value in capabilities

Re: Qwen 3.7 Preview

#93

Post by argee »

jwr wrote: post #89
argee wrote: post #62

Earlier quoted context omitted.

> qwen models (qwen3.6-35b-a3b and earlier 3.5) are WAY better with vision than gemma4-26b-a4b Can you give an example? And/or is there a benchmark specifically for this?

I have my own benchmark: a photo of filament boxes in my drawer. The task is to list (inventory) the filaments. It's not an easy task: the letters are small, the boxes are in various orientations, the labels have lots of writing on them, etc. This is what qwen produces (basically perfect): Fiberlogy (Blue Boxes - Left Side) ASA Black: 750 g (0.75 kg) Easy PLA Graphite: 850 g (0.85 kg) Ref Easy PLA Gray: 850 g (0.85 k…

Thanks. Did you set the image min/max tokens for Gemma4 to 1120 for this? This might not be a fair comparison without that, to the differences in architecture.

https://www.reddit.com/r/KoboldAI/comments/1sjnjic/imagemin_...

https://github.com/ollama/ollama/issues/15626

I think 1120 vs 280 tokens is a big difference, and you were perhaps using the latter value?

Re: Qwen 3.7 Preview

#94

Post by Alifatisk »

kethinov wrote: post #14

Can someone explain what the current state of model benchmarking is? If you try to look up what the best locally runnable model is, you get a bunch of random blog posts using idiosyncratic criteria to rank things seemingly based on one dude's opinion. Ideally I would love to see a leaderboard with relatively objective ranking criteria that 1. lets you filter by open weight / locally runnable, 2. filter by date of rel…

> Ideally I would love to see a leaderboard with relatively objective ranking criteria that 1. lets you filter by open weight / locally runnable, 2. filter by date of release (nothing older than x), and 3. is agnostic to hardware requirements. I just want to know what the best model is. Let me worry about how I will afford to run it.

Stick to artificialanalysis.ai it has become the norm

Re: Qwen 3.7 Preview

#95

Post by throwa356262 »

Computer0 wrote: post #58
genxy wrote: post #37

Earlier quoted context omitted.

Understood, but look at their larger cadence over the years and the breadth of models. They are clearly not all finetunes. Meta for all its billions, doesn't have anything comparable.

competent leadership goes a long way

That was true up to Qwen 3.5, everything after that is made by the same people that made Gemini 3.1 suck

Re: Qwen 3.7 Preview

#96

Post by throwa356262 »

bachmeier wrote: post #57

I'm not much interested in vibe coding (for those who aren't aware that LLMs have other uses). The specific model I've been using with Ollama is hf.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:UD-Q4_K_XL and it's amazing how fast it is on 64 GB of RAM and i5-13400 CPU. No GPU on this computer. Gemma 4 E4B will think for a couple of minutes vs 3-5 seconds for Qwen. It's hard to believe how much you can do with such li…

Since you are using unsloth models from HF, why not use Unsloth Studio instead of Ollama?

It is supposed to be faster + they will update the new models multiple times during the first month to correct bugs and performance issues

https://unsloth.ai/docs/new/studio#quickstart

Re: Qwen 3.7 Preview

#97

Post by jwr »

argee wrote: post #93
jwr wrote: post #89

Earlier quoted context omitted.

I have my own benchmark: a photo of filament boxes in my drawer. The task is to list (inventory) the filaments. It's not an easy task: the letters are small, the boxes are in various orientations, the labels have lots of writing on them, etc. This is what qwen produces (basically perfect): Fiberlogy (Blue Boxes - Left Side) ASA Black: 750 g (0.75 kg) Easy PLA Graphite: 850 g (0.85 kg) Ref Easy PLA Gray: 850 g (0.85 k…

Thanks. Did you set the image min/max tokens for Gemma4 to 1120 for this? This might not be a fair comparison without that, to the differences in architecture. https://www.reddit.com/r/KoboldAI/comments/1sjnjic/imagemin_... https://github.com/ollama/ollama/issues/15626 I think 1120 vs 280 tokens is a big difference, and you were perhaps using the latter value?

I did not, and I had no idea such a setting even existed. This could definitely change things. However, I don't see a way to set this in LM Studio, which is what I currently use to run models.

Re: Qwen 3.7 Preview

#98

Post by argee »

jwr wrote: post #97
argee wrote: post #93

Earlier quoted context omitted.

Thanks. Did you set the image min/max tokens for Gemma4 to 1120 for this? This might not be a fair comparison without that, to the differences in architecture. https://www.reddit.com/r/KoboldAI/comments/1sjnjic/imagemin_... https://github.com/ollama/ollama/issues/15626 I think 1120 vs 280 tokens is a big difference, and you were perhaps using the latter value?

I did not, and I had no idea such a setting even existed. This could definitely change things. However, I don't see a way to set this in LM Studio, which is what I currently use to run models.

Seems like you can't set it, for now. There's an issue for it: https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1...

Re: Qwen 3.7 Preview

#99

Post by throwa356262 »

mempko wrote: post #26

I love that open weight models are catching up so quickly. Also hilarious how far behind Grok is. I guess demand for Grok must be poor if Anthropic is able to rent resources from xAI.

Would xAI do this if Grok was competitive?

https://www.nytimes.com/2026/04/03/business/spacex-ipo-grok-...

Re: Qwen 3.7 Preview

#100

Post by Jach »

boppo1 wrote: post #83
Jach wrote: post #63

Earlier quoted context omitted.

I sort of thought this about qwen3.5 35b, finally a local model that isn't a complete waste of electricity, but "upgrading" to 3.6 35b left me disappointed. It seemed more like a downgrade. But honestly I've barely used either. Subjectively they still seem far from the frontier models, but for what they can do, it's great to be able to do locally.

How are they just for chat / questions?

Pretty decent, it's given similar book recommendations as Claude when I feed it my list of read books and thoughts on them. You'll have to tell them to never use emojis. I was using 3.5 a while ago to generate some flavor text while I was playing a bit of an old-school dungeon-crawler game (it's like Wizardry), a genre I don't particularly enjoy much, but it's funner with the flavor text. Worth setting up something like open webui or other front-ends since a pure CLI experience via ollama is pretty bad.
Post reply on HN