Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

241–250 of 848 posts

Re: Qwen 3.8 27B

#241

Post by expedited123 »

peri-cl wrote: post #175
expedited123 wrote: post #123

Earlier quoted context omitted.

Mind sharing your laptops specs? Just interested to see what is needed to locally run Qwen3.6-35B-A3B

Don't mind! It's 64 GiB dual-channel DDR5-6400, i.e. roughly 100 GiB/s of bandwidth. (AMD 7840U (Zen 4)) I'm using a Q4 quantization from unsloth (Qwen3.6-35B-A3B-UD-Q4_K_XL). It gets up to ~20 tokens/second in generation. I don't know precisely how much KV cache I can safely use, but it's in between 140k–256k. (I.e., 140k reliably works, 256k kernel-crashes from OOM. Don't feel like bisecting). Inference is llama.cp…

Thanks! I don't have any knowledge of running models locally.

I assume it would not be able to handle an unquantized Qwen3.6-35B or is it irrelevant as you almost always would want to run a quantized version of the model on consumer hardware?

Re: Qwen 3.8 27B

#242

Post by satvikpendem »

altruios wrote: post #235
satvikpendem wrote: post #177

Earlier quoted context omitted.

You should check out Grok, it's quite a good deal from the Cursor subscription side but it's cheap even by API prices.

[flagged]

Yeah I'm sure everyone on r/cursor or in previous HN threads about Grok 4.5 or 4.6 are all unserious and insane.

No one actually cares about the politics as long as the model codes well.

Edit, quite interesting to see the reception to this comment compared to essentially the same type of comment I made on a Grok 4.6 benchmark HN post: https://news.ycombinator.com/item?id=49275385#49275571

It's true that Cursor gives a lot of usage with Grok, most users of Cursor don't care about Musk.

Re: Qwen 3.8 27B

#243

Post by ferrouswheel »

redox99 wrote: post #143
NitpickLawyer wrote: post #34

Earlier quoted context omitted.

> Beats Opus 4.7 Max I'm a huge open model fan, and have used them since forever, even have daily drivers for on-prem dev, but no. They do not beat opus on real-world usage. Qwen models are impressively good for what they are, are "good enough" for plenty tasks, can be ran locally on decently priced hardware, and so on. They certainly have their uses, and the field in general has advanced faster than my early expecta…

Yep. These small models are actually worse than GPT 3.5 at some tasks (like recalling facts). You can definitely make models smarter at specific tasks (like tool calling, coding) but you can't compress the entire human knowledge into a 30GB file. It's just not enough bits.

But why would you use a model to store factual knowledge, that is stupid. We want intelligence, not a database.

Re: Qwen 3.8 27B

#244

Post by seanmcdirmid »

expedited123 wrote: post #241
peri-cl wrote: post #175

Earlier quoted context omitted.

Don't mind! It's 64 GiB dual-channel DDR5-6400, i.e. roughly 100 GiB/s of bandwidth. (AMD 7840U (Zen 4)) I'm using a Q4 quantization from unsloth (Qwen3.6-35B-A3B-UD-Q4_K_XL). It gets up to ~20 tokens/second in generation. I don't know precisely how much KV cache I can safely use, but it's in between 140k–256k. (I.e., 140k reliably works, 256k kernel-crashes from OOM. Don't feel like bisecting). Inference is llama.cp…

Thanks! I don't have any knowledge of running models locally. I assume it would not be able to handle an unquantized Qwen3.6-35B or is it irrelevant as you almost always would want to run a quantized version of the model on consumer hardware?

not parent, but 4-bit quantization is generally consider a good trade off for speed/performance, so you might use it even when you aren't on consumer hardware, but definitely when you are on consumer hardware.

Re: Qwen 3.8 27B

#245

Post by alienbaby »

tinfoilhatter wrote: post #196
nubg wrote: post #165

Earlier quoted context omitted.

just to clarify. yes YOU built it. just because you used some tool doesn't mean the idea, prompting, reprompting, babysitting was not your creative input and effort. put differently, if you put a random person infront of whatever model you used (say, a 50yo receptionist at a pharmacy in india), they would not have been able to create that, because they would have lacked the motivation, idea, background knowledge, tas…

So if I hire an artist and am a motivated individual, have an idea for a painting, have background knowledge about paintings and have taste in paintings and can offer a critique of the painting as the artist paints it, then somehow I created the painting? Absurd logic. The AI built the website.

You know how many pieces of art Damien Hurst creates himself Vs his studio assistants creating them under his direction?

For example, of his 1500 spot paintings, he only actually made 5 of them.

It's not uncommon at all for artists to work this way.

Re: Qwen 3.8 27B

#246

Post by satvikpendem »

skrebbel wrote: post #178
satvikpendem wrote: post #161

As usual, the Jinja templates are messed up so use this [0] to reduce or turn off thinking, fix tool calling, keep a 100% KV cache hit rate, etc. [0] https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

I'd love to understand this more. Are you saying the Qwen team spends their very impressive human and compute resources on publishing these amazing models and then botches the chat template with mundane bugs? Like maybe I just misunderstand what's the hard part but wouldn't you assume that people who can put together an impressive model can also write a proper jinja chat template for it?

Yes for the first question. Google of all companies didn't even get it right with Gemma for a while until recently. For some reason it doesn't seem like people can actually get these templates right.

Re: Qwen 3.8 27B

#247

Post by williamcotton »

tinfoilhatter wrote: post #196
nubg wrote: post #165

Earlier quoted context omitted.

just to clarify. yes YOU built it. just because you used some tool doesn't mean the idea, prompting, reprompting, babysitting was not your creative input and effort. put differently, if you put a random person infront of whatever model you used (say, a 50yo receptionist at a pharmacy in india), they would not have been able to create that, because they would have lacked the motivation, idea, background knowledge, tas…

So if I hire an artist and am a motivated individual, have an idea for a painting, have background knowledge about paintings and have taste in paintings and can offer a critique of the painting as the artist paints it, then somehow I created the painting? Absurd logic. The AI built the website.

There have been plenty of workshops where artists hire assistant painters while maintaining authorship over the works themselves, from Rembrandt to Warhol to Hirst.

Re: Qwen 3.8 27B

#248

Post by Balinares »

I wonder if Anthropic and OpenAI possibly missed the window to go public. A 27B open-weight model trading blows with the SOTA from just half a year ago is not great news for trillion-dollar investments...

Re: Qwen 3.8 27B

#249

Post by ferrouswheel »

Foobar8568 wrote: post #67
scrlk wrote: post #8

Beats Opus 4.7 Max (w/ Claude Code) on DeepSWE (42.2 vs 40). Looks like Qwen's 27B models continue to pack some punch. Unsloth's GGUF quants are up: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

Considering the clusterfuck that is opus 5 or even fable, if Qwen 27B is trully better than Opus 4.7 Max, I will rejoice.

Yeah Opus 5 is almost unusable as a daily driver without making me go insane from excessive claude babble.
Post reply on HN