Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

431–440 of 848 posts

Re: Qwen 3.8 27B

#431

Post by theplumber »

simonw wrote: post #421
simonw wrote: post #419

Absolutely the best pelican I've seen from a model that runs on my laptop: https://tools.simonwillison.net/markdown-svg-renderer#url=ht... Bicycle is the right shape. Pelican beak is excellent. Nice background. Most importantly, the pelican has one leg on each side of the bicycle - that's very rare. (No chain on this bicycle though - in the reasoning trace it says "already chainstay... skip chain detail; maybe a smal…

For anyone who followed yesterday's Gemini 3.7 Flash pelican which rendered in Safari but not in Firefox or Chrome... https://news.ycombinator.com/item?id=49289112#49290012 . - that turned out to be my fault, not the model. My SVG rendering software was stripping some attributes. Here's the Gemini 3.7 Flash pelicans in the fixed renderer: https://tools.simonwillison.net/markdown-svg-renderer.html#u...

Gemini is worse

Re: Qwen 3.8 27B

#432

Post by altruios »

jLaForest wrote: post #317
satvikpendem wrote: post #242

Earlier quoted context omitted.

Yeah I'm sure everyone on r/cursor or in previous HN threads about Grok 4.5 or 4.6 are all unserious and insane. No one actually cares about the politics as long as the model codes well. Edit, quite interesting to see the reception to this comment compared to essentially the same type of comment I made on a Grok 4.6 benchmark HN post: https://news.ycombinator.com/item?id=49275385#49275571 It's true that Cursor gives…

i care about not financially supporting a person that is actively trying to disenfranchise me, why is that a difficult concept for some people? that not everyone is motivated exclusively by financial profit? is moral bankruptcy so pervasive that some people assume it is unanimous?

'They' - in this case - are those who are unable to empathize. People who only learn of other's struggles through visceral personal experience. Conservatives, in general, struggle with this class reasoning... Very much a 'if I don't experience it then it doesn't exist' kind of mentality. Frustrating and exhausting. A lack of imagination, a lack of empathy, they just are lacking in these regards.

Re: Qwen 3.8 27B

#433

Post by seanmcdirmid »

I'm struggling to figure out what to use this for. From the intelligence benchmarks in OMLX. If only they would release another MoE model.

  Intelligence Benchmark Comparison

  --- Detail ---

  Model: scottlowry--Qwen3.8-27B-oQ4e-mtp
  Benchmark         Accuracy   Correct   Total   Time(s)   Think
  --------------------------------------------------------------
  GSM8K                93.3%        28      30       282      No
  MATHQA               46.7%        14      30      26.3      No
  HUMANEVAL            96.7%        29      30     156.5      No
  MBPP                 83.3%        25      30      71.5      No
  LIVECODEBENCH        43.3%        13      30    1040.4      No

  Model: stamsam--Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-MLX-oQ4-MTP
  Benchmark         Accuracy   Correct   Total   Time(s)   Think
  --------------------------------------------------------------
  GSM8K                96.7%        29      30      51.9      No
  MATHQA               60.0%        18      30       9.1      No
  HUMANEVAL            83.3%        25      30      82.9      No
  MBPP                 80.0%        24      30      29.6      No
  LIVECODEBENCH        36.7%        11      30     283.7      No

Re: Qwen 3.8 27B

#434

Post by mlsu »

kristjansson wrote: post #204
ramon156 wrote: post #69

Earlier quoted context omitted.

hence the "former deepseek v4 pro". I tried it out this morning and have had no complaints. I already liked glm 5.2

> Deepseek v4 pro 0813 Which itself released yesterday? You're writing, reading, and evaluating enough software in a ~36 hour period to form, reject, and form another opinion about which model makes better _architectural_ choices?

He's 100x, bro. 36 hours is basically like a whole 2 week sprint for you.

Re: Qwen 3.8 27B

#435

Post by tommica »

tyfon wrote: post #363
hypfer wrote: post #109

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…

To add for me (and claude assisted but tuned myself), here is for 3060 12gb + 5950x 128 gb ram without image support: https://pastebin.com/HsAUBGSY Here is the chat template I used (and renamed to qwen.jinja): https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/r... For image support, drop to NGL 24 and add: --mmproj-url https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/main/mmproj-F16.gguf \ --no-mmproj…

Thanks for this! I got the same card, but not the same CPU nor RAM, but this is a great starting point to start tweaking

Re: Qwen 3.8 27B

#436

Post by stusmall »

hypfer wrote: post #68
ramon156 wrote: post #64

People will claim it's not comparable to Opus despite it beating the score. I'm not sure I disagree, but I'm also unsure whether I care. Most new models nowadays are "good enough". I cannot complain because I'd rather spend that time improving my prompts and docs. Opus might be a _slight bit better_ at picking up vague hints, but it's also extremely expensive, and I hit the 5 hour limit way too quick. I care a lot ab…

> I've settled on GLM-5.3 (formerly Deepseek v4 pro 0813) for architecting Dude, GLM-5.3 released _today_. The phrasing "I've settled on" is incorrect for this context.

I'm generally extremely skeptical about a lot of the model hype that show up in comments. Except when there is an extreme mismatch the performance, quirks and quality of these things are difficult to nail down. You wouldn't know that from the comment section of every single release.

I think some of these are excited, eager users always ready to hype up the new thing. The same crowd that previously would constantly push for a rewrite from angular->react->svelt->god knows what. Instead now it is on a 6 week cycle and about models/harnesses.

I think most of it is bot driven spam by the various labs. It's hard not to notice 3 month old accounts with very strong opinions about various frontier labs and little else.

Ultimately, I think some of it is legitimate shifts in who's in lead and what is the best. You gotta dig through a lot of crap to get to that, and I don't really know how to to.

Ultimately I'm saying is that I always applied a fair amount of skepticism about what I see in comment sections but these day it is extreme amounts.

Re: Qwen 3.8 27B

#437

Post by jedbrooke »

prometheus1992 wrote: post #402
jedbrooke wrote: post #28

I hope the bonsai team makes another 1bit quant of this model (or releases code/instructions on how to do it), using the Qwen3.6 27B on my 16GB mac mini has been wild . The 1bit quant feels like opus level… for the first couple turns. Then it has trouble eg switching from plan mode to act mode. This is mostly mitigated by starting a new session. (tbf this limitation is called out on the hf page) I saw unsloth has 1bi…

16GB mac mini - what chip? m4 pro i assume?

nope just my normal m2 mac mini. I bought it just as a normal computer to do my taxes and whatever, so it’s mind blowing that I can run this kind of AI workload on it. Well, “run” might be generous, it gets like 3-5tok/s

I’m working on a setup that’s more geared towards running tasks overnight so the slow tok/s doesn’t matter as much

Re: Qwen 3.8 27B

#438

Post by cmrdporcupine »

colordrops wrote: post #278
KronisLV wrote: post #10

I hope really badly that we'll get a new 35B A3B or similar MoE model! I also miss the Qwen 3 Coder Next, which was 80B A3B, there are quite a few use cases where a non-dense model Also alternate link for viewing the images without signing in: https://xcancel.com/Alibaba_Qwen/status/2088280182356611304

I'm still confused about Qwen 3.6 35B A3B. Everything I read said that the 27B model performs better at coding tasks, so what's the purpose of the 35B model?

MoE models have less active parameters in play at any particular moment, so they perform much faster on lower bandwidth memories (like Strix Halo or DGX Spark) and also use less memory generally.

Dedicated GPU memory tends to be much faster (either DDR6 or HBM). So if you can fit a whole dense model in it, you're probably better off with that.

Re: Qwen 3.8 27B

#439

Post by satvikpendem »

apitman wrote: post #366
satvikpendem wrote: post #161

As usual, the Jinja templates are messed up so use this [0] to reduce or turn off thinking, fix tool calling, keep a 100% KV cache hit rate, etc. [0] https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

Interesting. Why don't the unsloth guides ( https://unsloth.ai/docs/models/qwen3.8 ) mention this? Do they already include the fixes in their GGUFs?

They usually do include fixes yes.

Re: Qwen 3.8 27B

#440

Post by jacquesm »

amluto wrote: post #391
augusto-moura wrote: post #368

Earlier quoted context omitted.

IMO AI companies are selling the infrastructure and easier access to the models, beyond competing to get the smarter model. We can run Qwen locally, but at what cost? A good enough GPU for running 27B is more than 4,000 USD where I live (Brazil). Even if I were to spend 100s of hours in credits I would never reach a point where the 4000 USD investment pays off. Unless I'm doing it for the sport, building my own LLM r…

> RTX 3090 for each and every employee That’s quite inefficient. Get 2x RTX 6000 Pro and run DeepSeek v4 Flash for the whole company. This may or may not be cost effective compared to cheap third party providers, but it lets you own your own data, is not actually that expensive, and reports suggest that you can get 2k tok/s or so from a setup like this (at respectable batch size, but that’s exactly what you want in t…

Depending on your compliance needs this may be the only way.
Post reply on HN