Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

771–780 of 848 posts

Re: Qwen 3.8 27B

#771

Post by jtrn »

I have been running it on my M5 Mac and was impressed with how well it worked with Pi coder. It can genuinely work as an assistant fully locally. It helps me configure Dockerfiles, fixed a couple of errors in a test Nuxt app, and so forth. Not very fast at 20 tps (8-bit quant for total memory usage around 30 GB), but enough to feel that I have a true local coding buddy.

Then came the cold water shower. The agent kept trying to figure out a Nuxt icon package issue and was working on it. On the positive side, it was making steady and slow progress without getting stuck in doom loops. But after 20 minutes, I decided to test with Luna. So I switched in Pi and asked it to review the problem and fix it. Same session. Thirty seconds later, it was fully fixed. API cost on open router was $0.02, probably most of it due to the inheritance of the previous session.

At that rate, the power consumption for local would be FAR higher than the API cost to solve the task.

I wish it wasn’t so, but the cost per intelligence is just off the charts now with Luna.

Now I am really liking that GLM 5.3 will probably run fine on 4x DGX Spark. If nothing else, the local models are truly usable for basic coding and assistance. I would have been blown away by the support I could have gotten with Qwen 3.8 when I was starting out coding. Hopefully, the local models will catch up AND the hardware becomes affordable in the future. Local models are keeping the largest LLM providers on their toes.

But right now, it does not make economic or capability sense to run locally. It does make privacy, security, and vendor lock prevention sense, though.

Re: Qwen 3.8 27B

#773

Post by kamranjon »

cjbprime wrote: post #714
dexterlagan wrote: post #704

Tested the model briefly with my usual eval: a couple of questions on general knowledge most small models often get wrong, then write a fully-featured todo list app in JS, then rewrite the same app in Rust with Tauri. Granted, most models are well trained on basic todo apps, but it gives me an idea of the basic SWE capabilities I can build on. As far as I'm concerned, if it can successfully setup a local git repo, wr…

> General knowledge: I usually ask 2 questions many small models get wrong: summarize Operation Trojan Horse by John Keel, give publication year. Summarize the Ariel school incident of 1994. Qwen3.8 got the first question right, along correct publication year, gave glorious detail on Keel's theory, but got the second wrong. It thought that school was located in the USA. Ah well. I'm not sure that this means anything.…

I actually disagree that this doesn’t mean anything. I understand the contention that it’s not measuring the quality of the model in general, but I think it is measuring something useful.

A good example of this is planning hardware projects - a larger 200b plus model like DeepSeek V4 flash will recommend parts like motors, real time clocks, voltage regulators etc and it will do so providing exact model names and specifications.

I wouldn’t expect a smaller model to encode all of this information, but it is helpful to understand where that cutoff is because it changes what the model might be useful for. It is a very crude way of measuring because it comes down to the balance of training data at sizes this small - but I do think it conveys something that is helpful in real world tasks.

Re: Qwen 3.8 27B

#774

Post by monster_truck »

CMay wrote: post #518

Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it. Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks. The VRAM usage see…

In my experience MTP's speed increase doesn't seem to justify the apparent loss of success at the edge, it would have to be at least 4x faster to meaningfully churn through the first 3 failures in the time it would have taken to do it once without

What exactly are you doing that the prompt is eating an entire 65536 window? Surely it would be better to let it use any number of approaches that call tools to access that in parts and reason/summarize to an output file as it works through the whole thing. This allows it to keep the initial instructions in the start of the window and toss out the middle as it goes. IME many people who have written off local models entirely are, for lack of a better term "not holding them right" and consider them worthless.

Definitely also try IQ4_NL for K/V if you haven't. Because it's non linear it's far more hit/miss from model to model and especially quant to quant, I've found generally that it shines brightest when you start with a Q6K+ quant that you otherwise wouldn't bother with because of its size, which it then makes up for in both inference speed and often a larger context.

I do agree about Glimmer, though. It is quite good, far better than the benchmarks let on, especially in heavily agentic cases where it needs to rampage around the OS and utilize many different utilities to zero in on things. It is especially good at being told to try something itself, and if/when it fails, try Qwen, and if Qwen can't do it, call out to Deepseek.

Re: Qwen 3.8 27B

#775

Post by _ache_ »

scirob wrote: post #769

We maintain German Langauge index as no one publishes or reruns these sepeartly. Qwen 3.8 27B is a small improvement with some regressions in our benchmarks not a huge jump like benchmarks listed. https://dach.peerbench.ai/compare?models=qwen%2Fqwen3.8-27b,... German language has never been a big focus for asian models but they still outperform Gemma models https://dach.peerbench.ai/compare?models=openai%2FQwen%2FQwe…

From your benchmark, Qwen3.8 is nearer than Opus 4.8 than Qwen3.6. 0.1pp but still.

Also, a lot of people don't really care about german language capacity, maybe people programming in DDP idk.

PS: You benchmark seems saturated. Most values sit @>75% in a benchmark generally indicate that it's no longer as useful as a <70% one. I mean, Qwen3.8 is 77.5% and Fable5 80%, the poll of values is from 65% to 90%.

Re: Qwen 3.8 27B

#776

Post by apitman »

Almondsetat wrote: post #453
apitman wrote: post #369

Earlier quoted context omitted.

I wonder how this would stack up against 4x RTX 3060, assuming you have the physical room for them.

Since the 3060s would have to communicate constantly using pcie, I think I'd go with intel

Currently software is king when it comes to inference performance though.

Re: Qwen 3.8 27B

#777

Post by Topfi »

arjie wrote: post #597
Topfi wrote: post #382

Earlier quoted context omitted.

Anything from ML pipelines for language specific pruning over a Rust/JS/CSS mix codebase to assistance in motorcycle maintenance and different canvas coatings. Most of my evals build on those requirements and especially past failures, whether in pure information, task execution and coding or tool calling beyond the overfitted mainstream. All stuff derived from actual failures encountered, some still only few models c…

Most of us aren’t deploying these in general purpose use-cases. E.g. I use Qwen mostly for vision in my personal assistant. I have an eval set for that. Pretty much each of my use cases has a pre-computed problem set. Likewise I have a tool-use eval set and a browser-use eval set. I use frontier models for most interactive tasks that are not home assistant or background agents. I can imagine someone could build evals…

Deepseek v4 Pro or GLM 5.3 for software architecture I feel are only deployed for general, rather doubtful those are for narrow stuff, if we are sticking with this threads mentioned models. For small models, sure, narrowly targeted sets which can be self evaluated are amazingly valuable, but I feel beyond 500B we are in a different dimension. Rating any model the size of GLM or V4 Pro in hours I doubt is done beyond pure vibes.

Re: Qwen 3.8 27B

#778

Post by maldie »

simonw wrote: post #419

Absolutely the best pelican I've seen from a model that runs on my laptop: https://tools.simonwillison.net/markdown-svg-renderer#url=ht... Bicycle is the right shape. Pelican beak is excellent. Nice background. Most importantly, the pelican has one leg on each side of the bicycle - that's very rare. (No chain on this bicycle though - in the reasoning trace it says "already chainstay... skip chain detail; maybe a smal…

Even with "low" thinking effort and Q4 it makes reasonable pelican, only 4k tokens and half a minute on 4090: https://imgur.com/a/yd5rNCp

Re: Qwen 3.8 27B

#779

Post by HarHarVeryFunny »

awb wrote: post #596
svdr wrote: post #562

Wow. This model is so good, and we have GLM 5.3 (seems great voor security related work) and Deepseek. In a few months we'll have Fable/Sol-like capabilities that are not coming from the big US companies. I feel as a programmer that that is more than enough. How wil OpenAI and Anthropic survive when frontier model intelligence becomes commoditized?

The demand curve for speed and intelligence seems pretty steep to me. If you look at the hiring marketplace, being just marginally better than your peers can be very lucrative. If you’re competing on speed or capability as a company (or as an employee), you’re probably going to be willing to pay for the frontier.

> If you look at the hiring marketplace, being just marginally better than your peers can be very lucrative.

I would say that in software this is completely false.

Someone straight out of college, not very useful, makes 75-100K.

Top level senior outside of FAANG is making twice that at best (and at least 10x more capable).

Re: Qwen 3.8 27B

#780

Post by sgt »

wincy wrote: post #579
kimsey0 wrote: post #571

If anyone else is running this on an RTX 5090, https://github.com/Neroued/ninfer as inference engine gets me ~138 tokens/second, roughly double what I get with a naive llama.cpp setup.

Amazingly enough, Hacker News decided to show this to me as the top comment, I'm also running an rtx 5090 and trying it out now, thanks for the tip! Edit: Absolutely blazing fast! Getting 163 tokens/sec on WSL and it generated a pretty sweet Pelican. https://gist.github.com/hansale/ed9e73fe35165a58ea2af6b1632a...

Amazing. I'll give it a shot on my 5090. I already tried using vLLM but it ran out of GPU memory. I guess it's likely Llama.cpp will work.
Post reply on HN