Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

351–360 of 848 posts

Re: Qwen 3.8 27B

#351

Post by alyandon »

xscott wrote: post #300
alyandon wrote: post #58

Earlier quoted context omitted.

There is a down to earth guy on YT that performs a series of tests against LLMs running on non-god-tier commodity hardware. He will likely be testing this soon enough. https://www.youtube.com/@lukesdevlab I don't know if that is what you are looking for or not and as always your experiences may be different.

So much potential for that channel. He's got a nice range of tests and a no nonsense presentation style. However, watching tests of heavily quantized models that weren't designed for it (non-QAT) is frustrating. There's no way to tell if the actual model fails because it's dumb or if the lobotomy made it that way.

I noticed he does pay attention to feedback on his videos and I think some people have pointed that out.

Re: Qwen 3.8 27B

#352

Post by apitman »

XCSme wrote: post #348
swalsh wrote: post #338

WOW, my first try running on my 2 3090's, it was a bit slow... but it FEELS like opus 4.5, i gave it an image and a broad overview of what I wanted it to build, and it built the whole thing from beginning to end.

Why slow? I see ~50tps on a single 3090

Yeah make sure you're using MTP and potentially tensor parallelism.

Re: Qwen 3.8 27B

#353

Post by secretluver »

jacquesm wrote: post #340
hypfer wrote: post #109

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…

I absolutely love this comment. I wished there was a website where people would post their working command lines as well as what hardware they are using to run that stuff on + tokens / sec prefill + gen.

[dead]

Re: Qwen 3.8 27B

#354

Post by alentred »

CapsAdmin wrote: post #336
alentred wrote: post #330

Earlier quoted context omitted.

Oh, goody goody goody! Can't wait for a Q4M with *MTP* support. Does anyone know if there are plans for it? I am currently using Qwen 3.6 on RTX 3090 and I have to admit that without MTP it would be too slow to be acceptable for me (30-35 tok/sec without MTP, 60-70 with MTP). Without MTP I would just use OpenRouter and rather pay for speed despite having a capable local setup.

It comes with MTP already, they just don't mention it in the filename as it's now the default.

Oh, thank you for telling me this. Trying it out now.

Re: Qwen 3.8 27B

#355

Post by apitman »

jhonof wrote: post #343

Yeah this is the first model I have been able to run locally that actually feels useful, this is unreal I am considering cancelling my claude sub and going to just api (maybe GLM?) for really hard tasks.

Check out OpenCode Go as well. They give some Kimi K3, Qwen3.8 Max, and GLM5.2 (probably 5.3 soon?) usage which may cover your needs for $10/mo

Re: Qwen 3.8 27B

#356

Post by hypfer »

jacquesm wrote: post #340
hypfer wrote: post #109

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…

I absolutely love this comment. I wished there was a website where people would post their working command lines as well as what hardware they are using to run that stuff on + tokens / sec prefill + gen.

Problem with that is I think that it quickly devolves into cargo culting, nonsense and noise.

Arguably, what I am doing is also very very close to that, with the only difference being that I am somewhat less of an idiot than the average internet dweller you'd get on such a site. Or rather a different flavor of idiot.

Ideally, the people building the tools build them in a way that just does the right thing - which I am confident that llama.cpp does or will do in the future.

So you encode that knowledge not in language and online comments but in code and with a filter for actual expertise.

And, frankly, there's really not all that much to it. It's like maybe 3 parameters to play around with.

The valid solution space is pretty small, but people will want to make it "theirs" regardless, so you get non-solutions just so that everyone could also be a part of it. The usual social dynamics foo.

Re: Qwen 3.8 27B

#357

Post by Almondsetat »

kamranjon wrote: post #315
LeBit wrote: post #190

Earlier quoted context omitted.

I understand the B70 is a bargain vs AMD and especially nVidia offerings, but to me it feels like I would be buying something that would feel too limited in less than a year. 48G would be much more confortable. And I know the 96G nVidia cards are selling for over 10k$. The future can’t arrive fast enough!

You can buy two b60s for $1300 right now (650 each) if you want a total of 48gb. Intel recently raised the price on all of their gpu's except the b60 series, so they are currently the best deal per gb I think.

They have terrible cross connectivity, it's not wise to share a model between them

Re: Qwen 3.8 27B

#358

Post by jhonof »

apitman wrote: post #355
jhonof wrote: post #343

Yeah this is the first model I have been able to run locally that actually feels useful, this is unreal I am considering cancelling my claude sub and going to just api (maybe GLM?) for really hard tasks.

Check out OpenCode Go as well. They give some Kimi K3, Qwen3.8 Max, and GLM5.2 (probably 5.3 soon?) usage which may cover your needs for $10/mo

Yeah I was thinking open router but I will look around at options, I genuinely think this model is good enough for like 90+% of my use cases, and the top frontier models are still not that good at architecture so I have to do that myself still so I won't be losing out.

Re: Qwen 3.8 27B

#359

Post by mordae »

SparkyMcUnicorn wrote: post #302
mordae wrote: post #258

Earlier quoted context omitted.

DeepSeek V4 Flash is natively FP4 MoE with very compact KV cache. Say 8 GB/s. Qwen 27B is about 60 GB/s at full FP16 precision.

If active weight bandwidth is everything, then wouldn't Qwen 35b a3b (FP8) be roughly half the cost of dsv4f (FP4)? dsv4f appears bigger and more memory/computationally expensive to run any way I look at it.

Yeah, it should be basically free. No idea why it is not. I guess KV cache taking up RAM and possibly bad business sense or amortized engineering costs, I honestly do not know.

Re: Qwen 3.8 27B

#360

Post by freshnode »

satvikpendem wrote: post #242
altruios wrote: post #235

Earlier quoted context omitted.

[flagged]

Yeah I'm sure everyone on r/cursor or in previous HN threads about Grok 4.5 or 4.6 are all unserious and insane. No one actually cares about the politics as long as the model codes well. Edit, quite interesting to see the reception to this comment compared to essentially the same type of comment I made on a Grok 4.6 benchmark HN post: https://news.ycombinator.com/item?id=49275385#49275571 It's true that Cursor gives…

Voting with your wallet is still very much a valid way to protest that odious man.

Some people might not mind (or even know), but I sleep better at night trying to work as ethically as I can.

Post reply on HN