Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

181–190 of 848 posts

Re: Qwen 3.8 27B

#181

Post by williamcotton »

nubg wrote: post #165
hypfer wrote: post #124

Earlier quoted context omitted.

According to this shitty vibecoded thing "I" built https://hypfer.github.io/will-it-fit-llama-cpp/ (and I guess according to math too), FP16 K/V would give me something like 90k context at the same model quant, which doesn't really fit my usage. But maybe someone else has experience to share there

just to clarify. yes YOU built it. just because you used some tool doesn't mean the idea, prompting, reprompting, babysitting was not your creative input and effort. put differently, if you put a random person infront of whatever model you used (say, a 50yo receptionist at a pharmacy in india), they would not have been able to create that, because they would have lacked the motivation, idea, background knowledge, tas…

I generally agree and expect this to be the case from a legal perspective.

Legal questions of authorship are going to have to be established in terms of doctrines like SSO [0] and AFC [1]. Currently the incredibly sparse caselaw around this has yet to involve such non-literal notions of copyright.

[0] https://en.wikipedia.org/wiki/Structure,_sequence_and_organi...

[1] https://en.wikipedia.org/wiki/Abstraction–filtration–compari...

Re: Qwen 3.8 27B

#182

Post by bilekas »

nubg wrote: post #165
hypfer wrote: post #124

Earlier quoted context omitted.

According to this shitty vibecoded thing "I" built https://hypfer.github.io/will-it-fit-llama-cpp/ (and I guess according to math too), FP16 K/V would give me something like 90k context at the same model quant, which doesn't really fit my usage. But maybe someone else has experience to share there

just to clarify. yes YOU built it. just because you used some tool doesn't mean the idea, prompting, reprompting, babysitting was not your creative input and effort. put differently, if you put a random person infront of whatever model you used (say, a 50yo receptionist at a pharmacy in india), they would not have been able to create that, because they would have lacked the motivation, idea, background knowledge, tas…

You sound like your trying to reassure yourself of something.

I sure hope my boss doesn't think he built my work! He'd probably get fired pretty quickly during on call!

Re: Qwen 3.8 27B

#183

Post by CuriouslyC »

jrflo wrote: post #148
scrlk wrote: post #8

Beats Opus 4.7 Max (w/ Claude Code) on DeepSWE (42.2 vs 40). Looks like Qwen's 27B models continue to pack some punch. Unsloth's GGUF quants are up: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

That kind of result makes me suspicious of benchmaxxing. Qwen 27B is 100x smaller than Opus 4.7. Is it really 100x more parameter-efficient? Two orders of magnitude is hard to believe. I don't have the hardware to run a 27B, but I'm curious what real world use is like. Maybe I'll have to buy some usage on a cloud provider to run my own tests, but this seems fishy to me.

Qwen small models are heavily coding focused, whereas Opus is everything to everybody (even if code is their bread and butter). The downside is they'll frequently hallucinate world knowledge so they need to be RL'd to double check their knowledge against sources and verify facts/library names/etc.

Re: Qwen 3.8 27B

#184

Post by [deleted] »

bilekas wrote: post #182
nubg wrote: post #165

Earlier quoted context omitted.

just to clarify. yes YOU built it. just because you used some tool doesn't mean the idea, prompting, reprompting, babysitting was not your creative input and effort. put differently, if you put a random person infront of whatever model you used (say, a 50yo receptionist at a pharmacy in india), they would not have been able to create that, because they would have lacked the motivation, idea, background knowledge, tas…

You sound like your trying to reassure yourself of something. I sure hope my boss doesn't think he built my work! He'd probably get fired pretty quickly during on call!

[deleted]

Re: Qwen 3.8 27B

#185

Post by ai_fry_ur_brain »

satvikpendem wrote: post #177
ramon156 wrote: post #64

People will claim it's not comparable to Opus despite it beating the score. I'm not sure I disagree, but I'm also unsure whether I care. Most new models nowadays are "good enough". I cannot complain because I'd rather spend that time improving my prompts and docs. Opus might be a _slight bit better_ at picking up vague hints, but it's also extremely expensive, and I hit the 5 hour limit way too quick. I care a lot ab…

You should check out Grok, it's quite a good deal from the Cursor subscription side but it's cheap even by API prices.

[dead]

Re: Qwen 3.8 27B

#186

Post by mips_avatar »

D4Ha wrote: post #133
hypfer wrote: post #109

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…

Do you find it useful or worthwhile to split a large LLM across two GPUs on a desktop? If you've tried it, what worked well and what didn't? I'm especially interested in mismatched VRAM setups, e.g. a 16 GB GPU + a 24 GB GPU. How much overhead did you see from inter-GPU transfers, and did the extra usable VRAM outweigh the performance hit?

Depends on your pcie connection. If they're both x16 then it's pretty low overhead, x8 is ok, but x4 is too slow. Also it's a bit tricky getting an optimal setups with mismatched vram, I think you could probably still make use of the full vram if you're clever but it's trickier.

Re: Qwen 3.8 27B

#187

Post by davidw »

I don't know much about the production of these models. How hard would it be to 'fork' something like this and have it not be full of CCP indoctrination?

Re: Qwen 3.8 27B

#188

Post by SparkyMcUnicorn »

piyh wrote: post #57

Qwen 3.6 is ~$2/m tok, 3.8 should be drop in replacement. Gemma 31B is $0.34/m tok. The price differential on these models is massive on openrouter.

Yeah, I would appreciate if someone could make sense of the pricing differences between these models. How can a provider run DSv4F at lower cost than a 27B dense or 35B A3B model?

Does it come down to utilization and/or specific model tricks and efficiencies (attention, kv cache, etc.)?

DeepInfra prices:

Qwen 3.6 27B: $0.32 in / $3.20 out

Gemma 3 27B: $0.08 in / $0.16 out

DeepSeek V4 Flash 0731: $0.08 in / $0.18 out

Qwen 3.6 35B A3B: $0.10 in / $0.95 out

https://openrouter.ai/qwen/qwen3.6-27b

https://openrouter.ai/google/gemma-3-27b-it

https://openrouter.ai/qwen/qwen3.6-35b-a3b

https://openrouter.ai/deepseek/deepseek-v4-flash-0731

Re: Qwen 3.8 27B

#189

Post by bilekas »

D4Ha wrote: post #133
hypfer wrote: post #109

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…

Do you find it useful or worthwhile to split a large LLM across two GPUs on a desktop? If you've tried it, what worked well and what didn't? I'm especially interested in mismatched VRAM setups, e.g. a 16 GB GPU + a 24 GB GPU. How much overhead did you see from inter-GPU transfers, and did the extra usable VRAM outweigh the performance hit?

I haven't tried this either but I'm guessing if you could pool the GPU memory over whatever the kids are using these days, I think it was SLI back in my day. The GPU memory should still be faster than the RAM?

Re: Qwen 3.8 27B

#190

Post by LeBit »

Almondsetat wrote: post #132

The $1500 Intel B70 with 32GB of VRAM can run this model at max context with good performance, btw. If you don't want to drop $5-10k for running DeepSeek this is your best budget option for local refactor/small scale dev help

I understand the B70 is a bargain vs AMD and especially nVidia offerings, but to me it feels like I would be buying something that would feel too limited in less than a year. 48G would be much more confortable.

And I know the 96G nVidia cards are selling for over 10k$.

The future can’t arrive fast enough!

Post reply on HN