Unsloth Q4_K_M on a single 3090, llama.cpp "Generate an SVG of a pelican riding a bicycle" first try https://www.reddit.com/r/LocalLLaMA/comments/1voa3ch/comment...
Most people cannot draw a bicycle that well!
Qwen 3.8 27B
261–270 of 848 posts
Re: Qwen 3.8 27B
#262Earlier quoted context omitted.
What kind of things you only get with a larger model?
Similar to the way they asked Sol to solve Erdos problems, that's what I want my model to do for programming. I don't want to try to take my best educated guess at what the best design is BEFORE implementation - especially if you're designing a feature for a codebase you're not an expert in, you don't know like the back of your hand (i.e. one that is mostly or entirely LLM generated). What sounds good on paper - ofte…
You're going to be waiting for a while.
Even Fable is bad at this, I would constantly have to fix it going down architectural dead ends or just making obvious mistakes.
Which sucks for people that want LLMs to do everything like a genie, but does mean senior engineers have a few more years before they become redundant.
Re: Qwen 3.8 27B
#263Earlier quoted context omitted.
Do you find it useful or worthwhile to split a large LLM across two GPUs on a desktop? If you've tried it, what worked well and what didn't? I'm especially interested in mismatched VRAM setups, e.g. a 16 GB GPU + a 24 GB GPU. How much overhead did you see from inter-GPU transfers, and did the extra usable VRAM outweigh the performance hit?
Depends on your pcie connection. If they're both x16 then it's pretty low overhead, x8 is ok, but x4 is too slow. Also it's a bit tricky getting an optimal setups with mismatched vram, I think you could probably still make use of the full vram if you're clever but it's trickier.
[deleted]
Re: Qwen 3.8 27B
#264[deleted]
Re: Qwen 3.8 27B
#265Earlier quoted context omitted.
> Beats Opus 4.7 Max I'm a huge open model fan, and have used them since forever, even have daily drivers for on-prem dev, but no. They do not beat opus on real-world usage. Qwen models are impressively good for what they are, are "good enough" for plenty tasks, can be ran locally on decently priced hardware, and so on. They certainly have their uses, and the field in general has advanced faster than my early expecta…
> They do not beat opus on real-world usage We have an internal eval that measures performance on tasks for a handful of embedded systems repos for our mmWave radios (mostly Rust, some C for microcontroller stuff). Qwen3.6-27B scores only 4% lower for pass@1, n=250 compared to Opus-4.8. For the labeled dataset, the average PR size they're being measured against is around 1.5k SLOC. This is very much "real-world usage…
[dead]
Re: Qwen 3.8 27B
#266Earlier quoted context omitted.
[flagged]
Yeah I'm sure everyone on r/cursor or in previous HN threads about Grok 4.5 or 4.6 are all unserious and insane. No one actually cares about the politics as long as the model codes well. Edit, quite interesting to see the reception to this comment compared to essentially the same type of comment I made on a Grok 4.6 benchmark HN post: https://news.ycombinator.com/item?id=49275385#49275571 It's true that Cursor gives…
This is the “Mussolini made the trains run on time” of ai hot takes.
(Btw, mussolini didn’t make the trains run on time)
Re: Qwen 3.8 27B
#267Earlier quoted context omitted.
Depends on your pcie connection. If they're both x16 then it's pretty low overhead, x8 is ok, but x4 is too slow. Also it's a bit tricky getting an optimal setups with mismatched vram, I think you could probably still make use of the full vram if you're clever but it's trickier.
for layer parallelism (e.g. to get more vram) the bandwidth between layers is essentially nothing (like 16kb per token I think), so I don't think x4 would even be a problem!
Good point. It's much more of an issue when running dense models with tensor parallelism. In that case, I'd look for an MoE model instead.
Re: Qwen 3.8 27B
#268Why is anyone even using video cards these days? You may as well be burning cash. This is the perfect candidate for just splattering it on your nvme and then reading it off there and into memory. All of these run perfectly fine on simple m4 silicone: https://github.com/drumih/turbo-fieldfare https://github.com/leonickson1/Swiftlet https://github.com/sqliteai/warp
Lol, "burn money on apple hardware instead!"
Re: Qwen 3.8 27B
#269The $1500 Intel B70 with 32GB of VRAM can run this model at max context with good performance, btw. If you don't want to drop $5-10k for running DeepSeek this is your best budget option for local refactor/small scale dev help
You don't need $10k to run DeepSeek, I run it on a $1000 system.
Could you elaborate please? Genuinely interested
Re: Qwen 3.8 27B
#270Qwen 3.6 is ~$2/m tok, 3.8 should be drop in replacement. Gemma 31B is $0.34/m tok. The price differential on these models is massive on openrouter.
Where do you see that? From what I can see on Open Router, Qwen 3.6 27B (the closest dense equivalent to Gemma 31) is $0.28/m. Am I missing something? https://openrouter.ai/qwen/qwen3.6-27b
They're comparing Qwen 3.8 Max to Gemma 31B, fundamental mistake.