Qwen 3.6 27B is the sweet spot for local development • Hacker News Board

Re: Qwen 3.6 27B is the sweet spot for local development

#651

Post by sleepyeldrazi » Tue, Jun 30, 2026, 9:35 AM UTC

danilocesar wrote: post #225

sleepyeldrazi wrote: post #137

Earlier quoted context omitted.

I can't speak for the US, but in Germany (where hardware is usually more expensive, not less), I got my 3090 3 months ago for 750 euro and have been running the iq4_nl 27B using q4 kv (which after recent patches in llama.cpp is in my xp indistinguishably accurate from q8 of f16) at full ctx, with MTP at 2, peaking around 70 t/s on small ctx, around 50 t/s when im around 64k and ends around 40 t/s near the cap. The re…

I can't find a 3090 for less than 2k CADs (or 1200 eur). Is this the average price in Germany? It's pretty cheap.

I got it off kleinanzeigen, its a ebay-like site (but mostly 'pick it up yourself' instead of delivery). Looking at it right now, i do see multiple sales for 850-900. I did spot the 750 one after frequenting the site for a week or two, so it may be a bit of a 'better than average' deal, and it seems most are in the 1k euro range, but there are a handful available under.

As of writing this, it shows 24 offers between 700 and 950.

Re: Qwen 3.6 27B is the sweet spot for local development

#652

Post by paintbox » Tue, Jun 30, 2026, 9:42 AM UTC

meta-level wrote: post #648

why does everyone imply you need a $10k laptop which then starts burning when you run Qwen 3.6? Get any other system with enough VRAM for a third of the price. Framework Desktop (Strix Halo 128GB) still costs under 4k nowadays, is nearly silent even on 100% GPU + CPU. (also it gets only slightly 'warm', but with a desktop you don't care anyway, I guess).

But how will I signal my status to other people then?

On a serious note, I run my models on desktop pc, simple api and i can use them wherever whenever.

Re: Qwen 3.6 27B is the sweet spot for local development

#653

Post by boppo1 » Tue, Jun 30, 2026, 9:50 AM UTC

SchemaLoad wrote: post #494

mashygpig wrote: post #455

It's fun to run a model locally, but I don't think the economics make sense for anyone just trying to use models atm. It's absurdly cheap to use the same model via openrouter in comparison. Seriously, just put $10 into openrouter and play with models that are cheap but bigger than what you'd reasonably be able to run locally like deepseek v4 flash (unquantized). You'll be surprised by how far that $10 goes for a mode…

Agreed, I'm waiting for the time when 48GB+ ram is just the standard that computers come with rather than being the absolute top tier option. It just doesn't make sense to spend extra on a local AI computer right now when the same money would last for a decade of API pricing.

Have you considered this may never happen? What if datacenters continue to swallow all capacity?

Re: Qwen 3.6 27B is the sweet spot for local development

#654

Post by LoganDark » Tue, Jun 30, 2026, 9:51 AM UTC

I see OpenCode mentioned in the article, and I would strongly warn against using it for local development because it disrespects caching (the content of the first turn / system prompt is NOT stable). I use Pi which works much better.

Re: Qwen 3.6 27B is the sweet spot for local development

#655

Post by throw1234567891 » Tue, Jun 30, 2026, 9:52 AM UTC

wahnfrieden wrote: post #168

throw1234567891 wrote: post #125

Earlier quoted context omitted.

But the tokens or credits are gone. MacBook stays. You can run other models on the same MacBook. What I read people burn every month on saas… for that money you break even on that MacBook in 5 months. Edit: it’s not just “data privacy”, when you are using Claude, you are shipping EVERYTHING to Anthropic. It’s crazy.

It's much slower, and often quantized

Okay, and?

Re: Qwen 3.6 27B is the sweet spot for local development

#656

Post by bfjvibybd6cuvu6 » Tue, Jun 30, 2026, 9:53 AM UTC

iagooar wrote: post #624

astrostl wrote: post #474

Earlier quoted context omitted.

> MacBook Pro M5 128GB RAM 614 GB/s of memory bandwidth > MacMini M4 with 64GB of RAM 273 GB/s of memory bandwidth (also only currently available with 48GB) When it comes to inference speed, you want your model to fit in memory, and then to have as much memory bandwidth as possible. In this case a hypothetical Mini with 1TB of memory would still be over 2x slower with 27-35B models. And FWIW I have an M4 Max MBP 128G…

On paper the M4 should be roughly 1/3 of the M5, in practice it is only 1/2. With the right, optimized model like qwen3.6 35B MoE MLX you can get over 40 tok / sec on it. I run dozens of background jobs that are not time-critical on it.

What kind of jobs?

Re: Qwen 3.6 27B is the sweet spot for local development

#657

Post by boppo1 » Tue, Jun 30, 2026, 9:57 AM UTC

accrual wrote: post #54

beastman82 wrote: post #43

FWIW I'm running gemma4 31b on my 5090 and it's pretty great as well. QAT, MTP, 128k context. I liked Qwen 3.6 27b too, it just seems that Gemma4 is a bit underrated.

Nice. I flip flop between Qwen 3.5 9B Q6_M and Gemma4 12B Q4_K_M on a 4080 Super. They run at about the same speed and I can have them review each other's plan or diffs. For smaller projects I find them very capable, and I can step up to a better quant for slightly more challenging work.

Have you tried qwen 27b q4_K_XL? It's a little bigger than the 4080 but not too much

Re: Qwen 3.6 27B is the sweet spot for local development

#658

Post by ThunderSizzle » Tue, Jun 30, 2026, 10:15 AM UTC

oceanplexian wrote: post #207

iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

If you want to do coding with a local LLM your best bet is a 6 year old Nvidia 3090 which is substantially more powerful than the highest end overhyped Apple product for 1/5th the price.

The cheapest 3090s I could find with any sort of guarantee were pushing $1500.

An AMD AI Pro R9700 32GB brand new is $1350 right now.

After some tweaking, I had it running faster than the models the 3090 could run, and it could obviously run with higher context limits and bigger models due to the extra vram.

Re: Qwen 3.6 27B is the sweet spot for local development

#659

Post by ThunderSizzle » Tue, Jun 30, 2026, 10:28 AM UTC

androiddrew wrote: post #541

Dual AMD Radeon AI Pro 9700s (600 watts total 64GB of vram) runs Qwen 3.6 27B at FP8 with mtp on vLLM at 50ish TPS for decode. Cards cost $1300 a piece. Enough KV cache to fully max out two concurrent sessions. It was super rough going to get started with them back in January, but right now the cards purrrr and I haven't even tried tuning yet. You need to use a patched vLLM image with aiter but besides that things ar…

Agreed. I have a single 9700 and I'm able to fit Q6 27B at 30tps or Q5 35B at 100tps very easily via llamacpp running vulkan.

The results are impressive considering the amount of people trashing AMD and still trying to recommend 3090s. I hope to buy a 2nd one at some point, but I also hate the version hell of vLLM, the R9700, the ROCM version, and Qwen3.6 all not agreeing with each other. I haven't gotten vLLM to run properly for Qwen3.6, since the version that runs on a 9700 doesn't support 3.6 yet.

I'm trying to quickly hack out a optimized path for just Qwen3.6 to run against rocm natively (e.g. my own inference server for 9700s basically) and see if it can perform better than llamacpp vulkan's results.

Word of caution - the last llamacpp with good performance was b9209 from a month ago. After that, for some reason, vulkan performance dropped by 10x, which has made me lose confidence in llamacpp in the long run.

Having said all that, 3x is 96GB for 4k and peak 900 watts. A 96GB Blackwell is $12k and peak 600 watss. And they will have a similar memory throughput (minor negative to the AMD cards for split processing). It's crazy how price efficient the r9700 is compared to the Nvidia cards.

Re: Qwen 3.6 27B is the sweet spot for local development

#660

Post by konart » Tue, Jun 30, 2026, 10:30 AM UTC

hypfer wrote: post #629

konart wrote: post #602

>Real work This part should have featured something about real work. But instead it features a paragraph about one-shot bs that creates "something". Unless your work is to create thousands wordpress tremplates to sell - this is not a "real work". Give it a repository (any kind of OSS project will do for an example) and a github issue requesting a knew feature or describing a confirmed bug. (you can and probably shoul…

If your expectation is to treat it as a coworker, then you're right. If your expectation is to treat it as a tool, then you're wrong. I guess that's where the disconnect lies.

Define "a tool" for me and we can talk.

I already have tools for autocomplete, working with structured data and many more. Deterministic tools.

Obviously you do not expect something like that from a model with some harness. It can read some input (user's or other tools) and give you some output.

My expectation is that this tool, given some meaning full input (instructions, expectations, motivations and an optional source files to work with), will produce something that will actually be aligned with the input.

For example: consider I have a services that has some sort of events created now and then. I what those events to be available for other services. So I decide it to have a transactional outbox and an observer that will pull events from the outbox and put them into a kafka topic.

My expectation is that I can give this tool some context (source code and description), state my instructions, expectations, motivations, design decisions and have an implementation as a result.

My other expectation is that given my context etc and agent's context (skills etc) were correct and adequate - the outout will also be correct and adequate.