Open-weight AI is having its Kubernetes moment

While I am grateful for open weights models I never found much use of them in the past, barring those I could run myself. This changed with deepseek 4 - it is staggeringly cheap, even if the performance definitely isn't near sota and it's not particularly fast either.

When I expect to need a lot of tokens and the task isn't too difficult I use sota to plan and create a thorough set of instructions and let deepseek chip away at it. With thorough instructions the quality tends to be satisfactory, and you pay something silly like $15 for 600m tokens.

GLM 5.2 seems like a decent price/perf and Kimi 3 has some real nice performance for an open weights model, but gpt 5.6 is unexpectedly affordable (especially if you don't automatically use Sol at max) so I don't think either is worth it atm. The exception is when you're working on something that US models get cold feet about, which seems like a constantly growing list. For me Fable is already too much of a headache in this regard, but chatgpt is still okay-ish. Hopefully it'll last. If not, there's Kimi.

tldr SOTA for most things because gpt 5.6 is token efficient. If I expect to burn a lot of tokens I use deepseek 4.

Sadly until china scales production of hardware it really isn’t economical to run this stuff yourself. It is good it exists though to put pressure against the labs.

Honestly imo this is just proof apple will win in the end. Eventually a phone will be able to run a model good enough to do most things and it then is game over.

I'm using Kimi K3 + OpenCode. I pay their API pricing, costs about $5 / hour (and chews through ~10 million tokens / hour) during continuous use when I have one or two sessions running and doing their thing.

Can't comment on how it compares to plans (I really don't like the limitations and general shenanigans I see around plans, so I've never tried them).

It is notably slower than Fable / Opus / Gemini, but also vastly cheaper than their API pricing.

Why is this so difficult to understand?

1. the field was nascent and new efficiencies were discovered

2. supply and demand

3. its in the company's incentive to make their models more efficient to increase overall usage so that while the margin remains the same, the total revenue + profit increases

I genuinely don't know what puzzles everyone?

I don’t know if others would find this useful, but previous did have custom harnesses etc.. but tools have improved so much that I drastically simplified.

That said, even the foundational models fail at the hard parts of my code so I use it opportunistically.

I have reduced down to just using zed, will three locally hosted models.

Qwen 3.6 27b on 1x3090 llama.cpp with 128k context ~50tps

Qwen 3.6 35B-A3B on 1x titan v + 2x1080ti llama.cpp with full context ~30tps

GPT-OSS 120b on pure cpu (slow)

I just use zeds parallel agents, task switching, stopping and fixing the code when a model gets stuck.

This still lets me stay engaged, and to modify code to be maintainable etc…

It gets me 80% there and I use to keep a subscription but often times just using googles AI mode is just as good.

That said I have 30 years of experience and insist on knowing how my code works, so this gets me 80% of the short term benefits while not depending on a 3rd party to keep my code moving forward.

Your mileage will vary and 2*5060ti 16gb cards would get around 100/tps with Qwen 3.6 35B-A3B on cards that are widely available.

To be honest the more modern cloud models are using draft tokens etc… that while they are superior for common coding tasks are degrading with more domain specific tasks.

That is just the cost of the draft model being ~10-20% of the foundation models size, and even the biggest Blackwell GPU is limited to ~250/tps so MoE or draft models are required for scaling performance at the foundational level IMHO.

The hard part is my use case are the OOD or small examples in corpus level, the above hurts there.

A Lamborghini may be nice, but I personally need a minivan more.