111–120 of 145 posts
Earlier quoted context omitted.
I usually see doom loops when working with quants. Likely theyre trying to maximize the viability of a efficient model quant that can bw upgraded. Like cutting coke to get crack, quantiry over quality.
Nit pick; cutting (adulterating) cocaine doesn't produce crack cocaine.
Earlier quoted context omitted.
I really want to see hard evidence of distillation before I buy into it. Seems like a lot of sour grapes over not having the sort of lead assumed. In this field, it has been shown repeatedly that leaps in performance come swiftly and without notice.
FWIW, the way GLM-5.2 (and 5.3) talk is clearly claude, so it is for sure also trained using distillation. The metric used there is me screaming at my screen per operating hours. Does it matter? IMO not really. Weights are open after all. (Or.. soon at least for 5.3)
I had Ox Alpha working on coding tasks for a couple days non-stop, via OpenRouter and OpenCode Zen. Crush harness. It was able to complete tasks at a level that I'd put between Sonnet and Opus. It makes few mistakes, but is not that smart. The main issue for me, is that it degraded into a doom loop several times. One of them was running the same bash command about a thousand times. The last model I've used that had t…
Releasing weights is the right move. Keeps them competitive with DeepSeek on the open side.
I had good experience with GLM 5.3, but... Z.AI is the only provider for GLM 5.3 on OpenRouter. I don't see 5.3 on Hugging Face. Not sure if this new model is "full GLM" or something smaller, or if they will like Moonshot AI publish weights but put restrictive license [1], which will again leave Z.AI as single GLM model provider on OpenRouter. [1] https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE
Earlier quoted context omitted.
How about proof that black-box distillation can deliver these results without a very sophisticated RL pipeline doing the heavy lifting?
"Black-Box On-Policy Distillation of Large Language Models", Microsoft Research, https://aka.ms/GAD-project > 'GAD consistently surpasses standard sequence-level distillation, delivering superior generalization and achieving performance that rivals the proprietary teacher. These results validate GAD as an effective and robust solution for black-box LLM distillation.' No RL, although I'm a little bit surprised to see…
> We reserve 500 samples of LMSYS-Chat-1M-Clean as the primary test set. We also include test datasets consisting of a 500-sample subset split from Dolly [6], the 252-sample SelfInst dataset [37], and the 80-question Vicuna benchmark [3] to evaluate out-of-distribution generalization. We report the GPT-4o evaluation scores [45, 10], where GPT-4o first generates reference answers and then scores the output of the student model against them. We also conduct human evaluations on the LMSYS-Chat-1M-Clean test set for qualitative assessment.
Funny how all china companies are expected to release weights by default
All smaller models and models behind frontier are expected to be released by default. Otherwise there’s no reason to produce them. Chinese labs are not releasing all of their model weights. Qwen is known as an open weight model by most, but their top model is not open weight. Releasing weights is a marketing strategy for newer labs to get their brand out there.
Which harness are u folks using, I have tried opencode and claude code. Both absolutely keep hanging due to the model running into loops and becoming unavailable. Unable to do even simple things
I gave it an abandoned repo for an Aseprite MCP someone made and told it to iterate with a laundry list of things I wanted from it to include thousands of plugins.
Came back 20 hours later and it shit out a pretty surprising little tool, will post the public repo when I get time.