These style of comparisons are decent at showing capability but they don't really show me what I truly want - a sounding board and implementer with senior engineer-level execution. When I look back at all the teams that I've been part of, the best outcomes came from white-boarding (sometimes in the metaphorical sense) with one or two people, at times arguing, then finally compromising on a plan. Instead of synthetic benchmarks that try to be objective, I wonder if there's a way test this, or maybe I'm opining on a way of working that will soon be gone?
GLM 5.2 vs. Opus
211–220 of 367 posts
Re: GLM 5.2 vs. Opus
#212I seriously dont' know all this big hullabaloo about one shot prompting. by definition, a single prompt wont' constitute the complexity of a software project. ergo, what you'll get is a series of assumptions made by the model based on preexisting code in its training corpus. I'd rather see a coding agent that can follow steps in a plan file to a T while following guardrails and adhering to the proper coding conventio…
The streetlight effect: > A policeman sees a drunk man searching for something under a streetlight and asks what the drunk has lost. He says he lost his keys and they both look under the streetlight together. After a few minutes the policeman asks if he is sure he lost them here, and the drunk replies, no, and that he lost them in the park. The policeman asks why he is searching here, and the drunk replies, "this is…
The minute an open model breaks through and beats Claude Opus/Fable, it's over.
There are far more opportunities that can be served when the world's intellectuals have the raw weights and can fine tune, splice, distill, and reapply.
Imagine having raw unfettered access to Fable. It can be refit to structural biology. It can be fine tuned on the repo for smaller context requirements. It can be run cheaper and air gapped.
The world wants this.
Re: GLM 5.2 vs. Opus
#213No one has really talked about hybrid and using Opus to plan and orchestrate GLMs work both through initial build and code reviews. That’s a true best of both worlds and there doesn’t need to be a winner.
This is the way but Anthropic doesn’t make it easy, so I use GPT 5.5 in that role since I can use my subscription in OpenCode or OMP.
I also use MiniMax-M3 in utility roles like explore/library tasks.
I’ve had a z.ai subscription for several months so I’m on the older pricing. I’m really not sure it would make sense to do this at current rates - I could bump my Codex plan instead.
Re: GLM 5.2 vs. Opus
#214Earlier quoted context omitted.
I disagree fundamentally. If I do that, I'm literally slower then just doing the change without sufficiently specifying it to the model. I can see how a junior dev or generally someone that's not particularly knowledgeable about the language or framework they're working with may benefit from such usage, but for experienced people there is very little value in that approach. I say this because I've just had to face th…
I disagree, fundamentally . I see little value in throwing a ton of context at an llm and waiting 10-20 minutes for a coin flip on whether or not its going to produce junk. I'd rather do quick 60 second turns, get most of the way there and fix the rest myself if I have to. I'd rather honestly just not use them.
The trick is to do something else in those 20 minutes (or, ideally, even longer).
That's the main value I've been getting out of coding agents. I have them do (comparatively) simpler tasks or explorative tasks in the background while I'm in a meeting, doing code reviews, or otherwise working on something else.
Re: GLM 5.2 vs. Opus
#215Earlier quoted context omitted.
Its also nice that you can see its entire reasoning trace. I can see it going off the rails - or see something I forgot to tell it - and stop and correct it. Or I'll learn WHY it made the choice it did and not have to question it after.
Strong agree! I deeply appreciate this aspect of GLM. Watching it think & being able to nudge early is incredibly useful. Being able to point at bad assumptions is incredibly useful. Watching what it's seeing is super informative. It's always a shock to me how opaque most other models are! It also is pretty resilience to letting you inject in while it's working without going off course or while getting back on track…
> It's always a shock to me how opaque most other models are!
This is (unfortunately) by design. The proprietary models hide their reasoning traces so they can't be used for model distillation. Sometimes even when they do show reasoning, it isn't the model's real trace - IIRC, someone was able to demonstrate that Opus' reasoning is usually a summary made with Haiku behind the scenes.
Re: GLM 5.2 vs. Opus
#216No one has really talked about hybrid and using Opus to plan and orchestrate GLMs work both through initial build and code reviews. That’s a true best of both worlds and there doesn’t need to be a winner.
I mostly use Opus for skill development. Once I have a solid skill implementation with a good eval, I move ongoing execution to a cheaper model running under Goose. With the eval you can see if the cheaper model works well enough.
Re: GLM 5.2 vs. Opus
#217Earlier quoted context omitted.
What? It is less than 20% of the cost of Opus at API rates. 1.40/4.40 vs 5/25.
Not when you factory in token efficiency. It burns a lot more tokens to do the same job, so when I compared to GPT5.5 I was frankly not really much ahead, and with weaker thinking. Maybe makes sense if you have z.AI's (not greatly priced) subscription plan, but it's not competitive against an OpenAI or Anthropic monthly coding subscription plan. I burned through almost $10 worth of tokens just doing an hour of work.
Take a look at Ollama Cloud: https://ollama.com/pricing
You get access to a whole bunch of bleeding edge open models including GLM-5.2, Kimi K2.7, DeepSeek 4 Pro, etc. Inference is run on US/SG/EU cloud providers with zero data retention policies. The $20/mo tier is very generous, in my experience.
Re: GLM 5.2 vs. Opus
#218Earlier quoted context omitted.
The streetlight effect: > A policeman sees a drunk man searching for something under a streetlight and asks what the drunk has lost. He says he lost his keys and they both look under the streetlight together. After a few minutes the policeman asks if he is sure he lost them here, and the drunk replies, no, and that he lost them in the park. The policeman asks why he is searching here, and the drunk replies, "this is…
Sure, for casual evaluation, I agree. But are there serious analyses that are evaluating this kind of thing? I mean, these are the kinds of things I evaluate in my own work when a new model comes out, or when I'm evaluating a harness. But this is all very ad hoc and intuitional. I'd love to start bringing rigor to it, but I haven't found much prior art on this. In another thread someone said that's because it's proba…
It's hard to bring much rigor to it. I'm not saying impossible, but it's not like it's completely obvious how to do it and people are just too lazy. Intrinsically, if I'm going to test a back-and-forth with a model I have a human in the loop making frequent decisions. Did the model fail or succeed at whatever rate it did that because of the model or the human? Did the testing protocols capture the actual problem, e.g., maybe if the model was given some particular bit of information that a normal human would have given it it would have done much better or worse, but the testing protocol in the interests of "rigor" excluded the human in the loop from doing it. Is the human going to be willing to sit down and do the same task 25 times, refreshing the model from scratch each time for a "valid" test? Can you get the same human to analyze every model in the test? Is their 10th pass of the problem an invalid test because you can't as easily erase the human's knowledge of the previous 9 tests? What do you do with a model that succeeds wildly 75% of the time and spins off into a loop the other 25%? Is that loop real or, again, did your "rigorous" testing protocol prevent the human from saving the model from the loop like any developer would?
And so on and so forth. Again, I'm not saying this is impossible but I am saying that if you tried to do it, and you got the money, and you built the test, and got the human subjects clearance, and you ignored that during the process of all that at least one more frontier model would come out, you can count on HN anklebiting your "rigorous" study even so, and probably being correct about a lot of the issues it could have because it would take several iterations of this to build a reasonable protocol... at which point it would quite possibly also be obsoleted by progress again.
Re: GLM 5.2 vs. Opus
#219I seriously dont' know all this big hullabaloo about one shot prompting. by definition, a single prompt wont' constitute the complexity of a software project. ergo, what you'll get is a series of assumptions made by the model based on preexisting code in its training corpus. I'd rather see a coding agent that can follow steps in a plan file to a T while following guardrails and adhering to the proper coding conventio…
> a single prompt wont' constitute the complexity of a software project.
The top agent is for steering, but all subagents are mostly oneshot prompts
Re: GLM 5.2 vs. Opus
#220How are people running this locally? I just checked llama.cpp and it appears unsloth has a version but it hacks a bunch of things to make it work and isn't optimal. https://github.com/ggml-org/llama.cpp/issues/24730
No one is doing that for a model this size it would have to be so heavily quantized that it wouldn’t be useful - or you’d need to spend a half million dollars on hardware. People use hosted APIs. Open weight means cloud vendors can host it.
Can you recommend any US based cloud providers?