251–260 of 559 posts
Earlier quoted context omitted.
Yes, and the article author is fully aware of that. Thank you for pointing out this small mistake though.
It looks like the author is specifically avoiding model's name, because results are really weird. Opus 4.8/4.7 scored 28% Opus 4.6 score 37% So the author thought as let's not get into that just write Claude.
Does a bit worse than Opus 4.8 in my tests[0], but it's 5x cheaper and 3x slower. [0]: https://aibenchy.com/compare/anthropic-claude-opus-4-8-mediu...
I have taken another look on these open models after the fiasco of Fable and GPT 5.6 this weekend and... GLM-5.2 truly is a good workhorse model for daily programming. I consider myself a heavy user of LLMs and a seasoned developer. A typical session for me with GPT is usually over a hundred dollars... This weekend I programmed a matrix bot with encryption and a Rust agent with some tools. Because I need one and Open…
> A typical session for me with GPT is usually over a hundred dollars. I don't think a $100 session is "typical". I use GPT for months. $20/m plus plan is enough for my daily work.
Likewise, I don't understand how anyone survives on the basic plans. It's funny seeing these two camps not understanding what the other is doing :)
Does a bit worse than Opus 4.8 in my tests[0], but it's 5x cheaper and 3x slower. [0]: https://aibenchy.com/compare/anthropic-claude-opus-4-8-mediu...
Not sure what to make if your benchmark because GPT 5.5(low) ranks higher than GPT 5.5 (medium) -- #4 vs #9
It depends a lot on the task and harness too (using plans and to-do lists, vs one-shot answers), but for simply answering directly to an inquiry, often extra thinking doesn't necessarily improve the answer, especially if the answer is binary, or can be correct or wrong, as opposed to having more time to refine a creative output.
I have taken another look on these open models after the fiasco of Fable and GPT 5.6 this weekend and... GLM-5.2 truly is a good workhorse model for daily programming. I consider myself a heavy user of LLMs and a seasoned developer. A typical session for me with GPT is usually over a hundred dollars... This weekend I programmed a matrix bot with encryption and a Rust agent with some tools. Because I need one and Open…
> A typical session for me with GPT is usually over a hundred dollars. I don't think a $100 session is "typical". I use GPT for months. $20/m plus plan is enough for my daily work.
My typical session ranges from $100-$400 - higher end when using workflows with lots of subagents. $100/session is expected when using the API without the subsidized subscription pricing. Most larger orgs have to use API pricing AFAIK.
Does a bit worse than Opus 4.8 in my tests[0], but it's 5x cheaper and 3x slower. [0]: https://aibenchy.com/compare/anthropic-claude-opus-4-8-mediu...
Not sure what to make if your benchmark because GPT 5.5(low) ranks higher than GPT 5.5 (medium) -- #4 vs #9
https://aibenchy.com/compare/google-gemini-3-1-flash-lite-hi...
Earlier quoted context omitted.
The rumour is that it's trained on Opus, but who knows
Oh of course all deepseek and glm are. Multiple people have seen GLM self report that it is claude, which makes it super obvious. I think the surprising thing is I expect flash to be a pure distillation and strictly worse quality but clearly it’s more nuanced than that.
https://www.reddit.com/r/DeepSeek/comments/1rd5jw7/claude_so...
Earlier quoted context omitted.
GLM 5.2 is a great model, but if you only want to use the best model available, it isn't there yet. Every lab releases models that memorize benchmark answers, both intentionally and unintentionally. But we consistently find that models from Chinese labs have a wider gap between public benchmarks and our evaluations, which we designed to be less vulnerable to benchmaxxing. In multi-agent coding environments, GLM 5.2 i…
Opus 4.6 is still my preferred model for work, so this is great to hear.
Sounds like this is the year for coding.
I added GLM 5.2 to my security bug hunting benchmark when it came out, and found it to be a good performer, but not the best open model. The benchmark tests whether models can find bugs Mythos found. The best open models in the initial benchmark were DeepSeek V4 Pro or MiMo 2.5 Pro. But it turned out MiMo got lucky, it's performed worse on almost every test I've done since, while DeepSeek has consistently been among…
Deepseek v4 pro being in preview with less "safety" training makes it stronger for that reason. Thinking will be different and in the end, it will actually try to be useful. Just expect future Chinese LLMs to further push out "safety" guided LLMs. The future is bleak for open weight models. Prepare to have "guidelines" enforced unceremoniously to all.
I have taken another look on these open models after the fiasco of Fable and GPT 5.6 this weekend and... GLM-5.2 truly is a good workhorse model for daily programming. I consider myself a heavy user of LLMs and a seasoned developer. A typical session for me with GPT is usually over a hundred dollars... This weekend I programmed a matrix bot with encryption and a Rust agent with some tools. Because I need one and Open…
Twenty dollars? How are you comfortable spending that much to write something as simple as a matrix bot? Are people doing this kind of thing just super rich or am I missing something?