241–250 of 364 posts
Any benchmark showing Opus 5 as the best just loses credibility for me. Anyone who's actually used Opus 5 daily knows what I'm talking about.
Agreed, it's extremely frustrating. It's the only model that actually makes me curse when talking to it, even knowing how counterproductive it is.
Opus is still first in Intelligence Index followed by Fable, GPT 5.6, Kimi K3 then Qwen 3.8 max. https://artificialanalysis.ai/#intelligence Our leaderboard combines Arena ELO, AA Intelligence index, latency and speed and goes: #1 Opus 5 #2 Kimi K3 #3 Qwen3.8 Max #4 GPT 5.6 Sol Source: http://pellmell.ai/leaderboard . This jumps around a lot based on the top throughput and latency of whatever provider happens to be b…
All these "intelligence" benchmarks miss something extremely important when using an LLM in a code-agent harness: How it communicates with you about what it did. Opus-5 is practically unusable (for complex tasks) in this sense - its updates are voluminous, and dense with cryptic language (there are numerous reddit threads complaining about this, so it's not just me). I often have to ask it to re-state concisely in pl…
Yeah if you ask it a medical question, it answers in that impenetrable jargony style that clinical journals use... full of unnecessarily custom adjectives ("orthopedic" instead of "of the bone") and discipline-specific terms (anterior, distal) even when the user didn't display mastry of this terminology (hint to frontier labs: add training cases for this; it will improve your model's usability).
My theory is that LLMs perceive the writing styles of various fields as being like different (but related) languages, and they're inclined to answer a question in the language of its source material unless specifically asked otherwise. If you add "ELI5" the model treats it as a question plus a translation task.
I think this is why programming questions are answered with an exaggerated cringey form of HN-speak ("load bearing", "gate" as a verb, "dissolves") by some models.
Out of curiosity, what's currently the best model I can use locally?
$500k - Kimi K3 (maybe $250k? Haven’t done this one) $25k - DSv4 Flash $4k - Qwen 3.6 35A3B Q5 $1k - Qwen 3.6 27B Q4 Some people prefer the sense over the MoE YMMV.
Earlier quoted context omitted.
They have something like 80% gross margins, are at a $100B/yr ARR, and are growing at 10x per year... If that keeps up, they're going to be doing more revenue than Google in a year ($400B ARR, 20% per year growth)
How can you sanely project the last 12 months forward? We have seen a huge uptick in usage. Last summer AI was a toy to most devs, now every enterprise developer I talked to uses it every day. Coding agent providers are surely going to hit market saturation in the near future.
I don't think you're right about that last prediction, at all. And new use cases are opening up as these get smarter. I think things are going to get pretty weird.
But the point was that it really doesn't look like they're losing money on users, on average.
Earlier quoted context omitted.
Is that what it feels like when the models get smarter than us?
No the models are just ass at communication without being directed. Try asking them to make useful diagrams for some stuff in a codebase, out of the box without excessive hand holding they don't make good choices about what's worth communicating and how to do it. You see this in their pointless frontend copy all the time too.
Same concept of "over-sharing" seems to prevalent in a bunch of domains when it comes to LLMs, sometimes more visible, sometimes less.
China has caught up is the main takeaway here. The SOTA models are so close that it's really hard to compare them intelligence wise - you have to get a feel for them yourself and what works for you. What I'm really excited for is the 27B model. 3.6 is still the king of local, and if 3.8 makes the same improvements it could really legitimately make local viable as a default. I'd love to run a perpetual agent on 3.8 th…
Qwen 3.6 27b is already a viable default. I'm running it on a single 7900 XTX right now for Go development with pi. It's great.
Setting the memory to "fast timings" is good for 8-12% more tokens/second if you haven't tried yet. I miss the slightly older days of AMD when powerplay tables were unlocked and we could configure the timings and voltages manually, there's another 30% being left on the table ez
Earlier quoted context omitted.
If the legal system declares the first thief’s theft not theft then all bets are off.
> If the legal system declares the first thief’s theft not theft But they didn't find it. The Big LLM provider accepted guilt and paid a fine. You can argue whether it was a fair amount they paid, but there is no legal precedent that was set. It's still considered theft.
That's not how it works. You have to give it back.
Otherwise, the distiller can just pay a fine (no larger than the original did) and be okay then, right ?
Earlier quoted context omitted.
They JUST updated their methodology: https://artificialanalysis.ai/methodology/intelligence-bench... Edit to provide AA's article explaining it: https://artificialanalysis.ai/articles/artificial-analysis-i...
> HLE, AA-LCR and AA-Omniscience are now graded by GPT-5.6 Luna (medium), replacing GPT-4o, Qwen3 235B A22B 2507, and Gemini 3 Flash Preview respectively. These checks are now unified under a more capable modern model, selected for strong agreement with human judgment in our grader validation Interesting that they chose a nano-sized model from OpenAI to be a grader for benchmarks involving knowledge and hallucination…