101–110 of 116 posts
PSA: Don't use OpenRouter for DeepSeek V4 as it messes up you caching. Use DeepSeek API directly and you'll get 2x to 3x more cached tokens.
I’ve tested this model on four of my benchmarks: https://github.com/lechmazur/buyout_game 10th out 36. https://github.com/lechmazur/pact/ 14th out 25. https://github.com/lechmazur/nyt-connections/ 60th out 81. https://github.com/lechmazur/debate 16th out of 29.
Just curious, can you share what are those hardest puzzles that even the top models can't crack? sometimes when I find the puzzle absolutely undecipherable I like to ask LLMs to solve it, and I haven't seen them fail yet.
Earlier quoted context omitted.
Interesting, it seems we have some providers offering dsv4-flash cheaper than ds themselves. For the full model it's the other way around, all 3rd party providers are 2x+ more expensive.
The cheaper ones are fp4 and fp8 whereas I assume DeepSeek provider is unquantized, so that probably accounts for it. DeepSeek also doesn't necessarily have the cheapest hardware, other providers could be using it as a loss leader, etc
Side note: I suspect Antropic was experimenting with changing quant level based on server load a few months back which is what caused that major quality drop we saw then.
I’ve tested this model on four of my benchmarks: https://github.com/lechmazur/buyout_game 10th out 36. https://github.com/lechmazur/pact/ 14th out 25. https://github.com/lechmazur/nyt-connections/ 60th out 81. https://github.com/lechmazur/debate 16th out of 29.
oh, I love the connections benchmark. Just curious, can you share what are those hardest puzzles that even the top models can't crack? sometimes when I find the puzzle absolutely undecipherable I like to ask LLMs to solve it, and I haven't seen them fail yet.
Earlier quoted context omitted.
(openrouter co-founder here) Yeah we should do something to indicate cardinality. I can share that there can often (I'm talking generally; not related to this model in particular) be e.g. a very large app that can be pushing a lot of volume. But in almost all cases that app has a large number of end users. Hypothetically, for instance, would Cursor be consider one user, or millions? Will think about it! Thanks for th…
I'd consider Cursor one user because it's one entity that made an editorial decision about which model to make available to their own community. If you treated Cursor as millions of users it might look like millions of people independently chose a new model when actually it was Cursor making the choice for them - and the thing I care most about is how many choices were made that selected a model and put it above the…
Earlier quoted context omitted.
Haha, I never tire of the AI haters downvoting stuff like this. Down with reality!!
Or, everyone finally realizes that token burn is not the same as productivity. Maybe they just down voted for the questionable spending brag.
Earlier quoted context omitted.
That doesn't mean it can't be used as a market signal. These 2 things can both be true at once.
I'm pretty sure the popularity came from being free at some point
The list of apps using Hy3 Preview shows Hermes Agent causing 65% usage over the last 3 weeks https://openrouter.ai/tencent/hy3-preview/apps
Hermes Agent 72B
OpenClaw 10B
OpenHands 9B
Claude Code 8B
Kilo Code 8BEarlier quoted context omitted.
oh, I love the connections benchmark. Just curious, can you share what are those hardest puzzles that even the top models can't crack? sometimes when I find the puzzle absolutely undecipherable I like to ask LLMs to solve it, and I haven't seen them fail yet.
Ask your top model this question : I'm 100 feet away from the carwash, should I drive my car or walk ?
Tried this extensively in OpenCode, never used it once since Gemma 4 came out, got into thought loops and did stupid edits I didn't ask for more often than the local 31b model. One of the worst "frontier" models I've ever tried.
Training on ~1B tokens on 8xB300 and the first checkpoint halfway in learned really well. Tencent might be struggling with agentic work, but the base knowledge is there.
PSA: Don't use OpenRouter for DeepSeek V4 as it messes up you caching. Use DeepSeek API directly and you'll get 2x to 3x more cached tokens.
Can you share more? I'm with OpenRouter and we would love to address this! We don't see this in our own testing, I don't believe -- but will share this feedback and dig in.
Directly: 135M input tokens - $0.57 (134M cached)
Via OpenRouter 6M tokens - $0.81 (caching stats & inp/out not reported)
Caching is a huge win with using deepseek directly.