Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The mysterious Hy3 LLM is topping OpenRouter Model Rankings by a large margin

minimaxir.com

Post reply on HN
Print view View original

81–90 of 116 posts

Re: The mysterious Hy3 LLM is topping OpenRouter Model Rankings by a large margin

#81

Post by 0xbadcafebee »

NitpickLawyer wrote: post #35
0xbadcafebee wrote: post #29

Earlier quoted context omitted.

I just checked my settings and I have everything enabled. https://openrouter.ai/deepseek/deepseek-v4-flash?sort=price (per-1M price) shows DeepSeek provider as #5. https://openrouter.ai/deepseek/deepseek-v4-flash/pricing?sor... (effective price) shows them as #3. The effective price will change your total cost since each provider has a different price for input vs output vs cache, so what's #1 and #5 for one person c…

Interesting, it seems we have some providers offering dsv4-flash cheaper than ds themselves. For the full model it's the other way around, all 3rd party providers are 2x+ more expensive.

The cheaper ones are fp4 and fp8 whereas I assume DeepSeek provider is unquantized, so that probably accounts for it. DeepSeek also doesn't necessarily have the cheapest hardware, other providers could be using it as a loss leader, etc

Re: The mysterious Hy3 LLM is topping OpenRouter Model Rankings by a large margin

#82

Post by TZubiri »

estearum wrote: post #62
selcuka wrote: post #55

Earlier quoted context omitted.

If it's worthless to AI vendors, they won't include it in the training corpus, so third parties won't have access to it.

They're alluding to something more like espionage of just selling the interesting stuff you put in the text box.

Wow I thought this was quite obvious, apparently not, so I'll explain.

Llm provider sells usage of their model. You use it to write code. Other clients use it to write code as well. If the llm provider trains with user data, then the usage benefits other users. If you pay the company to generate code,then by definition it is useful, and highly likely that other customers care about it.

Replace writing code with anything, a lawyer, a psychologist, a confessional. The IO is inherently useful to users of the same category.

That is to say nothing of adversarial use, that is, being useful because a counterparty might find it useful, so an attacker might find common code patterns, a lawyer might see what the opposition might be advised, a boy might see what a girl asks or gets advised, etc..

If this sounds too complex to you, just think of training on data as exfiltration with added steps, because that's what it is

Re: The mysterious Hy3 LLM is topping OpenRouter Model Rankings by a large margin

#83

Post by TZubiri »

selcuka wrote: post #55
TZubiri wrote: post #44

Earlier quoted context omitted.

Third parties that can be clients of the AI vendor...

If it's worthless to AI vendors, they won't include it in the training corpus, so third parties won't have access to it.

But it isn't worthless because the user is paying for that, and third parties are paying for that as well. Unless the input output is completely different, which it's not because you are human, and I bet you have a profession which other humans have, and many other qualities which you share with other humans.

In any case, relying on the chance that the LLM inference won't train on your data because of it's presumably low value is as good a strategy as crossing your fingers or venerating the god of rain. You should be relying on contractual clauses at least when including professional and client data.

Re: The mysterious Hy3 LLM is topping OpenRouter Model Rankings by a large margin

#84

Post by estearum »

TZubiri wrote: post #82
estearum wrote: post #62

Earlier quoted context omitted.

They're alluding to something more like espionage of just selling the interesting stuff you put in the text box.

Wow I thought this was quite obvious, apparently not, so I'll explain. Llm provider sells usage of their model. You use it to write code. Other clients use it to write code as well. If the llm provider trains with user data, then the usage benefits other users. If you pay the company to generate code,then by definition it is useful, and highly likely that other customers care about it. Replace writing code with anyth…

Oh well this is a bad argument. I made a mistake by assuming you made a good argument instead.

Re: The mysterious Hy3 LLM is topping OpenRouter Model Rankings by a large margin

#85

Post by CamperBob2 »

zone411 wrote: post #21

I’ve tested this model on four of my benchmarks: https://github.com/lechmazur/buyout_game 10th out 36. https://github.com/lechmazur/pact/ 14th out 25. https://github.com/lechmazur/nyt-connections/ 60th out 81. https://github.com/lechmazur/debate 16th out of 29.

Would be interesting to see the 27B dense Qwen 3.6 model thrown into the mix.

Re: The mysterious Hy3 LLM is topping OpenRouter Model Rankings by a large margin

#86

Post by PunchyHamster »

simonw wrote: post #10

First model I've tried that gave me back HTML with a "Change Pelican Color" button: https://static.simonwillison.net/static/2026/hy3-preview-pel... (Transcript: https://gist.github.com/simonw/c2a0d8ecd3056a2681319eae8fc3f... )

...the animate wheels button makes sun start to spin

Re: The mysterious Hy3 LLM is topping OpenRouter Model Rankings by a large margin

#87

Post by numlocked »

alecco wrote: post #32

PSA: Don't use OpenRouter for DeepSeek V4 as it messes up you caching. Use DeepSeek API directly and you'll get 2x to 3x more cached tokens.

Can you share more? I'm with OpenRouter and we would love to address this! We don't see this in our own testing, I don't believe -- but will share this feedback and dig in.

Re: The mysterious Hy3 LLM is topping OpenRouter Model Rankings by a large margin

#88

Post by numlocked »

simonw wrote: post #11

OpenRouter rankings frustrate me, because they show the total number of tokens but they provide no indication of how many unique users a model has. Which means if a surprise model tops the leaderboard one week we can never be sure if it was because a single whale user pushing billions of tokens a day switched to it, or if it represents a genuine community trend towards that model.

(openrouter co-founder here)

Yeah we should do something to indicate cardinality. I can share that there can often (I'm talking generally; not related to this model in particular) be e.g. a very large app that can be pushing a lot of volume. But in almost all cases that app has a large number of end users. Hypothetically, for instance, would Cursor be consider one user, or millions?

Will think about it! Thanks for the feedback.

Re: The mysterious Hy3 LLM is topping OpenRouter Model Rankings by a large margin

#89

Post by prmph »

embedding-shape wrote: post #76
preek wrote: post #54

Earlier quoted context omitted.

It actually rendered an SVG inline in the HTML page. I just tested the SVG and it renders itself just fine, including colors. So, tbh, I'd say the task has been properly achieved.

Maybe I'm just extremely nitpicky, but I'd consider that a failure, as the prompt is asking for SVG, not HTML. Bit like asking for CSS and then getting a HTML file back with the CSS embedded, that was not what I was asking for!

Welcome to the ambiguities of natural language

Re: The mysterious Hy3 LLM is topping OpenRouter Model Rankings by a large margin

#90

Post by martinald »

numlocked wrote: post #88
simonw wrote: post #11

OpenRouter rankings frustrate me, because they show the total number of tokens but they provide no indication of how many unique users a model has. Which means if a surprise model tops the leaderboard one week we can never be sure if it was because a single whale user pushing billions of tokens a day switched to it, or if it represents a genuine community trend towards that model.

(openrouter co-founder here) Yeah we should do something to indicate cardinality. I can share that there can often (I'm talking generally; not related to this model in particular) be e.g. a very large app that can be pushing a lot of volume. But in almost all cases that app has a large number of end users. Hypothetically, for instance, would Cursor be consider one user, or millions? Will think about it! Thanks for th…

Hi! Big fan of OpenRouter and the data you provide. It'd be awesome if you would consider providing volume of tokens per hour, mostly for my own curiosity as to quite how peaky demand is.

Thanks!

Post reply on HN