Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

State of AI: An Empirical 100T Token Study with OpenRouter

openrouter.ai

Post reply on HN
Print view View original

91–97 of 97 posts

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#91

Post by veunes »

lukev wrote: post #9

Super interesting data. I do question this finding: > the small model category as a whole is seeing its share of usage decline. It's important to remember that this data is from OpenRouter... a API service. Small models are exactly those that can be self-hosted. It could be the case that total small model usage has actually grown , but people are self-hosting rather than using an API. OpenRouter would not be in a pos…

Yeah, using an API aggregator to run a 7B model is economically strange if you have even a consumer GPU. OpenRouter captures the cream of complex requests (Claude 3.5, o1) that you can't run at home. But even for local hosting, medium models are starting to displace small ones because quantization lets you run them on accessible hardware, and the quality boost there is massive. So the "Medium is the new Small" trend likely holds true for the self-hosted segment as well.

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#92

Post by veunes »

syspec wrote: post #8

According to the report, 52% of all open-source AI is used for *roleplaying*. They attribute it to fewer content filters and higher creativity. I'm pretty surprised by that, but I guess that also selects for people who would use openrouter

I'm not surprised. Roleplay means endless sessions with huge context (character history, world, previous dialogues). On commercial APIs (OpenAI/Anthropic), that long-context costs a fortune. On OpenRouter, many OSS models, especially via providers like DeepInfra or Fireworks, cost pennies or are even free, like some Free-tier models. The RP community is very price-sensitive, so they massively migrate to cheap OSS models via aggregators. It skews the stats but highlights a real niche for cheap inference

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#93

Post by veunes »

greatgib wrote: post #42

I like to see stats like that, but I find it very concerning that OpenRouter don't mind inspecting its user/customer data without shame. Even if you pretend that the classifier respect anonymity, if I pay for the inference, I would expect that it would be a closed tube with my privacy respected. If at least it was for "safety" checks, I don't like that but I would almost understand, now it is for them to have "market…

This is the inevitable evil of the man in the middle. OpenRouter by definition decrypts your traffic to route it to the provider (OpenAI, Anthropic). Technically, they can read everything The problem is that for the Enterprise segment, this is a showstopper. No bank or hospital will route data through an aggregator that openly states it classifies prompts via Google API (even sampled ones). This confirms that OpenRouter remains a tool for indie hackers and researchers, not for serious B2B

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#94

Post by Alex-Programs »

YetAnotherNick wrote: post #77
lukev wrote: post #9

Super interesting data. I do question this finding: > the small model category as a whole is seeing its share of usage decline. It's important to remember that this data is from OpenRouter... a API service. Small models are exactly those that can be self-hosted. It could be the case that total small model usage has actually grown , but people are self-hosting rather than using an API. OpenRouter would not be in a pos…

The bigger issue is that they count small based on fixed number of parameters, and not the active parameter for MoE, didn't account for any hardware improvements etc. If they counted small based on the price or computational cost, I think they would have seen increase in small models.

I think using total parameters is fair, it correlates well with the RAM prerequisites to run it. Otherwise Kimi K2 would be "small" despite being a trillion parameters!

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#95

Post by orbital-decay »

olalonde wrote: post #58
m3h wrote: post #57

Earlier quoted context omitted.

Why do major LLMs block china? Isn't that a potentially huge market for them?

I'm not sure, but my guess is that it's due to pressure (or perceived pressure) from the U.S. government.

It's their own decisions they made long before the controls and presure. Besides being in bed with the US gov, people that run big AI shops tend to be fervently nationalistic and politically ambitious on their own. Leopold Aschenbrenner's dystopian rant [1] or Dario Amodei's [2] [3] are pretty representative.

[1] https://situational-awareness.ai/

[2] https://www.darioamodei.com/essay/machines-of-loving-grace

[3] https://www.darioamodei.com/post/on-deepseek-and-export-cont...

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#96

Post by YetAnotherNick »

Alex-Programs wrote: post #94
YetAnotherNick wrote: post #77

Earlier quoted context omitted.

The bigger issue is that they count small based on fixed number of parameters, and not the active parameter for MoE, didn't account for any hardware improvements etc. If they counted small based on the price or computational cost, I think they would have seen increase in small models.

I think using total parameters is fair, it correlates well with the RAM prerequisites to run it. Otherwise Kimi K2 would be "small" despite being a trillion parameters!

VRAM doesn't matter if you are using API. Price and performance is what matters.

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#97

Post by xanderatallah »

greatgib wrote: post #42

I like to see stats like that, but I find it very concerning that OpenRouter don't mind inspecting its user/customer data without shame. Even if you pretend that the classifier respect anonymity, if I pay for the inference, I would expect that it would be a closed tube with my privacy respected. If at least it was for "safety" checks, I don't like that but I would almost understand, now it is for them to have "market…

Alex here from OpenRouter. We take privacy really seriously and how we do manage prompts and completions is described in our terms of service: https://openrouter.ai/terms.

We don’t retain any customer prompts or completions by default. As others here mentioned, you can opt-in for a 1% discount. Prompt classification is performed using a zero-data-retention and zero-training service, just like our own.

Post reply on HN