Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

State of AI: An Empirical 100T Token Study with OpenRouter

openrouter.ai

Post reply on HN
Print view View original

61–70 of 97 posts

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#61

Post by nhaehnle »

m0rde wrote: post #25

> The noticeable spike [~20 percentage points] in May in the figure above [tool invocations] was largely attributable to one sizable account whose activity briefly lifted overall volumes. The fact that one account can have such a noticeable effect on token usage is kind of insane. And also raises the question of how much token usage is coming from just one or five or ten sizeable accounts.

It is quite interesting to ponder these usage statistics, isn't it?

According to their charts they're at a throughput of something like 7T tok/week total now. At 1$/Mtok, that's 7M$ per week. Less than half a billion per year. How much is that compared to the total inference market? And yet again, their throughput went like 20x in one year, who knows what's to come...

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#62

Post by sysguest »

IMTDb wrote: post #32
syspec wrote: post #8

According to the report, 52% of all open-source AI is used for *roleplaying*. They attribute it to fewer content filters and higher creativity. I'm pretty surprised by that, but I guess that also selects for people who would use openrouter

Or maybe it’s just strange classification. I see a lot of prompts on the internet looking like “act as a senior xxx expert with over 15 years of industry experience and answer the following: [insert simple question]” I hope those are not classified as “roleplaying” the “roleplay” here is just a trick to get better answer from the model, often in a professional setting that has nothing to do with creative writing of N…

"act as a senior xxx expert with over 15 years of industry experience"

... I just don't get why LLMs are affected by this kind of nonsense -- is it due to training rewards?

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#64

Post by armcat »

These are fantastic insights! I work in legaltech space so something to keep in mind is that legal space is very sensitive to data storage and security (apart from this of course: https://alexschapiro.com/security/vulnerability/2025/12/02/f...). So models hosted in e.g. Azure, or on-prem deployments are more common. I have friends in health space and similar story there. Finance (banking especially) is the same. Hence why those categories look more or less constant over time, and have smallest contributions in this study.

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#65

Post by mzl »

lukev wrote: post #9

Super interesting data. I do question this finding: > the small model category as a whole is seeing its share of usage decline. It's important to remember that this data is from OpenRouter... a API service. Small models are exactly those that can be self-hosted. It could be the case that total small model usage has actually grown , but people are self-hosting rather than using an API. OpenRouter would not be in a pos…

While it is possible to self-host small models, it is not easy to host them with high speeds. Many small-model use-cases are for large batches of work (processing large amounts of documents, agentic workflows, ...), and then using a provider that has high tps numbers would be motivated.

Still, I agree that self-hosting is probably a part of the decrease.

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#66

Post by slack2450 »

olalonde wrote: post #56
majdalsado wrote: post #23

Very interesting how Singapore ranks 2nd in terms of token volume. I wonder if this is potentially Chinese usage via VPN, or if Singaporean consumers and firms are dominating in AI adoption. Also interesting how the 'roleplaying' category is so dominant, makes me wonder if Google's classifier sees a system prompt with "Act as a X" and classifies that as roleplay vs the specific industry the roleplay was intended to s…

Almost certainly VPN traffic. Most major LLMs block both China and Hong Kong (surprisingly, not the other way around), so Singapore ends up being the fastest nearby endpoint that isn't restricted.

It’s not VPN traffic all data is aggregated by billing payment information so it’s Singaporean billing details.

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#67

Post by slack2450 »

charcircuit wrote: post #37
skywhopper wrote: post #20

This is interesting, but I found it moderately disturbing that they spend a LOT of effort up front talking about how they don’t have any access to the prompts or responses. And then they reveal that they did actually have access to the text and they spend 80% of the rest of the paper analyzing the content.

>And then they reveal that they did actually have access to the text I'm not seeing that. All I'm seeing is them analyzing metadata.

>All I'm seeing is them analyzing metadata Read the section about how they achieve classifications for prompts (hint: They read the prompts)

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#68

Post by mondojesus »

sysguest wrote: post #62
IMTDb wrote: post #32

Earlier quoted context omitted.

Or maybe it’s just strange classification. I see a lot of prompts on the internet looking like “act as a senior xxx expert with over 15 years of industry experience and answer the following: [insert simple question]” I hope those are not classified as “roleplaying” the “roleplay” here is just a trick to get better answer from the model, often in a professional setting that has nothing to do with creative writing of N…

"act as a senior xxx expert with over 15 years of industry experience" ... I just don't get why LLMs are affected by this kind of nonsense -- is it due to training rewards?

The way I think about it, the training data (i.e. the internet) has X% of people asking something like "explain it to me like I'm five years old" and Y% of people framing it like "I'm technical, explain this to me in detail". You use the "act as a senior XXX" when you want to bias the output towards something more detailed.

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#69

Post by CJefferson »

IMTDb wrote: post #32
syspec wrote: post #8

According to the report, 52% of all open-source AI is used for *roleplaying*. They attribute it to fewer content filters and higher creativity. I'm pretty surprised by that, but I guess that also selects for people who would use openrouter

Or maybe it’s just strange classification. I see a lot of prompts on the internet looking like “act as a senior xxx expert with over 15 years of industry experience and answer the following: [insert simple question]” I hope those are not classified as “roleplaying” the “roleplay” here is just a trick to get better answer from the model, often in a professional setting that has nothing to do with creative writing of N…

I can't be sure, but this sounds entirely possible to me.

There are many, many people, and websites, dedicated to roleplaying, and those people will often have conversations lasting thousands of messages with different characters. I know a people whose personal 'roleplay AI' budget is a $1,000/month, as they want the best quality AIs.

Re: State of AI: An Empirical 100T Token Study with OpenRouter

#70

Post by olalonde »

slack2450 wrote: post #66
olalonde wrote: post #56

Earlier quoted context omitted.

Almost certainly VPN traffic. Most major LLMs block both China and Hong Kong (surprisingly, not the other way around), so Singapore ends up being the fastest nearby endpoint that isn't restricted.

It’s not VPN traffic all data is aggregated by billing payment information so it’s Singaporean billing details.

Ah, you're right. Still, I wonder if it's because of Chinese people and companies using Singaporean bank accounts. It just seems odd that such a small country is so overrepresented here.
Post reply on HN