The 'Glass slipper' idea makes sense to me; people have a bunch of different ideas to try on AIs, and try it as new models come out, and once a model does it well they stick with it for a while.
State of AI: An Empirical 100T Token Study with OpenRouter
41–50 of 97 posts
Re: State of AI: An Empirical 100T Token Study with OpenRouter
#42I like to see stats like that, but I find it very concerning that OpenRouter don't mind inspecting its user/customer data without shame.
Even if you pretend that the classifier respect anonymity, if I pay for the inference, I would expect that it would be a closed tube with my privacy respected. If at least it was for "safety" checks, I don't like that but I would almost understand, now it is for them to have "marketing data".
Imagine, and regarding the state of the world it might come soon, that you have whatsapp or telegram that inspect all the messages that you send to give reports like:
- 20% of our users speak about their health issues
- 30% of messages are about annoying coworkers
- 15% are messages comparing dick sizes
Re: State of AI: An Empirical 100T Token Study with OpenRouter
#43Earlier quoted context omitted.
https://en.wikipedia.org/wiki/Central_limit_theorem For example, even 300 really random people is enough to correctly assertain the distribution of population for some measurement (say, some personality feauture). That’s the basis of all polls and what have you
I think you might be thrashing around 30 samples for a normal distribution and the Central Limit Theorem and accidentally added a zero! (OK, on rereading, you did link to a WP article about CLT, so 30 it is!)
You’re absolutely right! (c)
300 — I had in memory as a safe bet in a case of some skewed stuff like log-normal, exponential, etc.
Re: State of AI: An Empirical 100T Token Study with OpenRouter
#44Here is the thing: they made good enough open weight models available and affordable, then found that people used them more than before. I am not trying to diminish the value here but I don’t think this is the headline.
Re: State of AI: An Empirical 100T Token Study with OpenRouter
#45Earlier quoted context omitted.
"reasoning" models like GPT 5 et al do a pre-generation step where they: - Take in the user query (input tokens) - Break that into a game plan. Ex: "Based on user query: {query} generate a plan of action." (reasoning tokens) - Answer (output tokens) Because the reasoning step runs in a loop until it's run through it's action plan, it frequently uses way more tokens than the input/output step.
that was useful, thank you. I have sooo many issues with the naming scheme of this """""AI"""" industry", it's crazy! So the LLM gets a prompt, then creates a scheme to pull pre-weighted tokens post-user-phrasing, the constituents of which (the scheme) are called reasoning tokens, which it only explicitly distinguishes as such because there are hundreds or even thousands of output tokens to the hundreds and/or thousa…
You can call them planning if you want or pre-planning. But I would encourage you to play with the API version of your model of choice to see exactly what this looks like. It’s kind of like a human’s internal monologue: “got an email from my boss asking to write unit tests for the analytics API. First I have to look at the implementation to know how exactly it actually functions, then write out what kinds of tests make sense, then implement the tests. I should write a TODO list of these steps.”
It is essentially a way to expand the prompt further. You can achieve the same exact thing by turning off the “thinking” feature and just being more detailed and step by step in your prompt but this is faster.
My guess is that the next evolution of this will be models that do an edit or review step after to catch if any of the constraints were broken. But best I can tell a reasoning model can be approximated by doing two passes of a non-reasoning model: first pass you give it the user prompt with instructions that boil down to “make sense of this prompt and formulate a plan” and the second pass you give it the original prompt, the plan, and an explanation that the plan is to implement the original prompt using the plan.
Re: State of AI: An Empirical 100T Token Study with OpenRouter
#46*State of non-enterprise, indie AI All this data confirms that OpenRouter’s enterprise ambitions will fail. It’s a nice product for running Chinese models tho
They have SOTA models from OpenAI and Anthropic and Google and you can access them at a 5.5% premium. What you get is the ability to seamlessly switch between them. And also when one is down you can instantly switch to another. Whether that is valuable to you or not is use case dependent. But it isn’t without value.
What it does have I think is a problem that TaskRabbit had: you can hire a house cleaner through TR but once you find a good one you can just work directly with them and save the middleman fee. So OR is great for experimenting with a ton of models to see what is the cheapest one that still performs the tasks you need but then you no longer need OR unless it is for reliability.
Re: State of AI: An Empirical 100T Token Study with OpenRouter
#47I like to see stats like that, but I find it very concerning that OpenRouter don't mind inspecting its user/customer data without shame. Even if you pretend that the classifier respect anonymity, if I pay for the inference, I would expect that it would be a closed tube with my privacy respected. If at least it was for "safety" checks, I don't like that but I would almost understand, now it is for them to have "market…
They explicitly give you a discount if you opt in to allowing your data to be used for (anonymized) analytics. That’s pretty fair imho.
Re: State of AI: An Empirical 100T Token Study with OpenRouter
#48*State of non-enterprise, indie AI All this data confirms that OpenRouter’s enterprise ambitions will fail. It’s a nice product for running Chinese models tho
They have SOTA models from OpenAI and Anthropic and Google and you can access them at a 5.5% premium. What you get is the ability to seamlessly switch between them. And also when one is down you can instantly switch to another. Whether that is valuable to you or not is use case dependent. But it isn’t without value. What it does have I think is a problem that TaskRabbit had: you can hire a house cleaner through TR bu…
Use LiteLLM for model routing
Re: State of AI: An Empirical 100T Token Study with OpenRouter
#49I like to see stats like that, but I find it very concerning that OpenRouter don't mind inspecting its user/customer data without shame. Even if you pretend that the classifier respect anonymity, if I pay for the inference, I would expect that it would be a closed tube with my privacy respected. If at least it was for "safety" checks, I don't like that but I would almost understand, now it is for them to have "market…
>I would expect that it would be a closed tube with my privacy respected
Lol hahaha
Re: State of AI: An Empirical 100T Token Study with OpenRouter
#50Earlier quoted context omitted.
I'm curious what percentage of claude/codex users this is true for - I assumed their business models rely on this not being true for the majority.
Both Claude Code and Codex steer you towards the monthly subscription. Last time I tried Codex, I remember several aspects of it being straight up broken if used with an API key instead of a subscription account. The business model is likely built upon the assumption that most people aren't going to max out their limits every day, because if they were, it likely wouldn't be profitable.
I got $250 free Claude Code credit and I was surprised by how hard it was to actually use it all before it expired.