Why everyone insists that api inference is sold at cost and not at 1000% margin? This genre of article is based on that assumption.
Is This the Dawn of the Tokenpocalypse?
51–60 of 91 posts
Re: Is This the Dawn of the Tokenpocalypse?
#52Useful context for this is that token usage keeps rising at an exponential pace. I mean, we don't have numbers for the big labs, but Openrouter's numbers are quite telling (can't post link because corporate decided to block all "non-validated AI tools"), and I think they're probably representative of the global trend. +500% year to date, +50% over the month of May alone. It's unsurprising that providers are strugglin…
using openrouter leaderboards is no good. being on top of the board is marketing, so some labs are gaming that number. all marketing spend.
Re: Is This the Dawn of the Tokenpocalypse?
#53With no details, a bird told me of a project which estimated using several millions of tokens per day to automate a team work which got laid off. The operation is now a mess, there is no one willing to be considered liable and since the cheap model they used is about to be retired the company is going to see a 4x increase in price at least. I have the feeling that the age of 'i can't be blamed by AI stuff' will be a…
To me this is clearly a skill issue. Several millions of tokens per day is peanuts, even if uncached. gpt-5.5 is $5 per million of input tokens. Anybody doing things seriously understand how to optimize their workflows for smaller models once they start to lock in processes.
The expensive tokens are output, not input. A useful rule of thumb is that a million tokens per day means about ~10 tok/s on a 24/7 basis.
Re: Is This the Dawn of the Tokenpocalypse?
#54Earlier quoted context omitted.
The idea probably was to pour billions into technologies powering these LLMs, and gain a moat. It then turns out that this isn't as hard a problem as expected, it's just very expensive. So as long as you have money, you can be an AI company, the money is the moat (unless you take a shortcut, like DeepSeek) and money is running out. I don't think you're missing anything, but I am surprised that the forces behind the A…
> They do need to start making money, but I don't think anyone has a plan as to how they are going to do this. Doesn't this just mean price increase ? What is not clear is how much the price needs to increase for AI companies to break even some time. 3x increase ? 10x increase ? Even more ? No one seems willing to give a clear number.
You can only increase the price so much. With every price increase you're going to lose customers, which could lead to further price increases.
I'm not entirely convince that the AI companies can raise prices and keep enough of their customer base to make their current strategy commercially viable.
They could also lower their production cost, but that runs counter to building/buying new datacenter capacity. Realistically I think they need to look for applications where cheaper models are just as good and niches that where the ROI on AI is more clear.
Re: Is This the Dawn of the Tokenpocalypse?
#55Perhaps advanced AI isn't cheaper than humans. Assuming the intelligence of a model continuously improves with scale, the token price of the best model will become increasingly expensive. I know that tokens are currently experiencing rapid price drops, but they will eventually encounter physical limitations.
Why? What physical limitation will dictate that we can't have 1B tokens for cheap?
Assume you build a machine that can simulate some system 1:1. Then it means the machine is exactly the same as the system, and the cost of running it will no less than the system itself.
If you want to reduce the cost but still get something useful, you have to make some abstraction, and we all know that any abstraction is leaky.
Re: Is This the Dawn of the Tokenpocalypse?
#56Earlier quoted context omitted.
To me this is clearly a skill issue. Several millions of tokens per day is peanuts, even if uncached. gpt-5.5 is $5 per million of input tokens. Anybody doing things seriously understand how to optimize their workflows for smaller models once they start to lock in processes.
The expensive tokens are output, not input. A useful rule of thumb is that a million tokens per day means about ~10 tok/s on a 24/7 basis.
Even then, i highly doubt any sort of automation is producing on the order of several millions of tokens daily. The issue I see with the org in parent comment seems to stem from management and not any sort of token repricing.
Re: Is This the Dawn of the Tokenpocalypse?
#57With no details, a bird told me of a project which estimated using several millions of tokens per day to automate a team work which got laid off. The operation is now a mess, there is no one willing to be considered liable and since the cheap model they used is about to be retired the company is going to see a 4x increase in price at least. I have the feeling that the age of 'i can't be blamed by AI stuff' will be a…
On the topic of older (Claude) models being better... anyone knows anything close to 3.5 (or 3.6) era Sonnet? It was by far the best LLM I had ever asked my doubts too. It actually explained in a human way, not like some AI I need to re read thrice to understand.
(I've used modern Gemini 3.1 pro & claude too. Modern ChatGPT is just as useless, I've never heard a human speak in points. The human brain never encounters that irl.)
Re: Is This the Dawn of the Tokenpocalypse?
#58With no details, a bird told me of a project which estimated using several millions of tokens per day to automate a team work which got laid off. The operation is now a mess, there is no one willing to be considered liable and since the cheap model they used is about to be retired the company is going to see a 4x increase in price at least. I have the feeling that the age of 'i can't be blamed by AI stuff' will be a…
On the topic of older (Claude) models being better... anyone knows anything close to 3.5 (or 3.6) era Sonnet? It was by far the best LLM I had ever asked my doubts too. It actually explained in a human way, not like some AI I need to re read thrice to understand. (I've used modern Gemini 3.1 pro & claude too. Modern ChatGPT is just as useless, I've never heard a human speak in points. The human brain never encounters…
This was obviously a conscious choice from the leadership at he frontier labs, and especially OpenAI, considering how 4o turned out.
I don't think they expected the ELIZA effect [0] to explode as much as it did when they started including feedback directly from users into posttraining the next generation, so to be safe they've likely added several regimens of synthetic data ensuring ChatGPT tries to steer away from ELIZA.
Re: Is This the Dawn of the Tokenpocalypse?
#59As an exception to Betteridge's law of headlines: Yes.
Re: Is This the Dawn of the Tokenpocalypse?
#60With no details, a bird told me of a project which estimated using several millions of tokens per day to automate a team work which got laid off. The operation is now a mess, there is no one willing to be considered liable and since the cheap model they used is about to be retired the company is going to see a 4x increase in price at least. I have the feeling that the age of 'i can't be blamed by AI stuff' will be a…
Glad I'm not the only one. Almost every factual thing with new opus is wrong (and it now even happens with 4.6?). I asked it about car stuff yesterday and it totally misrepresented how a car axle even looks like fundamentally. Today I talked about my CV and it was just plain wrong. I don't know what happened, it wasn't like this a few weeks back and I'm even considering cancelling claude alltogether. GPT 5.5 for coding is fine and way more stable, but regular work is just broken.