Price per 1M tokens is meaningless
janilowski.pl
Price per 1M tokens is meaningless
1–10 of 113 posts
Re: Price per 1M tokens is meaningless
#2Well, not totally meaningless but certainly can be misleading.
Re: Price per 1M tokens is meaningless
#3[deleted]
Re: Price per 1M tokens is meaningless
#4Related to this, for our use case, setting thinking to high instead of low made tasks complete faster and cheaper (Gemini 3.0 flash).
Other aspects are caching, often at 0.1X cost, where providers really differ in how efficient they are (Anthropic really good, Google not so much) and how chatty a model is (costing output tokens).
Re: Price per 1M tokens is meaningless
#5cost per benchmark task is definitely interesting!
i've always wanted cost per prompt, but even that has too much variation.
Re: Price per 1M tokens is meaningless
#6I believe the future lies somewhere in between. I'm working on a hybrid application to reduce our company's token consumption. It runs on our data center's computing infrastructure and on laptops in our community. You might be interested; you can check out the code if you're interested: https://github.com/vfalbor/hibrid
Re: Price per 1M tokens is meaningless
#7Price per token is meaningless for more reasons than this, because all of the provider monthly subscriptions price tokens _extremely_ differently than their per-token billing rates. It's stupid to look only at what you get when paying more than you need to for a given service.
Re: Price per 1M tokens is meaningless
#8Cost per token doesn't say a lot, but "Cost per benchmark task" is also meaningless if your task is difficult enough that the cheaper model has no chance of cracking it.
Re: Price per 1M tokens is meaningless
#9My advice to any CEO / individual - throw your hands in the air and bring it in-house. Yeah the performance can dip depending on what GPUs you can salvage these days but the uncertainty over price is almost nothing compared to the uncertainty over the effective use of AI. It’s not just coding (do I go partly agentic or all out Steve Yegge). This is all over the enterprise - do we parse every email, rewrite PowerPoints? Or just stop using PowerPoints at all. Do we throw LLMs at the mess of wikis and word docs, do we pretend that the policies no-one has ever read actually are how the LLM should think or is it how the work actually gets done - barely documented
The uncertainty of how to use this vastly vastly outweighs the price in a data centre - so buckle up, buy enoughbGPUs to experiment at a known cost and one day you will find the approach that gives you 10x returns - at that point pay any price per token but not till then
Re: Price per 1M tokens is meaningless
#10This reminds me of cpu benchmarks vs actually running games and measuring FPS.