Price per 1M tokens is meaningless • Hacker News Board

Re: Price per 1M tokens is meaningless

#41

Post by janalsncm » Mon, Jul 06, 2026, 9:05 PM UTC

yreg wrote: post #34

sweetjuly wrote: post #18

Earlier quoted context omitted.

I think what you're really getting at is that it's only useful if the benchmarks are predictive of your workloads. If it predicts well (for example, your tasks are equally easy), then the fact that a larger model can complete it more quickly means that you may be able to complete the task more cheaply, depending on the token cost. If the benchmarks are non-predictive, well, you can't use them for much of anything, wh…

Yeah, if the benchmark is actually predictive of the tasks you have then it is trivial to conclude that the cheapest-per-benchmark-task model will be the cheapest one for your tasks…

It might vary between tasks though. A model that’s great at abstract reasoning might be great at writing math proofs but struggle to write software in .

Re: Price per 1M tokens is meaningless

#42

Post by shireboy » Mon, Jul 06, 2026, 9:07 PM UTC

Yup. I’ve been evaluating several on openrouter and find token cost meaningless for my work. I haven’t found a great alternative, though the “cost per task” he uses makes some sense.

Re: Price per 1M tokens is meaningless

#43

Post by lelandbatey » Mon, Jul 06, 2026, 9:08 PM UTC

nojito wrote: post #11

lifeisstillgood wrote: post #9

My advice to any CEO / individual - throw your hands in the air and bring it in-house. Yeah the performance can dip depending on what GPUs you can salvage these days but the uncertainty over price is almost nothing compared to the uncertainty over the effective use of AI. It’s not just coding (do I go partly agentic or all out Steve Yegge). This is all over the enterprise - do we parse every email, rewrite PowerPoint…

>bring it in-house People don't like to hear this but the open models just aren't good for end to end agentic workflows. There are some very very good small open models that can excel in certain finite bounded tasks, but the foundational models are essential to building out agentic pipelines that actually work.

Have you been using those models? I've been using a hand-rolled orchestrator with Mimo v2.5 (I seem to be paying $0.017 per million/tokens after their heavy caching) and it's been very impressive. I started with it in Opencode as a harness, then had it build its own micro-harness with stdlib-only Python, then used that to build a local stdlib-only Orchestrator with CLI and web harness, and now I'm using that for improving itself and now multi-project wider-ranging software. I talk to a steward who investigates and plans, then the plans are handed off to parallel worker agents who go through a work, test, interrogate, review, eval state machine for quality (all autonomously) with me at the end just reviewing the work or getting notified if the work items aren't progressing due to the workers getting stuck. So far the only "getting stuck" has been bugs/configs on my part, all at a pretty great quality bar, and at a price that makes me laugh at things like Opus.

I'm still using Claude at work (they're the only approved provider), but wow are the smaller models starting to SMOKE the big ones. At this point, all I'd consider paying out of my own pocket for is the lowest-limit Anthropic/GPT plan to get a big model as the Steward, but I wouldn't pay for ANY of the Anthropic models as the workers who do all the work. And as time passes, I don't know if I'd even do that; the open models are serving SO well.

Re: Price per 1M tokens is meaningless

#44

Post by hahahaa » Mon, Jul 06, 2026, 9:09 PM UTC

teraflop wrote: post #23

yreg wrote: post #8

Cost per token doesn't say a lot, but "Cost per benchmark task" is also meaningless if your task is difficult enough that the cheaper model has no chance of cracking it.

That's not meaningless at all, it's a great metric! If the "cost per correctly-solved benchmark task" is infinity, you know not to use the model.

Sisyphus doesn't care about energy per meter to move the rock.

Re: Price per 1M tokens is meaningless

#45

Post by PunchyHamster » Mon, Jul 06, 2026, 9:09 PM UTC

kpw94 wrote: post #28

In the context of local LLMs on limited hardware I've ran to the exact same conclusion: "tok/s" isn't the most useful metric when my personal North star metric, given my fixed hardware is: Model smart enough to execute my goals _in the minimum amount of time_. Some models I tried (Mistral I think) had better tok/s, and roughly same billion parameters / scores on various benchmark... But they were _so_ verbose, that t…

I feel like we need to see more proliferation of local LLMs to start seeing ones turned to be terse, rather than maxing the amount of tokens user pays for

Re: Price per 1M tokens is meaningless

#46

Post by HarHarVeryFunny » Mon, Jul 06, 2026, 9:10 PM UTC

nojito wrote: post #25

TacticalCoder wrote: post #15

Earlier quoted context omitted.

> People don't like to hear this but the open models just aren't good. Stuff like the latest DeepSeek, Kimchi and GLM are used and loved by many people. It's not using an open model that is difficult: it's having the hardware allowing to do so. It's pricey and require technical skills. That's why most people who are using (excellent btw) open-weight models are just renting compute online.

They just aren't good at agentic work. Also risking it all for some distilled models is a recipe for disaster.

All of the smaller models from anyone are distilled from larger ones. I assume you are just trying to disparage the Chinese models, but what you are actually saying is that people should only be using the largest non-distilled models, not smaller ones like Sonnet. I assume the upcoming Opus 5 will be distilled from Fable 5.

Re: Price per 1M tokens is meaningless

#47

Post by danielmarkbruce » Mon, Jul 06, 2026, 9:12 PM UTC

An LLM is an extremely complex thing used for all manner of purposes. The hope that there would be some simple pricing construct that would map nicely to value provided is a pipe dream.

Pricing per token is at least reasonably straight forward. If you aren't getting value, you don't use the service. One doesn't buy a Ferrari and then complain that in their town Ferrari doesn't help them pick up women and hence it should cost less.

Re: Price per 1M tokens is meaningless

#48

Post by dandaka » Mon, Jul 06, 2026, 9:15 PM UTC

Another important benchmark would be — cost per benchmark task using subscription tokens. Since most of us are using subscriptions and cost per token there is quite different from API costs.

Re: Price per 1M tokens is meaningless

#49

Post by Lerc » Mon, Jul 06, 2026, 9:16 PM UTC

Cost per tokens is as valid as price per unit volume of fuel.

Changing the fuel type, efficiency of your vehicle, driving distance, or driving conditions will all change how much it will cost you.

Fuel cost per unit volume does not become meaningless just because you are neglecting all of the other factors involved. That would be throwing away the only data point you have been using.

This is just asking for someone to amalgamate all of the factors involved into one simple, easy to game, index.

Re: Price per 1M tokens is meaningless

#50

Post by janalsncm » Mon, Jul 06, 2026, 9:19 PM UTC

HarHarVeryFunny wrote: post #36

janalsncm wrote: post #13

Pricing based on tokens always seemed a little weird to me.“Tokens” was and still is an engineering concept. The fundamental unit of transformer encoding and decoding. But I have a sinking feeling that many AI developers think “tokens” got their name from the same idea as “virtual tokens in a casino” which is more related to product pricing and business.

Tokens do reflect the provider's cost though - each token output required them to execute the model once, normally incurring a fixed amount of compute per token.

Providers amortize the compute across a batch.

If yours is the only request in the batch it will cost them one full pass through the model.

If yours is one of 1024 inputs in the batch the per token cost is 1024x less.