Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The real prices of frontier models

playcode.io

Post reply on HN
Print view View original

71–80 of 91 posts

Re: The real prices of frontier models

#71

Post by satvikpendem »

alansaber wrote: post #20
SwellJoe wrote: post #3

The fact that OpenAI documents theirs is already a big improvement over Anthropic. But, also, the OpenAI tokenizer got more efficient when they last updated it, rather than less. https://mdstudio.app/o200k-base-tokenizer

Interesting. New models are estimated at ~5T params, so 45,000x increase over BERT base (110m). But vocab size of 200k, so only an increase of 7x over BERT base (30k).

Interesting that Grok 4.5 nears or exceeds OpenAI and Anthropic models in some benchmarks at only 1.5 trillion parameters per their announcement post.

Re: The real prices of frontier models

#72

Post by MPSimmons »

luciana1u wrote: post #69

Anthropic's tokenizer being 2x less efficient means they're essentially charging you for whitespace. premium whitespace, mind you — each space character gets its own attention head.

I don't know if it's fair to say that a tokenizer is being less efficient if it generates more tokens per text. I think it's more fair to say the tokenizer is more nuanced. The question is whether the additional nuance permits better model output, which could justify the additional token cost in inference.

Re: The real prices of frontier models

#73

Post by MPSimmons »

recursivecaveat wrote: post #51
onlyrealcuzzo wrote: post #19

Earlier quoted context omitted.

Interesting... Naively I'd assume you'd have a pretty unfair advantage on quality if you have materially more information dense tokens. That doesn't really appear to be the case as GPT and Anthropic models appear evenly matched despite Anthropic encoding the same text into almost ~2x the tokens... I'd also - naively - assume this would make training their models more expensive. Though inference now dominates, and the…

If a given paragraph gets encoded into twice as many tokens, that means the model gets twice as many matmuls to process it. The amount of compute thrown at the problem is increased (everything else constant), which may improve the quality of the result. This is believed to be one of the reasons that 'thinking' tokens improve quality. For long tasks it will lead to more context compactions though which will harm the q…

It would be nice if inference could somehow perform token generation using "contractions" of "fluffy" tokens, where combining those tokens doesn't decrease nuance but provides additional efficiency. That may already be happening - I haven't looked at the most modern methods of inference in a long long time.

Re: The real prices of frontier models

#75

Post by golem14 »

dinobones wrote: post #48

Just start pricing in bytes input/output. This whole "token" and "tokenizer" thing is an implementation detail that shouldn't even be leaking out into the API. Providers change tokenizers all the time with model updates, and it's often not even possible to query/figure out how text is tokenized without actually just sending the LLM a request. Just switch to charging for bytes of intelligence. Please. Claude Shannon f…

That does not help you compare pricing between models since they can vary by (even internal, thinking) output size.

The best way to measure is really the end-2-end cost, price per task.

Re: The real prices of frontier models

#76

Post by NkVczPkybiXICG »

socketcluster wrote: post #64

It's a terrible metric, it's kind of like companies paying employees by the hour for white collar work. Most people here probably don't know what it was like to work a contract job and being paid based on actual deliverables. The incentive of AI companies is to create as many tokens as possible to solve any given problem. Just like your incentive as a software engineer is to create as much complexity as possible in o…

Big tech companies don’t usually pay hourly.

Re: The real prices of frontier models

#77

Post by ButlerianJihad »

NkVczPkybiXICG wrote: post #76
socketcluster wrote: post #64

It's a terrible metric, it's kind of like companies paying employees by the hour for white collar work. Most people here probably don't know what it was like to work a contract job and being paid based on actual deliverables. The incentive of AI companies is to create as many tokens as possible to solve any given problem. Just like your incentive as a software engineer is to create as much complexity as possible in o…

Big tech companies don’t usually pay hourly.

As a contractor, I was paid hourly all the time. At some points I needed to fill out an actual time sheet each week, that I would submit to my own company, and they used these time sheets to bill the clients.

So yes, "big tech companies" often paid hourly, even if that pay was indirect, to contractors and job shoppers and people who were not direct hires.

Re: The real prices of frontier models

#78

Post by marcosdumay »

nullsanity wrote: post #26
not-a-llm wrote: post #23

Earlier quoted context omitted.

you use the wrong word the Anthropic tokenizer is not worse, its more expensive/verbose

So, worse? Because we benchmark off token use when talking about token use, and everyone else understood that.

The most important feature of a tokenizer is dividing the inputs into independent values that the neural network can work with. It's not the size.

Re: The real prices of frontier models

#80

Post by wgd »

> DeepSeek and GLM are left out of the tables entirely: we only have rough characters-divided-by-four estimates for them, not real tokenizer counts, and this post is about measured numbers.

lolwut. The open-weight models are inscrutable black boxes for which we can't possibly get real token counts? Typical lazy clanker, BSing their way out of doing the whole job.

Post reply on HN