Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The real prices of frontier models

playcode.io

Post reply on HN
Print view View original

81–90 of 91 posts

Re: The real prices of frontier models

#81

Post by vlian2088 »

alansaber wrote: post #20
SwellJoe wrote: post #3

The fact that OpenAI documents theirs is already a big improvement over Anthropic. But, also, the OpenAI tokenizer got more efficient when they last updated it, rather than less. https://mdstudio.app/o200k-base-tokenizer

Interesting. New models are estimated at ~5T params, so 45,000x increase over BERT base (110m). But vocab size of 200k, so only an increase of 7x over BERT base (30k).

BERT is not an LLM, it's an encoder-only model, i.e. it doesn't generate new text. the first somewhat useful publicly accessible LLM was GPT-3 with 175B params, and it was also the last frontier model whose parameter count was disclosed.

Re: The real prices of frontier models

#82

Post by scottcha »

dinkleberg wrote: post #33
j45 wrote: post #17

An individual token, and the level of energy it represents (electricity, or relative effectiveness per model) increasingly seems the space of obsfucation. This space can be increasingly avoided by becoming, and remaining, efficient and effective with prompts.

That is one of the interesting things about Neuralwatt cloud. Their pricing is based on energy rather than tokens (actually they have a token-based alternative, but claim the energy pricing results in 95% cheaper results). I've tried out their subscription offer and it does seem like you get a lot more usage even on the cheap plan. However since the energy metering is pretty much unique to them (at least that I've se…

Hi, I'm co-founder of Neuralwatt. While there aren't other providers selling by energy we also produce the same tokens stats you get at others (input, output, cached) and you can compare using those. The CO2 number is really just multiplying the energy by the CO2-intensity at the time/place we served the request. So everything is actually measured and can be derived externally.

The main observation when you compare to token prices our input/cached energy is much lower than the equivalent token prices while the output energy is generally higher. One of the reasons this is a bit cheaper, especially for agentic systems, is that in an fully built/cached session your input & cached to output ratio is really massive input to very small output (multiple orders of magnitude usually). The other thing we do is we actively work to optimize the system around tokens/joule with the goal of making the most energy-optimal system.

Happy to answer any other questions.

Re: The real prices of frontier models

#84

Post by ianberdin »

iLoveOncall wrote: post #50

Very unpalatable completely LLM-written article, but on top of that a lot of the fundations and conclusions are completely wrong, the main one being this one: > You will see people claim Claude uses 2x to 4x the tokens of GPT. Our measurements do not support that, and overstating it would undercut the real point. It's not because a single prompt represents only 1.7x the number of tokens that a model doesn't use 4x as…

Just checked some of your comments. I see you mostly talk about how everything is AI written.

Re: The real prices of frontier models

#85

Post by robbie-c »

ianberdin wrote: post #58
robbie-c wrote: post #39

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides". I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it? Edit: this specific title has been deleted from the arti…

Well, criticizing is, of course, great. But the reality is that English is not my native language and I dictated most of it with my voice, then processed it with the help of AI, translated, added, corrected, and converted. It is actually a big result of work, a lot of research and attempts. And to just say that "oh, this is AI-slop," I consider unfair, but that is your choice. There is a difference: - There are peopl…

> There are people who do, - And there are those who criticize.

There’s no need for that. I’ve done plenty.

Re: The real prices of frontier models

#86

Post by iLoveOncall »

ianberdin wrote: post #84
iLoveOncall wrote: post #50

Very unpalatable completely LLM-written article, but on top of that a lot of the fundations and conclusions are completely wrong, the main one being this one: > You will see people claim Claude uses 2x to 4x the tokens of GPT. Our measurements do not support that, and overstating it would undercut the real point. It's not because a single prompt represents only 1.7x the number of tokens that a model doesn't use 4x as…

Just checked some of your comments. I see you mostly talk about how everything is AI written.

No I don't. But you seem butthurt. You sure are a great CEO.

Re: The real prices of frontier models

#89

Post by semiquaver »

ianberdin wrote: post #14
semiquaver wrote: post #5

Are there any advantages of the new tokenizer? Does it have a larger or smaller vocabulary or just differently weighted?

Well, in my view, it's just the most ordinary manipulation to avoid creating unrest. There is most likely no improvement inside. Of course, these are my guesses, but did anyone feel the difference in the transition from Opus 4.5 to 4.6? In my opinion, no. And it's unlikely to be a matter of the tokenizer.

No offense, but I didn’t ask for uninformed speculation. Why even bother chiming in? I can speculate on my own just fine.

Re: The real prices of frontier models

#90

Post by nullsanity »

marcosdumay wrote: post #78
nullsanity wrote: post #26

Earlier quoted context omitted.

So, worse? Because we benchmark off token use when talking about token use, and everyone else understood that.

The most important feature of a tokenizer is dividing the inputs into independent values that the neural network can work with. It's not the size.

[dead]
Post reply on HN