Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The real prices of frontier models

playcode.io

Post reply on HN
Print view View original

61–70 of 91 posts

Re: The real prices of frontier models

#61

Post by stingraycharles »

mft_ wrote: post #46
robbie-c wrote: post #39

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides". I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it? Edit: this specific title has been deleted from the arti…

"No generation, no estimates - just token counts:" "Authoritative: it is the same count Anthropic bills against." "This reframes a headline that looked like good news."

Yeah this is an extremely poorly written article. They didn’t even bother to add a “rewrite this article to make it sound less than AI”.

It’s also using a bazillion words to make a point that could be summed up in a single paragraph: there’s a huge variance in the number of tokens required to encode the same content, with code leading the charts.

To be fair, most of this was already known, and Anthropic communicated very clearly about the different tokenizer they started using.

Their compute is also mostly 1:1 correlated to the number of tokens, so I don’t believe in the conspiracy that this is just to inflate prices.

Re: The real prices of frontier models

#62

Post by phist_mcgee »

iainmerrick wrote: post #52
robbie-c wrote: post #39

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides". I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it? Edit: this specific title has been deleted from the arti…

I’m normally one to complain about people complaining about these LLM-isms, but yeah, this one really grates on you. It’s a shame because it’s making an excellent point! It just takes so long to get to the point that the reader loses the will to live. Yes, I could probably ask an LLM to summarise it for me. No, I’m not going to. I would prefer the author just take care of that for me.

If you couldn't bother writing it, I can't be bothered reading it.

Re: The real prices of frontier models

#63

Post by d1l »

robbie-c wrote: post #39

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides". I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it? Edit: this specific title has been deleted from the arti…

Bro, this was the only reason I opened the comments, and I'm so happy to see someone else noticed. Even if they removed that, the utter sloppiness of the prose is unbelievable. Offensive, even.

Re: The real prices of frontier models

#64

Post by socketcluster »

It's a terrible metric, it's kind of like companies paying employees by the hour for white collar work.

Most people here probably don't know what it was like to work a contract job and being paid based on actual deliverables.

The incentive of AI companies is to create as many tokens as possible to solve any given problem. Just like your incentive as a software engineer is to create as much complexity as possible in order to use up as many hours as possible.

This is why big tech companies have millions of lines of code... They've got thousands of engineers rapidly churning out tokens.

The difference in number of tokens I use in my day job vs side projects is massive. You can see the inefficiency quantified.

Show me the incentive, I show you the result.

Re: The real prices of frontier models

#65

Post by ac29 »

iLoveOncall wrote: post #50

Very unpalatable completely LLM-written article, but on top of that a lot of the fundations and conclusions are completely wrong, the main one being this one: > You will see people claim Claude uses 2x to 4x the tokens of GPT. Our measurements do not support that, and overstating it would undercut the real point. It's not because a single prompt represents only 1.7x the number of tokens that a model doesn't use 4x as…

> You will see people claim Claude uses 2x to 4x the tokens of GPT.

My guess is something to this effect was in the prompt and the LLM made a point of correcting it

Re: The real prices of frontier models

#67

Post by dolebirchwood »

ianberdin wrote: post #58
robbie-c wrote: post #39

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides". I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it? Edit: this specific title has been deleted from the arti…

Well, criticizing is, of course, great. But the reality is that English is not my native language and I dictated most of it with my voice, then processed it with the help of AI, translated, added, corrected, and converted. It is actually a big result of work, a lot of research and attempts. And to just say that "oh, this is AI-slop," I consider unfair, but that is your choice. There is a difference: - There are peopl…

> There is a difference: - There are people who do, - And there are those who criticize.

Weird shield to hide behind, considering there are also people who can do both.

Re: The real prices of frontier models

#68

Post by ianberdin »

robbie-c wrote: post #39

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides". I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it? Edit: this specific title has been deleted from the arti…

Thank you for the criticism. I heard you. I added a TLDR. I cleaned up many AI constructions. By the way, I tweaked it a bit, compressed it.

One way or another, I want to note that yes, this text was made in collaboration with AI. My English is non-native. It helps me translate, helps me structure better. Yes, there is a downside, it can bloat the text with unnecessary words. But that, unfortunately, is the price.

But the key thing is that I tried very hard to share my many years of experience, or rather a part of it, which I acquired, with all of you. And I am very glad that this information turned out to be useful to you.

The key here is: * The information that is written in the article. * Not how it is written, but what I was trying to convey to you.

Thank you very much for reading and responding.

Re: The real prices of frontier models

#70

Post by charcircuit »

This article doesn't address the inference side cost. Not all tokens cost the same. If the response is predictable you get a few output tokens for for the price of 1. The further an output token is the cost of generating it grows linearly due to attention.
Post reply on HN