Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The real prices of frontier models

playcode.io

Post reply on HN
Print view View original

51–60 of 91 posts

Re: The real prices of frontier models

#51

Post by recursivecaveat »

onlyrealcuzzo wrote: post #19
Tiberium wrote: post #8

Yeah, Anthropic's current tokenizer in Sonnet 5/Opus 4.8/Fable 5 is much worse than OpenAI's. Also, OpenAI has been using their current o200k_base from the day GPT-4o came out over two years ago. Just a few of my own tests: - A ~2000-2002 legacy C++ game codebase at about ~90kloc: GPT 1.12M, Claude 2.2M - A ~30kloc TypeScript codebase: GPT 260K, Claude 437K In the end, GPT's current tokenizer is ~1.6x-2x better than…

Interesting... Naively I'd assume you'd have a pretty unfair advantage on quality if you have materially more information dense tokens. That doesn't really appear to be the case as GPT and Anthropic models appear evenly matched despite Anthropic encoding the same text into almost ~2x the tokens... I'd also - naively - assume this would make training their models more expensive. Though inference now dominates, and the…

If a given paragraph gets encoded into twice as many tokens, that means the model gets twice as many matmuls to process it. The amount of compute thrown at the problem is increased (everything else constant), which may improve the quality of the result. This is believed to be one of the reasons that 'thinking' tokens improve quality. For long tasks it will lead to more context compactions though which will harm the quality to some degree as well.

Re: The real prices of frontier models

#52

Post by iainmerrick »

robbie-c wrote: post #39

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides". I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it? Edit: this specific title has been deleted from the arti…

I’m normally one to complain about people complaining about these LLM-isms, but yeah, this one really grates on you.

It’s a shame because it’s making an excellent point! It just takes so long to get to the point that the reader loses the will to live.

Yes, I could probably ask an LLM to summarise it for me. No, I’m not going to. I would prefer the author just take care of that for me.

Re: The real prices of frontier models

#53

Post by esafak »

robbie-c wrote: post #39

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides". I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it? Edit: this specific title has been deleted from the arti…

We need to make attribution standard. It's a lie to pretend you wrote something you merely prompted.

Re: The real prices of frontier models

#54

Post by jascha_eng »

robbie-c wrote: post #39

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides". I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it? Edit: this specific title has been deleted from the arti…

Aside of the claudeisms and the obvious AI smell, it overexplains everything and doesn't come to any useful conclusions. It's just not a good post.

The nudge to think about both "tokenization as variable" as well as actual tokens consumed per task is still good.

Re: The real prices of frontier models

#55

Post by sisve »

iainmerrick wrote: post #52
robbie-c wrote: post #39

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides". I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it? Edit: this specific title has been deleted from the arti…

I’m normally one to complain about people complaining about these LLM-isms, but yeah, this one really grates on you. It’s a shame because it’s making an excellent point! It just takes so long to get to the point that the reader loses the will to live. Yes, I could probably ask an LLM to summarise it for me. No, I’m not going to. I would prefer the author just take care of that for me.

Yeah. To often now a days there are articles with really good points, but they are just so verbose and clearly AI generated.

I could live with ai content if it was short and to the point. But it's always so lengthy. Hope that will change.

A tl;dr section at the top and then the long read from ai could also be OK if they marked it.

Re: The real prices of frontier models

#56

Post by jascha_eng »

nullsanity wrote: post #26
not-a-llm wrote: post #23

Earlier quoted context omitted.

you use the wrong word the Anthropic tokenizer is not worse, its more expensive/verbose

So, worse? Because we benchmark off token use when talking about token use, and everyone else understood that.

I mean it might lead to better performance on the model side. So the tokenizer is better but more expensive.

Re: The real prices of frontier models

#57

Post by conception »

robbie-c wrote: post #39

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides". I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it? Edit: this specific title has been deleted from the arti…

A problem is AI by default is not very good at anything. It’s pretty mediocre. With a good harness and a lot of prompting/context - you can get it to spit something out that’s pretty good. Coders have been learning and fighting this fight for a couple of years now.

The issue is that it’s not just code - they suck at writing. Really bad. Unreadable, incoherent, messy.

Humans are also bad at judging the quality of things they themselves aren’t very good at. So a senior swe sees what claude spits out and says “This is trash.” And spends x amount of time getting it to not be trash. And Jr dev thinks “this is magic!” And pushes it to a PR.

So my theory is the people “writing” this AI slop think its great! But actually just aren’t very good at writing copy and don’t have the skill to recognize it and prompt their way out of it.

Or they don’t care. That’s an option as well.

PS for anyone reading, next time AI does something that you aren’t super familiar with that looks pretty good… maybe find an expert to review it.

Re: The real prices of frontier models

#58

Post by ianberdin »

robbie-c wrote: post #39

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides". I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it? Edit: this specific title has been deleted from the arti…

Well, criticizing is, of course, great. But the reality is that English is not my native language and I dictated most of it with my voice, then processed it with the help of AI, translated, added, corrected, and converted.

It is actually a big result of work, a lot of research and attempts. And to just say that "oh, this is AI-slop," I consider unfair, but that is your choice.

There is a difference: - There are people who do, - And there are those who criticize.

Re: The real prices of frontier models

#59

Post by smusamashah »

ianberdin wrote: post #58
robbie-c wrote: post #39

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides". I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it? Edit: this specific title has been deleted from the arti…

Well, criticizing is, of course, great. But the reality is that English is not my native language and I dictated most of it with my voice, then processed it with the help of AI, translated, added, corrected, and converted. It is actually a big result of work, a lot of research and attempts. And to just say that "oh, this is AI-slop," I consider unfair, but that is your choice. There is a difference: - There are peopl…

Instead of getting offended by a fair criticism you should learn from it. In your articles consider adding a disclaimer that says exactly what you just said here in your comment here that you post-processed your voice and thoughts through LLM.

LLM speak is like the new corporate speak. Enterprise writing is fulll of fluff and nothings and they all read the same. That sameness is what most readers here are sick of.

(Your comment here that I replied to is also written by AI which is even more sad :| )

Re: The real prices of frontier models

#60

Post by SecretDreams »

iainmerrick wrote: post #52
robbie-c wrote: post #39

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides". I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it? Edit: this specific title has been deleted from the arti…

I’m normally one to complain about people complaining about these LLM-isms, but yeah, this one really grates on you. It’s a shame because it’s making an excellent point! It just takes so long to get to the point that the reader loses the will to live. Yes, I could probably ask an LLM to summarise it for me. No, I’m not going to. I would prefer the author just take care of that for me.

> that the reader loses the will to live.

My general vibe hearing about AI.

Post reply on HN