Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Measuring Claude 4.7's tokenizer costs

claudecodecamp.com

Post reply on HN
Print view View original

291–300 of 540 posts

Re: Measuring Claude 4.7's tokenizer costs

#291

Post by aulin »

ericol wrote: post #266

I did some work yesterday with Opus and found it amazing. Today we are almost on non-speaking terms. I'm asking it to do some simple stuff and he's making incredible stupid mistakes: This is the third time that I have to ask you to remove the issue that was there for more than 20 hours. What is going on here? and at the same time the compacting is firing like crazy. (What adds ~4 minute delays every 1 - 15 minutes) |…

They won't. These are not "issues", it's them trying to push the models to burn less compute. It will only get worse.

Re: Measuring Claude 4.7's tokenizer costs

#292

Post by iainmerrick »

louiereederson wrote: post #10

LLMs exist on a logaritmhic performance/cost frontier. It's not really clear whether Opus 4.5+ represent a level shift on this frontier or just inhabits place on that curve which delivers higher performance, but at rapidly diminishing returns to inference cost. To me, it is hard to reject this hypothesis today. The fact that Anthropic is rapidly trying to increase price may betray the fact that their recent lead is a…

That sounds very plausible. But it implies they could offer even higher performance models at much higher costs if they chose to; and presumably they would if there were customers willing to pay. Is that the case? Surely there are a decent number of customers who’d be willing to pay more, much more, to get the very best LLMs possible.

Like, Apple computers are already quite pricey -- $1000 or $2000 or so for a decent one. But you can spec up one that’s a bit better (not really that much better) and they’ll charge you $10K, $20K, $30K. Some customers want that and many are willing to pay for it.

Is there an equivalent ultra-high-end LLM you can have if you’re willing to pay? Or does it not exist because it would cost too much to train?

Re: Measuring Claude 4.7's tokenizer costs

#293

Post by criemen »

aulin wrote: post #291
ericol wrote: post #266

I did some work yesterday with Opus and found it amazing. Today we are almost on non-speaking terms. I'm asking it to do some simple stuff and he's making incredible stupid mistakes: This is the third time that I have to ask you to remove the issue that was there for more than 20 hours. What is going on here? and at the same time the compacting is firing like crazy. (What adds ~4 minute delays every 1 - 15 minutes) |…

They won't. These are not "issues", it's them trying to push the models to burn less compute. It will only get worse.

> it's them trying to push the models to burn less compute

I'm curious, how does using more tokens save compute?

Re: Measuring Claude 4.7's tokenizer costs

#294

Post by bornfreddy »

lbreakjai wrote: post #281
fragmede wrote: post #200

Earlier quoted context omitted.

That post doesn't address the human factor of cost, and I don't mean that in a good way. Even if AI costs more than a human, it's tireless, doesn't need holidays, is never going to have to go to HR for sexual harassment issues, won't show up hungover or need an advance to pay for a dying relative's surgery. It can be turned on and off with the flip of a switch. Hire 30 today, fire 25 of them next week. Spin another 5…

The cost to hire a human is highly predictable. The cost of AI isn't. I, as a human, need food and shelter, which puts a ceiling to my bargaining power. I can't withdraw my labour indefinitely. The power dynamics are also vastly against me. I represent a fraction of my employer's labour, but my employer represents 100% of my income. That dynamic is totally inverted with AI. You are a rounding error on their revenue s…

By continuously testing competitors and local LLMs? The reason for rising prices is that they (Anthropic) probably realized that they have reached a ceiling of what LLMs are capable of, and while it's a lot, it is still not a big moat and it's definitely not intelligence.

Re: Measuring Claude 4.7's tokenizer costs

#295

Post by zer00eyz »

lbreakjai wrote: post #281
fragmede wrote: post #200

Earlier quoted context omitted.

That post doesn't address the human factor of cost, and I don't mean that in a good way. Even if AI costs more than a human, it's tireless, doesn't need holidays, is never going to have to go to HR for sexual harassment issues, won't show up hungover or need an advance to pay for a dying relative's surgery. It can be turned on and off with the flip of a switch. Hire 30 today, fire 25 of them next week. Spin another 5…

The cost to hire a human is highly predictable. The cost of AI isn't. I, as a human, need food and shelter, which puts a ceiling to my bargaining power. I can't withdraw my labour indefinitely. The power dynamics are also vastly against me. I represent a fraction of my employer's labour, but my employer represents 100% of my income. That dynamic is totally inverted with AI. You are a rounding error on their revenue s…

> The cost of AI isn't.

This is why there are a ton of corps running the open source models in house... Known costs, known performance, upgrade as you see fit. The consumer backlash against 4o was noted by a few orgs, and they saw the writing on the wall... they didnt want to develop against a platform built on quicksand (see openweb, apps on Facebook and a host of other examples).

There are people out there making smart AI business decisions, to have control over performance and costs.

Re: Measuring Claude 4.7's tokenizer costs

#296

Post by criemen »

iainmerrick wrote: post #292
louiereederson wrote: post #10

LLMs exist on a logaritmhic performance/cost frontier. It's not really clear whether Opus 4.5+ represent a level shift on this frontier or just inhabits place on that curve which delivers higher performance, but at rapidly diminishing returns to inference cost. To me, it is hard to reject this hypothesis today. The fact that Anthropic is rapidly trying to increase price may betray the fact that their recent lead is a…

That sounds very plausible. But it implies they could offer even higher performance models at much higher costs if they chose to; and presumably they would if there were customers willing to pay. Is that the case? Surely there are a decent number of customers who’d be willing to pay more, much more, to get the very best LLMs possible. Like, Apple computers are already quite pricey -- $1000 or $2000 or so for a decent…

> Is there an equivalent ultra-high-end LLM you can have if you’re willing to pay? Or does it not exist because it would cost too much to train?

I guess at the time that was GPT-4.5. I don't think people used it a lot because it was crazy expensive, and not that much better than the rest of the crop.

Re: Measuring Claude 4.7's tokenizer costs

#297

Post by TomGarden »

Asked Opus 4.7 to extend an existing system today. After thorough exploration and a long back and forth on details it came up with a plan. Then proceeded to build a fully parallel, incompatible system from scratch with the changes I wanted but everything else incompatible and full of placeholders

Re: Measuring Claude 4.7's tokenizer costs

#298

Post by aulin »

speedgoose wrote: post #103

The "multiplier" on Github Copilot went from 3 to 7.5. Nice to see that it is actually only 20-30% and Microsoft wanting to lose money slightly slower. https://docs.github.com/fr/copilot/reference/ai-models/suppo...

Opus 4.6 also just got dumber. It's dismissive, hand-wavy, jumps to conclusions way too quickly, skips reasoning... Bubble is going to burst, either some big breakthrough comes up or we are going to see a very fast enshittificafion.

Re: Measuring Claude 4.7's tokenizer costs

#299

Post by threecheese »

_fizz_buzz_ wrote: post #272
Aurornis wrote: post #160

Earlier quoted context omitted.

I've also been using the Qwen3.5-27B and the new Qwen3.6 locally, both at Q6. I don't agree that they're as good as pre-Opus Claude. I really like how much they can do on my local hardware, but we have a long way to go before we reach parity with even the pre-Opus Claude in my opinion.

What hardware do you use? I want to experiment with running models locally.

OP’s Qwen3.6 27B Q6 seems to run north of 20GB on huggingface, and should function on an Apple Silicon with 32GB RAM. Smaller models work unreasonably well even on my M1/64GB MacBook.

I am getting 10tok/sec on a 27B of Qwen3.5 (thinking, Q4, 18GB) on an M4/32GB Mac Mini. It’s slow.

For a 9B (much smaller, non-thinking) I am getting 30tok/sec, which is fast enough for regular use if you need something from the training data (like how to use grep or Hemingways favorite cocktail).

I’m using LMStudio, which is very easy and free (beer).

Re: Measuring Claude 4.7's tokenizer costs

#300

Post by whalesalad »

smt88 wrote: post #289
al_borland wrote: post #274

Earlier quoted context omitted.

This seems like the experience I've had with every model I've tried over the last several years. It seems like an inherent limitation of the technology, despite the hyperbolic claims of those financially invested in all of this paying off.

Opus 4.6 pre-nerf was incredible, almost magical. It changed my understanding of how good models could be. But that's the only model that ever made me feel that way.

Yes! I genuinely got a LOT of shit done with Opus 4.6 "pre nerf" with regular old out-of-the-box config, no crazy skills or hacks or memory tweaks or anything. The downfall is palpable. Textbook rugpull.
Post reply on HN