[dead]
Measuring Claude 4.7's tokenizer costs
501–510 of 540 posts
Re: Measuring Claude 4.7's tokenizer costs
#502Earlier quoted context omitted.
I believe that's why 90% of the focus in these firms is on coding. There is a natural difficulty ramp-up that doesn't end anytime soon: you could imagine LLMs creating a line of code, a function, a file, a library, a codebase. The problem gets harder and harder and is still economically relevant very high into the difficulty ladder. Unlike basic natural language queries which saturate difficulty early. This is also w…
> the dimensionality of LLM output that is economically relevant keeps growing linearly for coding Doubt. Yes. there was at one point it suddenly became useful to write code in a general sense. I have seen almost no improvement in department of architecting, operations and gaslighting. In fact gaslighting has gotten worse. Entire output based on wrong assumption that it hid, almost intentionally. And I had to create…
I’ve started to pick up on some of the “unwilling to dig deeply into the humans perspective” & “provide ideation and then run with it” in 4.7. I actually think it’s consistent with confabulation, now that they’ve removed most of the models ability to observe its own reasoning in 4.7.
The effect is over-complicated engineering that takes way more time to review as to its right-size for the job.
Feels like hiding things, however.
Re: Measuring Claude 4.7's tokenizer costs
#503Earlier quoted context omitted.
That post doesn't address the human factor of cost, and I don't mean that in a good way. Even if AI costs more than a human, it's tireless, doesn't need holidays, is never going to have to go to HR for sexual harassment issues, won't show up hungover or need an advance to pay for a dying relative's surgery. It can be turned on and off with the flip of a switch. Hire 30 today, fire 25 of them next week. Spin another 5…
Why do you think it can't sexually harass someone or drive people to suicide. There are already lawsuits coming in on it causing suicides. This is an architecture that people are increasing begging to give network connectivity that can't differentiate its system prompt from user input
[dead]
Re: Measuring Claude 4.7's tokenizer costs
#504I find it interesting that folks are so focused on cost for AI models. Human time spent redirecting AI coding agents towards better strategies and reviewing work, remains dramatically more expensive than the token cost for AI coding, for anything other than hobby work (where you're not paying for the human labor). $200/month is an expensive hobby, but it's negligible as a business expense; SalesForce licenses cost fa…
That. We already shipped 3 things this year built using Claude. The biggest one was porting two native apps into one react native app - which was originally estimated to be a 6-7 month project for a 9 FTE team, and ended up being a 2 months project with 2 people. To me, the economic value of a claude subscription used right is in the range of 10-40k eur, depending on the type of work and the developer driving it. If…
You copied two human coded native apps into a vibe coded react app? If the vibe coding is so good why wouldn't you keep the native apps and vibe code on top of them instead of spending a bunch of money to reach feature parity with a worse version?
Re: Measuring Claude 4.7's tokenizer costs
#505Since the Opus 4.7 release. I hit my 5 hour window limit second time on claude code max plan, which never happened before. I am not happy for sure.
Re: Measuring Claude 4.7's tokenizer costs
#506Earlier quoted context omitted.
No, that's not my point. My point is that AI looks like something fairly unique in today's landscape: a resource that almost everyone is starting to depend on. It's a bit like the Internet, except usage is metered, and paying more can improve the quality of the result for the same underlying task, such as cybersecurity. In this context I also imagine we will have greater and greater local models, and the (dependency)…
It's just computation, which the world already depended on. we're in the mainframe era, but "AI" will go personal, and on-device.
Good analogy, but there's a key difference: mainframes were an institutional dependency, whereas the world's reliance on LLMs is consumer driven, ubiquitous, an uncapped (e.g. spend more on the same "loops"). Completely agree on the second point, though powerful local models are the inevitable next step, and they are arriving fast.
Re: Measuring Claude 4.7's tokenizer costs
#507We noticed this two weeks ago where we found some of our requests are unexpected took more tokens than measured by count_tokens call. At the end they were Anthropic's A/B testing routing some Opus 4.6 calls to Opus 4.7. https://matrix.dev/blog-2026-04-16.html (We were talking to Opus 4.7 twelve days ago)
Wonder what they do for their token cache if they swap mid-session like that.
Exactly! In https://matrix.dev/blog-2026-04-04-2.html#questions-this-rai..., we raised exactly the same concerns. In particular, we actually saw that a hot swap caused a 100% cache miss. If it's a session filled with 800k tokens, rebuilding the cache is very expensive.
Also looking back at their claim: "Token counts may include tokens added automatically by Anthropic for system optimizations. You are not billed for system-added tokens. Billing reflects only your content."
A/B testing sounds a bit different. Do they really count it as "system-added tokens" and not charge for this extra cost? If you consider the model you're requesting as the baseline, then yes. But technically it's an A/B test of a different model, so they might secretly charge 130% as "we didn't add any system prompt, we just routed you to a better model."
Re: Measuring Claude 4.7's tokenizer costs
#508Earlier quoted context omitted.
That's why open models are going to win in the long run.
I think the key question is “when”? In a highly competitive business environment, companies are going to naturally be attracted to the most capable model if it leads to a competitive advantage and the switching costs are low. This suggests that “open” (giving away inference despite ever-higher training costs) may not win for a very long time, if ever.
When frontier models plateau and efficiency increases sufficiently that it becomes a commodity like other cloud compute.
One driver of open models might be foreign actors. With the entire US economy being held up by AI, it's a crucial vulnerability for a capable foreign actor (guess who) to exploit if they wanted to.
Re: Measuring Claude 4.7's tokenizer costs
#509I find it interesting that folks are so focused on cost for AI models. Human time spent redirecting AI coding agents towards better strategies and reviewing work, remains dramatically more expensive than the token cost for AI coding, for anything other than hobby work (where you're not paying for the human labor). $200/month is an expensive hobby, but it's negligible as a business expense; SalesForce licenses cost fa…
It seems far from clear at this point what the dollar value of agentic coding tools is if measured objectively in terms of value delivered.
IF they can be shown to be multiplying developer productivity (completing more projects on time, without reduction in quality and associated costs) by some significant amount then they are providing value at current cost, but it's not at all clear whether that is in fact the case, especially since most of the claims of productivity are anecdotal and/or based on things like LOC generated rather than delivered functionality.
Meta's "token usage leaderboard" shows how far some companies are from measuring anything meaningful! It'd be exactly like some company in the .com era measuring employee's "productivity" by how many bytes they'd downloaded from the internet each day (even if that was just a cat video). "Woo hoo, we're out-internetting you! Our internet bill is enormous!" (then proceeds to fire the guy coding, and gives a bonus to the one downloading cat videos).
There have been some studies/polls done indicating that some very high percentage (90%?) of corporate AI projects are failing. Why is this? Are they ill-conceived, and or ill-executed? Is it the quality of what's being produced that is causing these projects to be abandoned and/or considered as a failure?
There have also been some separate studies indicating programmer productivity to be reduced, not increased, by use of AI coding tools, which is easy to understand. The developer struggles with the tool and it's fallibilities, eventually gets it to generate something that works, then closes his JIRA story with an "AI coded" tag (which shows up on the boss's dashboard, and is all that he sees). Was this an AI productivity success story? To the boss perhaps, but not if the developer admits that it would have just been faster to do it the old way by hand or cut-n-paste from stack overflow.
Re: Measuring Claude 4.7's tokenizer costs
#510Earlier quoted context omitted.
I still haven't seen any statistically sound data supporting that this is happening on the API (per-token pricing.) If you've got something to share I'd love to see it.
There's an interesting analysis here: https://github.com/anthropics/claude-code/issues/42796 >The most striking row is user prompts: 5,608 in February vs 5,701 in March. The human put in the same effort. But the model consumed 80x more API requests and 64x more output tokens to produce demonstrably worse results.
Sorry, "this" referred to the parent comment's claim.
> models starting becoming "moody" due to their proprietors arbitrarily modifying their performance capabilities
The tokenizer changes are measurable, the above is quite difficult to quantify.
There are a few sites floating around that purport to, but all of them have fatal flaws in their methodology.