Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Measuring Claude 4.7's tokenizer costs

claudecodecamp.com

Post reply on HN
Print view View original

71–80 of 540 posts

Re: Measuring Claude 4.7's tokenizer costs

#71

Post by ZeroCool2u »

_pdp_ wrote: post #29

IMHO there is a point where incremental model quality will hit diminishing returns. It is like comparing an 8K display to a 16K display because at normal viewing distance, the difference is imperceptible, but 16K comes at significant premium. The same applies to intelligence. Sure, some users might register a meaningful bump, but if 99% can't tell the difference in their day-to-day work, does it matter? A 20-30% cost…

Whenever we get the locally runnable 4k models things are going to get really awkward for the big 3 labs. Well at least Google will still have their ad revenue I guess.

Re: Measuring Claude 4.7's tokenizer costs

#72

Post by WarmWash »

devmor wrote: post #58
giwook wrote: post #52

Earlier quoted context omitted.

I agree, and I'm also quite skeptical that Anthropic will be able to remain true to its initial, noble mission statement of acting for the global good once they IPO. At that point you are beholden to your shareholders and no longer can eschew profit in favor of ethics. Unfortunately, I think this is the beginning of the end of Anthropic and Modei being a company and CEO you could actually get behind and believe that…

Skeptical is a light way to put it. It is essentially a forgone conclusion that once a company IPOs, any veil that they might be working for the global good is entirely lifted. A publicly traded company is legally obligated to go against the global good.

The problem is that people equate money to power and power to evil.

So no matter what, if you do something lots of people like (and hence compensate you for), you will be evil.

It's a very interesting quirk of human intuition.

Re: Measuring Claude 4.7's tokenizer costs

#73

Post by sysmax »

Well, LLMs are priced per token, and most of the tokens are just echoing back the old code with minimal changes. So, a lot of the cost is actually paying for the LLM to echo back the same code.

Except, it's not that trivial to solve. I tried experimenting with asking the model to first give a list of symbols it will modify, and then just write the modified symbols. The results were OK, but less refined than when it echoes back the entire file.

The way I see it is that when you echo back the entire file, the process of thinking "should I do an edit here" is distributed over a longer span, so it has more room to make a good decision. Like instead of asking "which 2 of the 10 functions should you change" you're asking it "should you change method1? what about method2? what about method3?", etc., and that puts less pressure on the LLM.

Except, currently we are effectively paying for the LLM to make that decision for *every token*, which is terribly inefficient. So, there has to be some middle ground between expensively echoing back thousands of unchanged tokens and giving an error-ridden high-level summary. We just haven't found that middle ground yet.

Re: Measuring Claude 4.7's tokenizer costs

#74

Post by gruez »

sysmax wrote: post #73

Well, LLMs are priced per token, and most of the tokens are just echoing back the old code with minimal changes. So, a lot of the cost is actually paying for the LLM to echo back the same code. Except, it's not that trivial to solve. I tried experimenting with asking the model to first give a list of symbols it will modify, and then just write the modified symbols. The results were OK, but less refined than when it e…

>and most of the tokens are just echoing back the old code with minimal changes

I thought coding harnesses provided tools to apply diffs so the LLM didn't have to echo back the entire file?

Re: Measuring Claude 4.7's tokenizer costs

#76

Post by mmastrac »

sysmax wrote: post #73

Well, LLMs are priced per token, and most of the tokens are just echoing back the old code with minimal changes. So, a lot of the cost is actually paying for the LLM to echo back the same code. Except, it's not that trivial to solve. I tried experimenting with asking the model to first give a list of symbols it will modify, and then just write the modified symbols. The results were OK, but less refined than when it e…

I think the ideal way for these LLMs to work will be using AST-level changes instead of "let me edit this file".

grit.io was working on this years ago, not sure if they are still alive/around, but I liked their approach (just had a very buggy transformer/language).

Re: Measuring Claude 4.7's tokenizer costs

#77

Post by hleszek »

beej71 wrote: post #65

News like this always makes me wonder about running my own model, something I've never done. A couple thousand bucks can get you some decent hardware, it looks like, but is it good for coding? What is your all's experience? And if it's not good enough for coding, what kind of money, if any, would make it good enough?

The latest Qwen3.6 model is very impressive for its size. Get an RTX 3090 and go to https://www.reddit.com/r/LocalLLaMA/ to see the latest news on how to run models locally. Totally fine for coding.

Re: Measuring Claude 4.7's tokenizer costs

#78

Post by testbjjl »

estearum wrote: post #69
taosx wrote: post #68

Claude seems so frustrating lately to the point where I avoid and completely ignore it. I can't identify a single cause but I believe it's mostly the self-righteousness and leadership that drive all the decisions that make me distrust and disengage with it.

using dumber models to own the libs

Definitely experimenting with less expensive ones. I have a few versions of my settings.json

I also wonder if token utilization has or will ever find its way to employee performance reviews as these models go up in price.

Re: Measuring Claude 4.7's tokenizer costs

#80

Post by cbm-vic-20 »

atonse wrote: post #14

Just yesterday I was happy to have gotten my weekly limit reset [1]. And although I've been doing a lot of mockup work (so a lot of HTML getting written), I think the 1M token stuff is absolutely eating up tokens like CRAZY. I'm already at 27% of my weekly limit in ONE DAY. https://news.ycombinator.com/item?id=47799256

Four day workweek!
Post reply on HN