Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Measuring Claude 4.7's tokenizer costs

claudecodecamp.com

Post reply on HN
Print view View original

461–470 of 540 posts

Re: Measuring Claude 4.7's tokenizer costs

#461

Post by jeffreygoesto »

skissane wrote: post #412
noosphr wrote: post #409

Earlier quoted context omitted.

Anything but the simplest tooling is not transferable between model generations, let alone completely different families.

> Anything but the simplest tooling is not transferable between model generations, let alone completely different families. It is transferable-yes, you will get issues if you take prompts and workflows tuned for one model and send them to another unchanged. But, most of the time, fixing it is just tinkering with some prompt templates People port solutions between models all the time. It takes some work, but the amoun…

Ha. Sounds a lot like the one 10x vs. predictable mediocre guys with a scaffolding of processes. Aim high and hit or miss or try to grind predictably and continuously. Same with humans and depends on the loss you can afford.

Re: Measuring Claude 4.7's tokenizer costs

#462

Post by hiroboto »

wsun19 wrote: post #381
what wrote: post #358

Earlier quoted context omitted.

No, they still have to act in the interest of shareholders even if they have no voting power.

As a PBC, the intent of the company is not only profit, but it's hard to analyze the counterfactuals of if Anthropic were a pure for-profit or a non-profit

thats the benefit of a pbc

Re: Measuring Claude 4.7's tokenizer costs

#463

Post by wiz21c »

abustamam wrote: post #373
wg0 wrote: post #360

Earlier quoted context omitted.

Who's going to review that output for accuracy? We'll leave performance and security as unnecessary luxuries in this age and time. In my experience, even Claude 4.6's output can't be trusted blindly it'll write flawed code and would write tests that would be testing that flawed code giving false sense of confidence and accomplishment only to be revealed upon closer inspection later. Additionally - it's age old known…

Well it's all tradeoffs, right? 6 months for 9 FTEs is 54 man months. 2 months for 2 FTEs is 4 man months. Even if one FTE spent two extra months perusing every line of code and reviewing, that's still 6 man months, resulting in almost 10x speed. Let's say you dont review. Those two extra months probably turns into four extra months of finding bugs and stuff. Still 8 man months vs 54. Of course this is all assuming t…

I do greenfield in fluid dynamics and Claude doesn't help: I need to be able to justify each line of my code (the physics part) and using Claude doesn't help.

On the UI side Claude helps a lot. So for me I'd say I have a 25% productivity increment. I work like this: I put the main architecture of the code in place by hand, to get a "feel" for it. Once that is done, I ask Claude to make incremental changes, review them. Very often, Claude does an OK job.

What I have hard times with is to have Claude automatically understand my class architectures: more often than not it tries to guess information about objects in the app by querying the GUI instead of the data model. Odd.

Re: Measuring Claude 4.7's tokenizer costs

#464

Post by dns_snek »

solenoid0937 wrote: post #159
cbg0 wrote: post #15

Earlier quoted context omitted.

That performance monitor is super easy to game if you cache responses to all the SWE bench questions.

You dramatically overestimate how much time engineers at hypergrowth startups have on their hands

There's a direct business incentive to game/cheat benchmarks, it wouldn't even be difficult to do, and besides, they have workforce-replacing AI to do it for them.

Re: Measuring Claude 4.7's tokenizer costs

#467

Post by jstummbillig »

uberman wrote: post #2

On actual code, I see what you see a 30% increase in tokens which is in-line with what they claim as well. I personally don't tend to feed technical documentation or random pros into llms. Given that Opus 4.6 and even Sonnet 4.6 are still valid options, for me the question is not "Does 4.7 cost more than claimed?" but "What capabilities does 4.7 give me that 4.6 did not?" Yesterday 4.6 was a great option and it is to…

I don't understand how people measure how much more or less work they need to do. It's not that gpt-4o was incapable of exuding enormous amounts of code quickly, it's that the tokens were relativ garbage.

How do you have an opinion on 4.6/4.7 here? It's less clear but I could totally see that 4.7 or beyond leads to project completion 20% faster, by removing dead ends, foot guns, less backtracking, etc.

How to tell / measure effectively? No clue.

Re: Measuring Claude 4.7's tokenizer costs

#468

Post by greatgib »

What annoys me the most with the proprietary side of Gemini and Claude is that you used to have the tokenizer (standard) and open sourced. So you could understand what was going on, how the model would understand/split the tokens. Now it is trade secret only usable through the api!

Re: Measuring Claude 4.7's tokenizer costs

#469

Post by zarzavat »

Aperocky wrote: post #301
highfrequency wrote: post #194

Earlier quoted context omitted.

I believe that's why 90% of the focus in these firms is on coding. There is a natural difficulty ramp-up that doesn't end anytime soon: you could imagine LLMs creating a line of code, a function, a file, a library, a codebase. The problem gets harder and harder and is still economically relevant very high into the difficulty ladder. Unlike basic natural language queries which saturate difficulty early. This is also w…

> the dimensionality of LLM output that is economically relevant keeps growing linearly for coding Doubt. Yes. there was at one point it suddenly became useful to write code in a general sense. I have seen almost no improvement in department of architecting, operations and gaslighting. In fact gaslighting has gotten worse. Entire output based on wrong assumption that it hid, almost intentionally. And I had to create…

Agreed. The proprietary nature of these tools is a huge impediment to their usefulness.

A intelligence plateau will happen sooner or later (my bet is on sooner), and when it does the open models will catch up. And everybody will be using open models and open source agents because they're so much more flexible.

Re: Measuring Claude 4.7's tokenizer costs

#470

Post by bilekas »

> The model will not silently generalize an instruction from one item to another.

I am clearly missing something but wouldn't this be an ideal thing to do? Surely if it was optimised it would use less tokens while not losing anything from the instructions ?

Post reply on HN