And what about with Caveman[1]? 1. https://github.com/juliusbrussee/caveman
isn’t caveman a joke? why would you use it for real work?
Measuring Claude 4.7's tokenizer costs
31–40 of 540 posts
Re: Measuring Claude 4.7's tokenizer costs
#32IMHO there is a point where incremental model quality will hit diminishing returns. It is like comparing an 8K display to a 16K display because at normal viewing distance, the difference is imperceptible, but 16K comes at significant premium. The same applies to intelligence. Sure, some users might register a meaningful bump, but if 99% can't tell the difference in their day-to-day work, does it matter? A 20-30% cost…
It probably depends what you're using the models for. If you use them for web search, summarizing web pages, I can imagine there's a plateau and we're probably already hitting it.
For coding though, there is kind of no limit to the complexity of software. The more invariants and potential interactions the model can be aware of, the better presumably. It can handle larger codebases. Probably past the point where humans could work on said codebases unassisted (which brings other potential problems).
Re: Measuring Claude 4.7's tokenizer costs
#33The compute is expensive, what is with this outrage? People just want free tools forever?
Re: Measuring Claude 4.7's tokenizer costs
#34I tried to do my usual test (similar to pelican but a bit more complex) but it ran out of 5 hour limit in 5 minutes. Then after 5 hours I said "go on" and the results were the worst I've ever seen.
Re: Measuring Claude 4.7's tokenizer costs
#35LLMs exist on a logaritmhic performance/cost frontier. It's not really clear whether Opus 4.5+ represent a level shift on this frontier or just inhabits place on that curve which delivers higher performance, but at rapidly diminishing returns to inference cost. To me, it is hard to reject this hypothesis today. The fact that Anthropic is rapidly trying to increase price may betray the fact that their recent lead is a…
They're also getting closer to IPO and have a growing user base. They can't justify losing a very large number of billions of other people's money in their IPO prospectus.
So there's a push for them to increase revenue per user, which brings us closer to the real cost of running these models.
Re: Measuring Claude 4.7's tokenizer costs
#36Re: Measuring Claude 4.7's tokenizer costs
#37The fundamental problem with these frontier model companies is that they're incentivized to create models that burn through more tokens, full stop. It's a tale as old as capitalism: you wake up every day and choose to deliver more value to your customers or your shareholders, you cannot do both simultaneously forever. People love to throw around "this is the dumbest AI will ever be", but the corollary to that is "thi…
> but the corollary to that is "this is the most aligned the incentives between model providers and customers will ever be" because we're all just burning VC money for now.
Please say this louder for everyone to hear. We are still at the stage where it is best for Anthropic's product to be as consumer aligned (and cost-friendly) as possible. Anthropic is loosing a lot of money. Both of those things will not be true in the near future.
Re: Measuring Claude 4.7's tokenizer costs
#38LLMs exist on a logaritmhic performance/cost frontier. It's not really clear whether Opus 4.5+ represent a level shift on this frontier or just inhabits place on that curve which delivers higher performance, but at rapidly diminishing returns to inference cost. To me, it is hard to reject this hypothesis today. The fact that Anthropic is rapidly trying to increase price may betray the fact that their recent lead is a…
I meant reference Toby Ord's work here. I think his framing of the performance/cost frontier hasn't gotten enough attention https://www.tobyord.com/writing/hourly-costs-for-ai-agents
Re: Measuring Claude 4.7's tokenizer costs
#39Re: Measuring Claude 4.7's tokenizer costs
#40Just yesterday I was happy to have gotten my weekly limit reset [1]. And although I've been doing a lot of mockup work (so a lot of HTML getting written), I think the 1M token stuff is absolutely eating up tokens like CRAZY. I'm already at 27% of my weekly limit in ONE DAY. https://news.ycombinator.com/item?id=47799256
yeah similar for me - it uses a bunch more tokens and I haven’t been able to tell the ROI in terms of better instruction following it seems to hallucinate a bit more (anecdotal)
I had it hallucinate a tool that didn't exist, it was very frustrating!