Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

tokens.billchambers.me

Post reply on HN
Print view View original

411–420 of 620 posts

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#411

Post by nixpulvis »

zaptrem wrote: post #226
tiffanyh wrote: post #52

I was using Opus 4.7 just yesterday to help implement best practices on a single page website. After just ~4 prompts I blew past my daily limit. Another ~7 more prompts & I blew past my weekly limit. The entire HTMl/CSS/JS was less than 300 lines of code. I was shocked how fast it exhausted my usage limits.

What's your reasoning effort set to? Max now uses way more tokens and isn't suggested for most usecases. Even the new default (xhigh) uses more than the old default (medium).

That's what I'm wondering. Is it people are defaulting to xhigh now and that's why it feels like it's consuming a lot more tokens? If people manually set it to medium, would it be comparable?

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#412

Post by bashtoni »

gck1 wrote: post #365
_blk wrote: post #164

Earlier quoted context omitted.

From what I understand you shouldn't wait more than 5min between prompts without compacting or clearing or you'll pay for reinitializing the cache. With compaction you still pay but it's less input tokens. (Is compaction itself free?)

Cache ttl on max subscriptions is 1h, FYI.

Only if you set `ENABLE_PROMPT_CACHING_1H`, which was mentioned in the release notes for a recent Claude Code release but doesn't seem to be in the official docs.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#413

Post by andersa »

bashtoni wrote: post #412
gck1 wrote: post #365

Earlier quoted context omitted.

Cache ttl on max subscriptions is 1h, FYI.

Only if you set `ENABLE_PROMPT_CACHING_1H`, which was mentioned in the release notes for a recent Claude Code release but doesn't seem to be in the official docs.

Bruh. It's getting hard to track down all these MAKE_IT_ACTUALLY_WORK settings that default to off for no reason.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#414

Post by bigyabai »

ACCount37 wrote: post #355
bigyabai wrote: post #317

Earlier quoted context omitted.

I can ask a nine-year-old human brain to solve that problem with a box of Crayola and a sheet of A4 printer paper. In-context learning is professedly not "good enough" to approximate continuous learning of even a child.

You're absolutely wrong! You can also ask an LLM to solve that problem by spelling the word out first. And then it'll count the letters successfully. At a similar success rate to actual nine-year-olds. There's a technical explanation for why that works, but to you, it might as well be black magic. And if you could get a modern agentic LLM that somehow still fails that test? Chances are, it would solve it with no inst…

> it's so powerful that finding "anything better" is incredibly hard.

We're back around to the start again. "Incredibly hard" is doing all of the heavy lifting in this statement, it's not all-powerful and there are enormous failure cases. Neither the human brain nor LLMs are a panacea for thought, but nobody in academia or otherwise is seriously comparing GPT to the human brain. They're distinct.

> There's a technical explanation for why that works, but to you, it might as well be black magic.

Expound however much you need. If there's one thing I've learned over the past 12 months it's that everyone is now an expert on the transformer architecture and everyone else is wrong. I'm all ears if you've got a technical argument to make, the qualitative comparison isn't convincing me.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#415

Post by colechristensen »

aerhardt wrote: post #290
colechristensen wrote: post #275

Earlier quoted context omitted.

I don't know what to tell you. It's more than a little absurd to make the qualification of being able to do something to be that the output has to be considered a great work of art for generations.

I agree that the argument starts from a reduction to the absurd. So at least we can agree that AI hasn't mastered the text medium , without further qualification? And what about my argument, further qualified, which is that I don't think it could even write as well as a good professional writer - not necessarily a generational one?

>AI hasn't mastered the text medium

I don't know what this means and I don't know what would qualify it as having "mastered" at all. Seems like a no-true-Scotsman thing where regardless there would always be someone that it couldn't actually do a thing because this and that.

>why can't they write a novel?

This is what I'm disagreeing with. I think an LLM can write a novel well enough that it's recognizably a pretty mediocre novel, no worse than the median written human novel which to be fair is pretty bad. You seem to have an unqualified bar something needs to pass before "writing a novel" is accomplished but it's not clear what that is. At the same time you're switching between the ability to do a thing and the ability to do a thing in a way that's honored as the best of the best for a century. So I don't know it kind of seems like you just don't like AI and have a different standard for it that adjusts so that it fails. This doesn't match what you'd consider some random Bob's ability to do a thing.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#416

Post by davidron »

dakiol wrote: post #292
dgellow wrote: post #153

Earlier quoted context omitted.

Another aspect I haven’t seen discussed too much is that if your competitor is 10x more productive with AI, and to stay relevant you also use AI and become 10x more productive. Does the business actually grow enough to justify the extra expense? Or are you pretty much in the same state as you were without AI, but you are both paying an AI tax to stay relevant?

Where's the evidence of competitors being 10x more productive? So far, everyone is simply bragging about how much code they have shipped last week, but that has zero relevance when it comes to productivity

I work at a 20-year-old mid-sized SaaS company. As long as the company has been around, product managers have longed for more engineers and strategies for engineers to ship features faster. As of around February, those same product managers across the org are complaining that they can't keep up with the pace at which engineers are shipping their features. This isn't just lines of code. This is the entire company trying to figure out how to help the PMs because engineers suddenly stopped being the bottleneck.

I don't know about 10x, but this could only happen if PMs suddenly got really lazy or the engineers actually got at least 1.5x faster. My gut says it's way more because we're now also consistently up to date on our dependencies and completing massive refactors we were putting off for years.

There are lots of reasons this could be the case. Quality suddenly changed, the nature of the work changed, engineers leveled up... But for this to have happened consistently across a bunch of engineering teams is quite the coincidence if not this one thing we are all talking about.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#417

Post by lelanthran »

Paradigma11 wrote: post #313
bluefirebrand wrote: post #82

Earlier quoted context omitted.

> I am learning at an incredible rate with LLMs Could you do it again without the help of an LLM? If no, then can you really claim to have learned anything?

So, you havent really learned anything from any teacher if you could not do it again without them?

> So, you havent really learned anything from any teacher if you could not do it again without them?

Well, yes?

What do you think "learning" means? If you cannot do something without the teacher, you haven't learned that thing.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#418

Post by g4cg54g54 »

bashtoni wrote: post #412
gck1 wrote: post #365

Earlier quoted context omitted.

Cache ttl on max subscriptions is 1h, FYI.

Only if you set `ENABLE_PROMPT_CACHING_1H`, which was mentioned in the release notes for a recent Claude Code release but doesn't seem to be in the official docs.

subusers supposedly get it automatic again after the fix (and now also with `DISABLE_TELEMETRY=1`)

but if you are api user you must set `ENABLE_PROMPT_CACHING_1H` as i understood

and when using your own api (via `ANTHROPIC_BASE_URL`) ensure `CLAUDE_CODE_ATTRIBUTION_HEADER=0` is set as well... https://github.com/anthropics/claude-code/issues/50085

and check out the other neckbreakers ive found pukes lots of malicious compliance by feels... :/

[BUG] new sessions will *never* hit a (full)cache #47098 https://github.com/anthropics/claude-code/issues/47098

[BUG] /clear bleeds into the next session (what also breaks cache) #47756 https://github.com/anthropics/claude-code/issues/47756

[BUG] uncachable system prompt caused by includeGitInstructions / CLAUDE_CODE_DISABLE_GIT_INSTRUCTIONS -> git status https://github.com/anthropics/claude-code/issues/47107

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#419

Post by matheusmoreira »

hgoel wrote: post #74

The bump from 4.6 to 4.7 is not very noticeable to me in improved capabilities so far, but the faster consumption of limits is very noticeable. I hit my 5 hour limit within 2 hours yesterday, initially I was trying the batched mode for a refactor but cancelled after seeing it take 30% of the limit within 5 minutes. Had to cancel and try a serial approach, consumed less (took ~50 minutes, xhigh effort, ~60% of the rem…

The most frustrating part is the quality loss caused by the forced adaptive thinking. It eats 5-10% of my Max 5x usage and churns for ten minutes, only to come back with totally untrustworthy results. It lazily hand-waves issues away in order to avoid reading my actual code and doing real reasoning work on it. Opus simply cannot be trusted if adaptive thinking is enabled.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#420

Post by rectang »

matheusmoreira wrote: post #389
andai wrote: post #239

For a fair comparison you need to look at the total cost, because 4.7 produces significantly fewer output tokens than 4.6, and seems to cost significantly less on the reasoning side as well. Here is a comparison for 4.5, 4.6 and 4.7 (Output Tokens section): https://artificialanalysis.ai/?models=claude-opus-4-7%2Cclau... 4.7 comes out slightly cheaper than 4.6. But 4.5 is about half the cost: https://artificialanalysi…

It thinks less and produces less output tokens because it has forced adaptive thinking that even API users can't disable. Same adaptive thinking that was causing quality issues in Opus 4.6 not even two weeks ago. The one bcherny recommended that people disable because it'd sometimes allocate zero thinking tokens to the model. https://news.ycombinator.com/item?id=47668520 People are already complaining about low quali…

Are the benchmarks being used to measure these models biased towards completing huge and highly complex tasks, rather than ensuring correctness for less complex tasks?

It seems like they're working hard to prioritize wrapping their arms around huge contexts, as opposed to handling small tasks with precision. I prefer to limit the context and the scope of the task and focus on trying to get everything right in incremental steps.

Post reply on HN