Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

tokens.billchambers.me

Post reply on HN
Print view View original

471–480 of 620 posts

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#471

Post by hattmall »

what wrote: post #456
matheusmoreira wrote: post #389

Earlier quoted context omitted.

It thinks less and produces less output tokens because it has forced adaptive thinking that even API users can't disable. Same adaptive thinking that was causing quality issues in Opus 4.6 not even two weeks ago. The one bcherny recommended that people disable because it'd sometimes allocate zero thinking tokens to the model. https://news.ycombinator.com/item?id=47668520 People are already complaining about low quali…

> Why are you handwaving things away though? I've got you on max effort. I even patched the system prompts to reduce this. Do you think it knows what max effort or patched system prompts are? It feels really weird to talk to an LLM like it’s a person that understands.

I thought that was really weird as well.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#472

Post by [deleted] »

Syzygies wrote: post #468

I'm a retired mathematician hoping to finish a second proof of a major theorem before I die. AI needs to understand my math and help me code. What I spend on AI isn't going to deplete my retirement savings. So far, Opus 4.7 seems a bit smarter than Opus 4.6 for my use case. That's my only concern. Is an $80 bottle of wine a better value than a $20 or $40 bottle of wine? Pretty much never. If there are those of us wil…

[deleted]

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#473

Post by sutterd »

matheusmoreira wrote: post #419
hgoel wrote: post #74

The bump from 4.6 to 4.7 is not very noticeable to me in improved capabilities so far, but the faster consumption of limits is very noticeable. I hit my 5 hour limit within 2 hours yesterday, initially I was trying the batched mode for a refactor but cancelled after seeing it take 30% of the limit within 5 minutes. Had to cancel and try a serial approach, consumed less (took ~50 minutes, xhigh effort, ~60% of the rem…

The most frustrating part is the quality loss caused by the forced adaptive thinking. It eats 5-10% of my Max 5x usage and churns for ten minutes, only to come back with totally untrustworthy results. It lazily hand-waves issues away in order to avoid reading my actual code and doing real reasoning work on it. Opus simply cannot be trusted if adaptive thinking is enabled.

You don't have to use adaptive thinking. It had been turned off on my main work computer. I was using a different computer on a trip and I started getting so angry at Claude for doing a bad job. I evetually figured out it was adaptive thinking and set it to "hard" and it started working again. At the time I think "hard" was the top choice. With 4.7, my computer now shows "xhard", which I assume is the equivelent setting. There is one higher setting than this, which I haven't tried yet. I would tell you how to change these settings, but I don't remember. By the way, I have been happy with 4.7 so far. I actually did not like 4.6 and preferred 4.5 and used that most of the time until this new release.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#474

Post by pitched »

DeathArrow wrote: post #243
pitched wrote: post #127

Earlier quoted context omitted.

For a business with ten or more engineers/people-using-ai, it might still make sense to set this up. For an individual though, I can’t imagine you’d make it through to positive ROI before the hardware ages out.

Since you need at least a few of H100 class hardware, I guess you need at least few tens of coders to justify the costs.

I see the 512GB Mac Studios aren’t for sale anymore but that was a much cheaper path

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#475

Post by the_gipsy »

holoduke wrote: post #395
glerk wrote: post #148

I'd be ok with paying more if results were good, but it seems like Anthropic is going for the Tinder/casino intermittent reinforcement strategy: optimized to keep you spending tokens instead of achieving results. And yes, Claude models are generally more fun to use than GPT/Codex. They have a personality. They have an intuition for design/aesthetics. Vibe-coding with them feels like playing a video game. But the resu…

You have to guide an ai. Not let roam freely. If you got skills to guide you can make it output high quality

> skills to guide

Is that what the soul is?

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#476

Post by sroussey »

nelox wrote: post #446
codethief wrote: post #430

Earlier quoted context omitted.

> > Why are you handwaving things away though? I've got you on max effort. I even patched the system prompts to reduce this. In my experience, prompts like this one, which 1) ask for a reason behind an answer (when the model won't actually be able to provide one), 2) are somewhat standoff-ish, don't work well at all. You'll just have the model go the other way. What works much better is to tell the model to take a st…

Precisely. I find Grok’s multi-agent approach very useful here. I have custom agent configured as a validator.

Do you have to use Grok? I don't anyhow that found it passed evaluations.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#477

Post by plaguuuuuu »

bashtoni wrote: post #412
gck1 wrote: post #365

Earlier quoted context omitted.

Cache ttl on max subscriptions is 1h, FYI.

Only if you set `ENABLE_PROMPT_CACHING_1H`, which was mentioned in the release notes for a recent Claude Code release but doesn't seem to be in the official docs.

no way, I didn't realise this worked.

My attention span is such that I get side tracked and wind up taking longer than 5 mins quite a bit :D

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#478

Post by copperx »

templar_snow wrote: post #284
copperx wrote: post #139

Earlier quoted context omitted.

I haven't seen a noticeable difference BUT I've been always using the context mode plugin.

You mean this? https://github.com/mksglu/context-mode Is it actually good or is this an ad?

correct. ad? it's not a paid product afaik.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#479

Post by noodletheworld »

codethief wrote: post #430
matheusmoreira wrote: post #389

Earlier quoted context omitted.

It thinks less and produces less output tokens because it has forced adaptive thinking that even API users can't disable. Same adaptive thinking that was causing quality issues in Opus 4.6 not even two weeks ago. The one bcherny recommended that people disable because it'd sometimes allocate zero thinking tokens to the model. https://news.ycombinator.com/item?id=47668520 People are already complaining about low quali…

> > Why are you handwaving things away though? I've got you on max effort. I even patched the system prompts to reduce this. In my experience, prompts like this one, which 1) ask for a reason behind an answer (when the model won't actually be able to provide one), 2) are somewhat standoff-ish, don't work well at all. You'll just have the model go the other way. What works much better is to tell the model to take a st…

> What works much better is to tell the model to take a step back and re-evaluate.

I desperately hate that modern tooling relies on “did you perform the correct prayer to the Omnissiah”

> to add some entropy to get it away from the local optimum

Is that what it does? I don't think thats what it does, technically.

I think thats just anthropomorphizing a system that behaves in a non deterministic way.

A more menaingful solution is almost always “do it multiple times”.

That is a solution that makes sense sometimes because the system is prob based, but even then, when youre hitting an opaque api which has multiple hidden caching layers, /shrug who knows.

This is way I firmly believing prompt engineering and prompt hacking is just fluff.

Its both mostly technically meaningless (observing random variance over a sample so small you cant see actual patterns) and obsolete once models/apis change.

Just ask Claude to rewrite your request “as a prompt for claude code” and use that.

I bet it wont be any worse than the prompt you write by hand.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#480

Post by xpe »

Bridged7756 wrote: post #233
glerk wrote: post #148

I'd be ok with paying more if results were good, but it seems like Anthropic is going for the Tinder/casino intermittent reinforcement strategy: optimized to keep you spending tokens instead of achieving results. And yes, Claude models are generally more fun to use than GPT/Codex. They have a personality. They have an intuition for design/aesthetics. Vibe-coding with them feels like playing a video game. But the resu…

Mirrors my sentiment. Those tools seem mostly useful for a Google alternative, scaffolding tedious things, code reviewing, and acting as a fancy search. It seems that they got a grip on the "coding LLM" market and now they're starting to seek actual profit. I predict we'll keep seeing 40%+ more expensive models for a marginal performance gain from now on.

> Those tools seem mostly useful for a Google alternative, scaffolding tedious things, code reviewing, and acting as a fancy search.

Just to get a sense for the rate of change, imagine if you took a survey. Compare what people said about AI tools... 3 years ago, 2 years ago, 1 year ago, 6 months ago. Then think about what is plausible that people will be saying in 3 months, 6 months, 9 months ...

Moving the goalposts has always happened, but it is happening faster than I've ever seen it. Many people seem to redefine their expectations on a monthly basis now. Worse, they seem to be unaware they are doing it.

Fancy search? Ok, I'll bite. Compare today's "fancy search" to what we had ~3 years ago according to your choice of metric. Here's one: minutes spent relative to information found. Today, in ~5 minutes I can do a literature review that would have taken me easily 10+ hours five years ago. We don't need to argue phrasing when we can pick some prototypical tasks and compare them.

We're going to have different takes about where various AI technologies will be in these future timelines. It is much better to run to where the ball is likely to be, even if we have different ideas of where that is.

The human brain, at best, struggles to grasp even linear change. But linear change is not a good way to predict compounding technological change.

Post reply on HN