Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

tokens.billchambers.me

Post reply on HN
Print view View original

421–430 of 620 posts

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#421

Post by mohamedkoubaa »

gsleblanc wrote: post #142

It's increasingly looking naive to assume scaling LLMs is all you need to get to full white-collar worker replacement. The attention mechanism / hopfield network is fundamentally modeling only a small subset of the full human brain, and all the increasing sustained hype around bolted-on solutions for "agentic memory" is, in my opinion, glaring evidence that these SOTA transformers alone aren't sufficient even when yo…

I think they're as good as they're going to get from scaling. They can still get more efficient, and tooling/harnesses around them will improve.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#422

Post by lelanthran »

gbro3n wrote: post #344
dakiol wrote: post #295

Earlier quoted context omitted.

> I completely see your point, but when my / developer time is worth what it is compared to the cost of a frontier model subscription, I'm wary of choosing anything but the best model I can. Don't you understand that by choosing the best model we can, we are, collectively, step by step devaluating what our time is worth? Do you really think we all can keep our fancy paychecks while keep using AI?

Do you think if you or me stopped using AI that everyone else will too? We're still what we always were - problem solvers who have gained the ability to learn and understand systems better that the general population, communicate clearly (to humans and now AIs). Unfortunately our knowledge of language APIs and syntax has diminished in value, but we have so many more skills that will be just as valuable as ever. As th…

> Unfortunately our knowledge of language APIs and syntax has diminished in value, but we have so many more skills that will be just as valuable as ever.

There were always jobs that required those "many more skills" but didn't require any programming skills.

We call those people Business Analysts and you could have been doing it for decades now. You didn't, because those jobs paid half what a decent/average programmer made.

Now you are willingly jumping into that position without realising that the lag between your value (i.e. half your salary, or less) would eventually disappear.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#423

Post by matheusmoreira »

rectang wrote: post #420
matheusmoreira wrote: post #389

Earlier quoted context omitted.

It thinks less and produces less output tokens because it has forced adaptive thinking that even API users can't disable. Same adaptive thinking that was causing quality issues in Opus 4.6 not even two weeks ago. The one bcherny recommended that people disable because it'd sometimes allocate zero thinking tokens to the model. https://news.ycombinator.com/item?id=47668520 People are already complaining about low quali…

Are the benchmarks being used to measure these models biased towards completing huge and highly complex tasks, rather than ensuring correctness for less complex tasks? It seems like they're working hard to prioritize wrapping their arms around huge contexts, as opposed to handling small tasks with precision. I prefer to limit the context and the scope of the task and focus on trying to get everything right in increme…

I don't think there's a bias here. I'd say my task is of somewhat high complexity. I'm using Claude to assist me in implementing exceptions in my programming language. It's a SICP chapter 5.4 level task. There are quite a few moving parts in this thing. Opus 4.6 once went around in circles for half an hour trying to trace my interpreter's evaluator. As a human, it's not an easy task for me to do either.

I think the problem just comes down to adaptive thinking allowing the model to choose how much effort it spends on things, a power which it promptly abuses to be as lazy as possible. CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING=1 significantly improved Opus 4.6's behavior and the quality of its results. But then what do they do when they release 4.7?

https://code.claude.com/docs/en/model-config

> Opus 4.7 always uses adaptive reasoning.

> The fixed thinking budget mode and CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING do not apply to it.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#424

Post by 3dfd »

glerk wrote: post #148

I'd be ok with paying more if results were good, but it seems like Anthropic is going for the Tinder/casino intermittent reinforcement strategy: optimized to keep you spending tokens instead of achieving results. And yes, Claude models are generally more fun to use than GPT/Codex. They have a personality. They have an intuition for design/aesthetics. Vibe-coding with them feels like playing a video game. But the resu…

[dead]

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#426

Post by 3dfd »

braebo wrote: post #403
danny_codes wrote: post #251

Earlier quoted context omitted.

I just don’t see how they’ll be able to make a profit. Open models have the same performance on coding tasks now. The incentives are all wrong. Why pay more for a model that’s no better and also isn’t open? It’s nonsense

Which open model has the same performance as Opus 4.7?

They dont have to be parity today.

If the frontier models reach a point of barely any noticeable improvements the trade off changes.

You do not need a perfect substitute if you are getting it for free...

People will factor in future expectations about the development of open source vs frontier models. Why do you think OAI and anthropic are pushing hard on marketing? its for this reason. They want to get contractual commitments that firms have to honour whilst open source closes the gap.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#427

Post by 3dfd »

Bridged7756 wrote: post #233
glerk wrote: post #148

I'd be ok with paying more if results were good, but it seems like Anthropic is going for the Tinder/casino intermittent reinforcement strategy: optimized to keep you spending tokens instead of achieving results. And yes, Claude models are generally more fun to use than GPT/Codex. They have a personality. They have an intuition for design/aesthetics. Vibe-coding with them feels like playing a video game. But the resu…

Mirrors my sentiment. Those tools seem mostly useful for a Google alternative, scaffolding tedious things, code reviewing, and acting as a fancy search. It seems that they got a grip on the "coding LLM" market and now they're starting to seek actual profit. I predict we'll keep seeing 40%+ more expensive models for a marginal performance gain from now on.

[dead]

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#428

Post by 3dfd »

mohamedkoubaa wrote: post #421
gsleblanc wrote: post #142

It's increasingly looking naive to assume scaling LLMs is all you need to get to full white-collar worker replacement. The attention mechanism / hopfield network is fundamentally modeling only a small subset of the full human brain, and all the increasing sustained hype around bolted-on solutions for "agentic memory" is, in my opinion, glaring evidence that these SOTA transformers alone aren't sufficient even when yo…

I think they're as good as they're going to get from scaling. They can still get more efficient, and tooling/harnesses around them will improve.

[dead]

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#429

Post by 3dfd »

SlinkyOnStairs wrote: post #213
xixixao wrote: post #174

Earlier quoted context omitted.

This is the “ad tax” reasoning, but ultimately I think the answer is greater efficiency. So there is a real value, even if all competitors use the tools. It’s like saying clothing manufacturers are paying the “loom tax” tax when they could have been weaving by hand…

Software development is not a production line, the relationship between code output and revenue is extremely non-linear. Where producing 2x the t-shirts will get you ~2x the revenue, it's quite unlikely that 10x the code will get you even close to 2x revenue. With how much of this industry operates on 'Vendor Lock-in' there's a very real chance the multiplier ends up 0x. AI doesn't add anything when you can already 1…

[dead]

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#430

Post by codethief »

matheusmoreira wrote: post #389
andai wrote: post #239

For a fair comparison you need to look at the total cost, because 4.7 produces significantly fewer output tokens than 4.6, and seems to cost significantly less on the reasoning side as well. Here is a comparison for 4.5, 4.6 and 4.7 (Output Tokens section): https://artificialanalysis.ai/?models=claude-opus-4-7%2Cclau... 4.7 comes out slightly cheaper than 4.6. But 4.5 is about half the cost: https://artificialanalysi…

It thinks less and produces less output tokens because it has forced adaptive thinking that even API users can't disable. Same adaptive thinking that was causing quality issues in Opus 4.6 not even two weeks ago. The one bcherny recommended that people disable because it'd sometimes allocate zero thinking tokens to the model. https://news.ycombinator.com/item?id=47668520 People are already complaining about low quali…

> > Why are you handwaving things away though? I've got you on max effort. I even patched the system prompts to reduce this.

In my experience, prompts like this one, which 1) ask for a reason behind an answer (when the model won't actually be able to provide one), 2) are somewhat standoff-ish, don't work well at all. You'll just have the model go the other way.

What works much better is to tell the model to take a step back and re-evaluate. Sometimes it also helps to explicitly ask it to look at things from a different angle XYZ, in other words, to add some entropy to get it away from the local optimum it's currently at.

Post reply on HN