Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

tokens.billchambers.me

Post reply on HN
Print view View original

431–440 of 620 posts

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#431

Post by matheusmoreira »

codethief wrote: post #430
matheusmoreira wrote: post #389

Earlier quoted context omitted.

It thinks less and produces less output tokens because it has forced adaptive thinking that even API users can't disable. Same adaptive thinking that was causing quality issues in Opus 4.6 not even two weeks ago. The one bcherny recommended that people disable because it'd sometimes allocate zero thinking tokens to the model. https://news.ycombinator.com/item?id=47668520 People are already complaining about low quali…

> > Why are you handwaving things away though? I've got you on max effort. I even patched the system prompts to reduce this. In my experience, prompts like this one, which 1) ask for a reason behind an answer (when the model won't actually be able to provide one), 2) are somewhat standoff-ish, don't work well at all. You'll just have the model go the other way. What works much better is to tell the model to take a st…

That's good advice. I managed to get the session back on track by doing that a few turns later. I started making it very explicit that I wanted it to really think things through. It kept asking me for permission to do things, I had to explicitly prompt it to trace through and resolve every single edge case it ran into, but it seems to be doing better now. It's running a lot of adversarial tests right now and the results at least seem to be more thorough and acceptable. It's gonna take a while to fully review the output though.

It's just that Opus 4.6 DISABLE_ADAPTIVE_THINKING=1 doesn't seem to require me to do this at all, or at least not as often. It'd fully explore the code and take into account all the edge cases and caveats without any explicit prompting from me. It's a really frustrating experience to watch Anthropic's flagship subscription-only model burn my tokens only to end up lazily hand-waving away hard questions unless I explicitly tell it not to do that.

I have to give it to Opus 4.7 though: it recovered much better than 4.6.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#432

Post by benjiro3000 »

rectang wrote: post #68

For now, I'm planning to stick with Opus 4.5 as a driver in VSCode Copilot. My workflow is to give the agent pretty fine-grained instructions, and I'm always fighting agents that insist on doing too much. Opus 4.5 is the best out of all agents I've tried at following the guidance to do only-what-is-needed-and-no-more. Opus 4.6 takes longer, overthinks things and changes too much; the high-powered GPTs are similarly f…

[dead]

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#433

Post by 8note »

bigyabai wrote: post #317
ACCount37 wrote: post #316

Earlier quoted context omitted.

If you think that "strawberry" is some kind of own, I don't know what to tell you. It takes deep and profound ignorance of both the technical basics of modern AIs and the current SOTA to do this kind of thing. LLMs get better release to release. Unfortunately, the quality of humans in LLM capability discussions is consistently abysmal. I wouldn't be seeing the same "LLMs are FUNDAMENTALLY FLAWED because I SAY SO" rep…

I can ask a nine-year-old human brain to solve that problem with a box of Crayola and a sheet of A4 printer paper. In-context learning is professedly not "good enough" to approximate continuous learning of even a child.

why is the breakdown from words to letters your highest priority thing to add to the training data?

what problem does this allow you to solve that you couldnt otherwise?

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#434

Post by rvz »

willis936 wrote: post #112
fny wrote: post #20

I'm going to suggest what's going on here is Hanlon's Razor for models: "Never attribute to malice that which is adequately explained by a model's stupidity." In my opinion, we've reached some ceiling where more tokens lead only to incremental improvements. A conspiracy seems unlikely given all providers are still competing for customers and a 50% token drives infra costs up dramatically too.

Never attribute to incompetence what is sufficiently explained by greed.

Correct.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#435

Post by RevEng »

senordevnyc wrote: post #190
pitched wrote: post #65

Earlier quoted context omitted.

Running an open like Kimi constantly for an entire month will cost around 100-200$, being roughly equal to a pro-tier subscription. This is not my estimate so I’m more than open to hearing refutations. Kimi isn’t at all Opus-level intelligent but the models are roughly evenly sized from the guesses I’ve seen. So I don’t think it’s the infra being subsidized as much as it’s the training.

I’m using Composer 2, Cursor’s model they built on top of Kimi, and it’s great. Not Opus level, but I’m finding many things don’t need Opus level.

It's all I use at work and I've yet to find anything it can't handle. Then again, I'm a principal engineer and I already have designs in mind, so I'm giving it careful instruction and checking its work every time.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#436

Post by benjiro3000 »

dgellow wrote: post #153
dakiol wrote: post #46

We dropped Claude. It's pretty clear this is a race to the bottom, and we don't want a hard dependency on another multi-billion dollar company just to write software We'll be keeping an eye on open models (of which we already make good use of). I think that's the way forward. Actually it would be great if everybody would put more focus on open models, perhaps we can come up with something like the "linux/postgres/git…

Another aspect I haven’t seen discussed too much is that if your competitor is 10x more productive with AI, and to stay relevant you also use AI and become 10x more productive. Does the business actually grow enough to justify the extra expense? Or are you pretty much in the same state as you were without AI, but you are both paying an AI tax to stay relevant?

[dead]

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#437

Post by djeastm »

3dfd wrote: post #427
Bridged7756 wrote: post #233

Earlier quoted context omitted.

Mirrors my sentiment. Those tools seem mostly useful for a Google alternative, scaffolding tedious things, code reviewing, and acting as a fancy search. It seems that they got a grip on the "coding LLM" market and now they're starting to seek actual profit. I predict we'll keep seeing 40%+ more expensive models for a marginal performance gain from now on.

[dead]

I think that's precisely why they're paying thousands of people in those other jobs to perform their tasks while collecting new data. Software was easiest because its already mostly written down, but other jobs can be quantized with enough data points. Just give it time

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#438

Post by RevEng »

fathermarz wrote: post #234

I have been seeing this messaging everywhere and I have not noticed this. I have had the inverse with 4.7 over 4.6. I think people aren’t reading the system cards when they come out. They explicitly explain your workflow needs to change. They added more levels of effort and I see no mention of that in this post. Did y’all forget Opus 4? That was not that long ago that Claude was essentially unusable then. We are peak…

I have used nothing but Sonnet and composer for a year and they work fine. LLMs were certainly not unusable before and Opus is certainly not necessary, especially considering the cost. People get excited by new records on benchmarks but for most day to day work the existing models are sufficient and far more efficient.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#439

Post by [deleted] »

danny_codes wrote: post #251
Bridged7756 wrote: post #233

Earlier quoted context omitted.

Mirrors my sentiment. Those tools seem mostly useful for a Google alternative, scaffolding tedious things, code reviewing, and acting as a fancy search. It seems that they got a grip on the "coding LLM" market and now they're starting to seek actual profit. I predict we'll keep seeing 40%+ more expensive models for a marginal performance gain from now on.

I just don’t see how they’ll be able to make a profit. Open models have the same performance on coding tasks now. The incentives are all wrong. Why pay more for a model that’s no better and also isn’t open? It’s nonsense

[deleted]

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#440

Post by stavros »

krackers wrote: post #407
_blk wrote: post #164

Earlier quoted context omitted.

From what I understand you shouldn't wait more than 5min between prompts without compacting or clearing or you'll pay for reinitializing the cache. With compaction you still pay but it's less input tokens. (Is compaction itself free?)

>pay for reinitializing the cache Why can't they save the kv cache to disk then later reload it to memory?

Probably because the costly operation is loading it onto the GPU, doesn't matter if it's from disk or from your request.
Post reply on HN