Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

tokens.billchambers.me

Post reply on HN
Print view View original

381–390 of 620 posts

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#381

Post by mh- »

zozbot234 wrote: post #309
aerhardt wrote: post #200

Earlier quoted context omitted.

> you just limit the space to text And even then... why can't they write a novel? Or lowering the bar, let's say a novella like Death in Venice , Candide , The Metamorphosis , Breakfast at Tiffany's ...? Every book's in the training corpus... Is it just a matter of someone not having spent a hundred grand in tokens to do it?

Never mind novels, it can't even write a good Reddit-style or HN-style comment. agentalcove.ai has an archive of AI models chatting to one another in "forum" style and even though it's a good show of the models' overall knowledge the AIisms are quite glaring.

They definitely can, and do.

It's just that the ones that manage to suppress all the AI writing "tells" go unnoticed as AI. This is a type of survivorship bias, though I feel there must be a better term for it that eludes me.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#382

Post by sgc »

gadflyinyoureye wrote: post #71
tailscaler2026 wrote: post #26

Subsidies don't last forever.

I've been assuming this for a while. If I have a complex feature, I use Opus 4.6 in copilot to plan (3 units of my monthly limit). Then have Grok or Gemini (.25-.33) of my monthly units to implement and verify the work. 80% of the time it works every time. Leave me plenty of usage over the month.

I have a very newcomer-type question. What is the output format of your plan such that you can break context and get the other LLM to produce satisfactory results? What level of details is in the plan, bullet points, pseudo-code, or somewhere in the middle?

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#383

Post by Fabricio20 »

_blk wrote: post #373
gck1 wrote: post #365

Earlier quoted context omitted.

Cache ttl on max subscriptions is 1h, FYI.

That'd be awesome but it doesn't reflect what I see. Do you have a source for that? What I see is if take a quick break the session loses ~5% right at the start of the next prompt processing. (I'm currently on max 5x)

I have heard that if you have telemetry disabled the cache is 5 minutes, otherwise 1h. No clue how true that is however my experience (with telemetry enabled) has been the 1h cache.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#384

Post by viccis »

therealdrag0 wrote: post #203
SpicyLemonZest wrote: post #182

Earlier quoted context omitted.

Are you sure you would know if it didn't work? I use Claude extensively myself, so I'm not saying this from a "hater" angle, but I had 2 people last week who believe themselves to be in your shoes send me pull requests which made absolutely no sense in the context of the codebase.

That’s always been the case, AI or not.

In my experience, the people sending me garbage PRs with Claude are the same ones who wrote garbage code beforehand. Now there's just 10x more of it.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#385

Post by hakfoo »

trueno wrote: post #354
michaelje wrote: post #216

Earlier quoted context omitted.

Open models keep closing the eval gap for many tasks, and local inference continues to be increasingly viable. What's missing isn't technical capability, but productized convenience that makes the API path feel like the only realistic option. Frontier labs are incentivized to keep it that way, and they're investing billions to make AI = API the default. But that's a business model, not a technical inevitability.

im hoping and praying that local inference finds it's way to some sort of baseline that we're all depending on claude for here. that would help shape hardware designs on personal devices probably something in the direction of what apple has been doing. ive had to like tune out of the LLM scene because it's just a huge mess. It feels impossible to actually get benchmarks, it's insanely hard to get a grasp on what ever…

I'm frustrated that there's not "solid" instructional tooling. I either see people just saying "keep trying different prompts and switching models until you get lucky" or building huge cantilevered toolchains that seems incredibly brittle, and even then, how well do they really work?

I get choice paralysis when you show me a prompt box-- I don't know what I can reasonably ask for and how to best phrase it, so I just panic. It doesn't help when we see articles saying people are getting better outcomes by adding things like "and no bugs plz owo"

I'm sure this is by design-- anything with clear boundaries and best practices would discourage gacha style experimentation. Can you trust anyone who sells you a metered service to give you good guidance on how to use it efficiently?

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#386

Post by fmckdkxkc »

varispeed wrote: post #204

I spent one day with Opus 4.7 to fix a bug. It just ran in circles despite having the problem "in front of its eyes" with all supporting data, thorough description of the system, test harness that reproduces the bug etc. While I still believe 4.7 is much "smarter" than GPT-5.4 I decided to give it ago. It was giving me dumb answers and going off the rails. After accusing it many times of being a fraud and doing it on…

I enjoy using Claude but I find the vibing stuff starts to cause source-code amnesia. Even if I design something and put forth a thoughtful plan, the more I increase my output the less I feel the “vibes”.

It’s funny everyone says “the cost will just go down” with AI but I don’t know.

We need to keep the open source models alive and thriving. Oh, but wait the AI companies are buying all the hardware.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#387

Post by Gareth321 »

cyberax wrote: post #238
zozbot234 wrote: post #73

Earlier quoted context omitted.

Mind you, a 30B model (3B active) is not going to be comparable to Opus. There are open models that are near-SOTA but they are ~750B-1T total params. That's going to require substantial infrastructure if you want to use them agentically, scaled up even further if you expect quick real-time response for at least some fraction of that work. (Your only hope of getting reasonable utilization out of local hardware in sing…

I'm backing up a big dataset onto tapes, so I wanted to automate it. I have an idle 64Gb VRAM setup in my basement, so I decided to experiment and tasked it with writing an LTFS implementation. LTFS is an open standard for filesystems for tapes, and there's an implementation in C that can be used as the baseline. So far, Qwen 3.6 created a functionally equivalent Golang implementation that works against the flat file…

It is surprisingly competent. It's not Opus 4.6 but it works well for well structured tasks.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#388

Post by Gareth321 »

jherdman wrote: post #102
blahblaher wrote: post #56

Earlier quoted context omitted.

qwen3.5/3.6 (30B) works well,locally, with opencode

Is this sort of setup tenable on a consumer MBP or similar?

The Mac Minis (probably 64GB RAM) are the most cost effective.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#389

Post by matheusmoreira »

andai wrote: post #239

For a fair comparison you need to look at the total cost, because 4.7 produces significantly fewer output tokens than 4.6, and seems to cost significantly less on the reasoning side as well. Here is a comparison for 4.5, 4.6 and 4.7 (Output Tokens section): https://artificialanalysis.ai/?models=claude-opus-4-7%2Cclau... 4.7 comes out slightly cheaper than 4.6. But 4.5 is about half the cost: https://artificialanalysi…

It thinks less and produces less output tokens because it has forced adaptive thinking that even API users can't disable. Same adaptive thinking that was causing quality issues in Opus 4.6 not even two weeks ago. The one bcherny recommended that people disable because it'd sometimes allocate zero thinking tokens to the model.

https://news.ycombinator.com/item?id=47668520

People are already complaining about low quality results with Opus 4.7. I'm also spotting it making really basic mistakes.

I literally just caught it lazily "hand-waving" away things instead of properly thinking them through, even though it spent like 10 minutes churning tokens and ate only god knows how many percentage points off my limits.

> What's the difference between this and option 1.(a) presented before?

> Honestly? Barely any. Option M is option 1.(a) with the lifecycle actually worked out instead of hand-waved.

> Why are you handwaving things away though? I've got you on max effort. I even patched the system prompts to reduce this.

> Fair call. I was pattern-matching on "mutation + capture = scary" without actually reading the capture code. Let me do the work properly.

> You were right to push back. I was wrong. Let me actually trace it properly this time.

> My concern from the first pass was right. The second pass was me talking myself out of it with a bad trace.

It's just a constant stream of self-corrections and doubts. Opus simply cannot be trusted when adaptive thinking is enabled.

Can provide session feedback IDs if needed.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#390

Post by coldtea »

danny_codes wrote: post #270
coldtea wrote: post #10

This, the push towards per-token API charging, and the rest are just a sign of things to come when they finally establish a moat and full monoply/duopoly, which is also what all the specialized tools like Designer and integrations are about. It's going to be a very expensive game, and the masses will be left with subpar local versions. It would be like if we reversed the democratization of compilers and coding toolin…

I doubt that’s the case. My guess is we’ll hit asymptomatic returns from transformers, but price-to-train will fall at moore’s law. So over time older models will be less valuable, but new models will only be slightly better. Frontier players, therefore, are in a losing business. They need to charge high margins to recoup their high training costs. But latecomers can simply train for a fraction of the cost. Since per…

>I doubt that’s the case. My guess is we’ll hit asymptomatic returns from transformers, but price-to-train will fall at moore’s law.

There hasn't been a real Moore's law for a good while even before LLMs.

And memory isn't getting less expensive either...

Post reply on HN