Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

tokens.billchambers.me

Post reply on HN
Print view View original

451–460 of 620 posts

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#451

Post by post-it »

Ifkaluva wrote: post #350
post-it wrote: post #122

Earlier quoted context omitted.

I was worried about skill atrophy. I recently started a new job, and from day 1 I've been using Claude. 90+% of the code I've written has been with Claude. One of the earlier tickets I was given was to update the documentation for one of our pipelines. I used Claude entirely, starting with having it generate a very long and thorough document, then opening up new contexts and getting it to fact check until it stopped…

What do you mean “cut out anything that was granular/one query away”? This was a very cool workflow to hear about—I will be applying it myself

For example, Claude was very eager to include function names, implementation details, and the exact variables that are passed between services. But all the info I need for a particular process is the names of the services involved, the files involved, and a one-sentence summary of what happens. If I want to know more, I can tell Claude to read the doc and find out more with a single query (or I can just check for myself).

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#452

Post by post-it »

SpicyLemonZest wrote: post #182
post-it wrote: post #122

Earlier quoted context omitted.

I was worried about skill atrophy. I recently started a new job, and from day 1 I've been using Claude. 90+% of the code I've written has been with Claude. One of the earlier tickets I was given was to update the documentation for one of our pipelines. I used Claude entirely, starting with having it generate a very long and thorough document, then opening up new contexts and getting it to fact check until it stopped…

Are you sure you would know if it didn't work? I use Claude extensively myself, so I'm not saying this from a "hater" angle, but I had 2 people last week who believe themselves to be in your shoes send me pull requests which made absolutely no sense in the context of the codebase.

Yeah, I test everything myself.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#453

Post by j-bos »

codethief wrote: post #430
matheusmoreira wrote: post #389

Earlier quoted context omitted.

It thinks less and produces less output tokens because it has forced adaptive thinking that even API users can't disable. Same adaptive thinking that was causing quality issues in Opus 4.6 not even two weeks ago. The one bcherny recommended that people disable because it'd sometimes allocate zero thinking tokens to the model. https://news.ycombinator.com/item?id=47668520 People are already complaining about low quali…

> > Why are you handwaving things away though? I've got you on max effort. I even patched the system prompts to reduce this. In my experience, prompts like this one, which 1) ask for a reason behind an answer (when the model won't actually be able to provide one), 2) are somewhat standoff-ish, don't work well at all. You'll just have the model go the other way. What works much better is to tell the model to take a st…

Yeah for anyone seriously using these models I highly reccomend reading the Mythos system card, esp the sections on analyzing it's internal non verbalized states. Save a lot of head wall banging.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#454

Post by post-it »

viccis wrote: post #392
post-it wrote: post #122

Earlier quoted context omitted.

I was worried about skill atrophy. I recently started a new job, and from day 1 I've been using Claude. 90+% of the code I've written has been with Claude. One of the earlier tickets I was given was to update the documentation for one of our pipelines. I used Claude entirely, starting with having it generate a very long and thorough document, then opening up new contexts and getting it to fact check until it stopped…

It's good that it's working for you but I'm not sure what this has to do with skill atrophy. It sounds like you never had this skill (in this case, working with that particular system) to begin with. >I have a significantly better understanding of the codebase than I would without AI at this point in my onboarding One of the pitfalls of using AI to learn is the same as I'd see students doing pre-AI with tutoring serv…

> If you were on a plane flight with no wifi, and you wanted to do some dev work locally on your laptop, how comfortable would you be vs if you had done all that work yourself rather than via Claude?

Probably about as comfortable as I would be if I also didn't have my laptop and instead had to sketch out the codebase in a notebook. There's no sense preparing for a scenario where AI isn't available - local models are progressing so quickly that some kind of AI is always going to be available.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#455

Post by mrandish »

codethief wrote: post #430
matheusmoreira wrote: post #389

Earlier quoted context omitted.

It thinks less and produces less output tokens because it has forced adaptive thinking that even API users can't disable. Same adaptive thinking that was causing quality issues in Opus 4.6 not even two weeks ago. The one bcherny recommended that people disable because it'd sometimes allocate zero thinking tokens to the model. https://news.ycombinator.com/item?id=47668520 People are already complaining about low quali…

> > Why are you handwaving things away though? I've got you on max effort. I even patched the system prompts to reduce this. In my experience, prompts like this one, which 1) ask for a reason behind an answer (when the model won't actually be able to provide one), 2) are somewhat standoff-ish, don't work well at all. You'll just have the model go the other way. What works much better is to tell the model to take a st…

> when the model won't actually be able to provide one

This is key. In my experience, asking an LLM why it did something is usually pointless. In a subsequent round, it generally can't meaningfully introspect on its prior internal state, so it's just referring to the session transcript and extrapolating a plausible sounding answer based on its training data of how LLMs typically work.

That doesn't necessarily mean the reply is wrong because, as usual, a statistically plausible sounding answer sometimes also happens to be correct, but it has no fundamental truth value. I've gotten equally plausible answers just pasting the same session transcript into another LLM and asking why it did that.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#456

Post by what »

matheusmoreira wrote: post #389
andai wrote: post #239

For a fair comparison you need to look at the total cost, because 4.7 produces significantly fewer output tokens than 4.6, and seems to cost significantly less on the reasoning side as well. Here is a comparison for 4.5, 4.6 and 4.7 (Output Tokens section): https://artificialanalysis.ai/?models=claude-opus-4-7%2Cclau... 4.7 comes out slightly cheaper than 4.6. But 4.5 is about half the cost: https://artificialanalysi…

It thinks less and produces less output tokens because it has forced adaptive thinking that even API users can't disable. Same adaptive thinking that was causing quality issues in Opus 4.6 not even two weeks ago. The one bcherny recommended that people disable because it'd sometimes allocate zero thinking tokens to the model. https://news.ycombinator.com/item?id=47668520 People are already complaining about low quali…

> Why are you handwaving things away though? I've got you on max effort. I even patched the system prompts to reduce this.

Do you think it knows what max effort or patched system prompts are? It feels really weird to talk to an LLM like it’s a person that understands.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#457

Post by post-it »

onemoresoop wrote: post #244
idopmstuff wrote: post #228

Earlier quoted context omitted.

Some people talk like skill atrophy is inevitable when you use LLMs, which strikes me as pretty absurd given that you are talking about a tool that will answer an infinite number of questions with infinite patience. I usually learn way more by having Claude do a task and then quizzing it about what it did than by figuring out how to do it myself. When I have to figure out how to do the thing, it takes much more time,…

You lose some, you win some. The win could be short-term much higher, however imagine that the new tool suddenly gets ragged pulled from under your feet. What do you do then? Do you still know how to handle it the old way or do you run into skill atrophy issues? I’m using Claude/Codex as well, but I’m a little worried that the environment we work in will become a lot more bumpy and shifty.

> however imagine that the new tool suddenly gets ragged pulled from under your feet

When you have a headache, do you avoid taking ibuprofen because one day it may not be available anymore? Two hundred years ago, if you gave someone ibuprofen and told them it was the solution for 99% of the cases where they felt some kind of pain, they might be suspicious. Surely that's too good to be true.

But it's not. Ibuprofen really is a free lunch, and so is AI. It's weird to experience, but these kinds of technologies come around pretty often, they just become ubiquitous so quickly that we forget how we got by without them.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#458

Post by cageface »

tiffanyh wrote: post #121
sync wrote: post #81

Earlier quoted context omitted.

Which plan are you on? I could see that happening with Pro (which I think defaults to Sonnet?), would be surprised with Max…

Pro. It even gave me $20 free credits, and exhausted free credits nearly instantly.

The pro plan is useless. You need at least the 5x max plan to get any real work done.

That said I find the GPT plans much better value.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#459

Post by vicchenai »

ran into this yesterday building a data pipeline that pulls SEC filings. same prompt, same context window, 4.7 chewed through noticeably more of my api budget than 4.6 did. the output wasnt obviously better either, just... more expensive.

what bugs me is the tokenizer change feels like a stealth price hike. if you're charging the same $/token but the same text now costs 35% more tokens, thats just a 35% price increase with extra steps. at least be upfront about it.

Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7

#460

Post by semcheck »

fooster wrote: post #397
hirako2000 wrote: post #70

Earlier quoted context omitted.

I haven't used Claude. Because I suspect this sort of things to come. With enterprise subscription, the bill gets bigger but it's not like VP can easily send a memo to all its staff that a migration is coming. Individuals may end their subscription, that would appease the DC usage, and turn profits up.

Sorry you are missing out. I use claude all day every day with max and what people are reporting here has not been my experience. My current usage is 16% and it resets Thursday.

[dead]
Post reply on HN