Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Opus 4.6

anthropic.com

Post reply on HN
Print view View original

851–860 of 1001 posts

Re: Claude Opus 4.6

#851

Post by insomagent »

I'm not super impressed with the performance, actually. I'm finding that it misunderstands me quite a bit. While it is definitely better at reading big codebases and finding a needle in a haystack, it's nowhere near as good as Opus 4.5 at reading between the lines and figuring out what I really want it to do, even with a pretty well defined issue.

It also has a habit of "running wild". If I say "first, verify you understand everything and then we will implement it."

Well, it DOES output its understanding of the issue. And it's pretty spot-on on the analysis of the issue. But, importantly, it did not correctly intuit my actual request: "First, explain your understanding of this issue to me so I can validate your logic. Then STOP, so I can read it and give you the go ahead to implement."

I think the main issue we are going to see with Opus 4.6 is this "running wild" phenomenon, which is step 1 of the eternal paperclip optimizer machine. So be careful, especially when using "auto accept edits"

Re: Claude Opus 4.6

#852

Post by vercaemert »

adastra22 wrote: post #849
vercaemert wrote: post #847

Earlier quoted context omitted.

It's impressive, even if the books and the posts you're talking about were both key parts of the training data. There are many academic domains where the research portion of a PhD is essentially what the model just did. For example, PhD students in some of the humanities will spend years combing ancient sources for specific combinations of prepositions and objects, only to write a paper showing that the previous scho…

I don’t think this example proves your point. There’s no indication that the model actually worked this out from the input context, instead of regurgitating it from the training weights. A better test would be to subtly modify the books fed in as input to the model so that there was actually 51 spells, and see if it pulls out the extra spell, or to modify the names of some spells, etc. In your example, it might be th…

Ah, that's a good point.

Re: Claude Opus 4.6

#853

Post by copilot_king_2 »

insomagent wrote: post #851

I'm not super impressed with the performance, actually. I'm finding that it misunderstands me quite a bit. While it is definitely better at reading big codebases and finding a needle in a haystack, it's nowhere near as good as Opus 4.5 at reading between the lines and figuring out what I really want it to do, even with a pretty well defined issue. It also has a habit of "running wild". If I say "first, verify you und…

[dead]

Re: Claude Opus 4.6

#854

Post by soulofmischief »

insomagent wrote: post #851

I'm not super impressed with the performance, actually. I'm finding that it misunderstands me quite a bit. While it is definitely better at reading big codebases and finding a needle in a haystack, it's nowhere near as good as Opus 4.5 at reading between the lines and figuring out what I really want it to do, even with a pretty well defined issue. It also has a habit of "running wild". If I say "first, verify you und…

I am having trouble with 4.6 following the most basic of instructions.

As an example, I asked it to commit everything in the worktree. I stressed everything and prompted it very explicitly, because even 4.5 sometimes likes to say, "I didn't do that other stuff, I'm only going to commit my stuff even though he said everything".

It still only committed a few things.

I had to ask again.

And again.

I had to ask four times, with increasing amounts of expletives and threats in order to finally see a clean worktree. I was worried at some point it was just going to solve the problem by cleaning the workspace without even committing.

4.5 is way easier to steer, despite its warts.

Re: Claude Opus 4.6

#855

Post by vinhnx »

vinhnx wrote: post #751

Just used Opus 4.6 via GitHub Copilot. It feels very different. Inference seems slow for now. I guess Opus 4.6 has adaptive thinking activated by default.

Confirm by PM lead at VS Code team

> "We have high thinking as default + adaptive thinking, first time we’ve run with these settings..."

> https://x.com/pierceboggan/status/2019645801769689486

Re: Claude Opus 4.6

#856

Post by vinhnx »

christophilus wrote: post #843
vinhnx wrote: post #751

Just used Opus 4.6 via GitHub Copilot. It feels very different. Inference seems slow for now. I guess Opus 4.6 has adaptive thinking activated by default.

It dos seem noticeably slower. I may stick with 4.5 which was good enough for me for most tasks.

VS Code confirms that they are experimenting with the new adaptive thinking and high reasoning effort params. https://x.com/pierceboggan/status/2019645801769689486

Re: Claude Opus 4.6

#857

Post by scwoodal »

soulofmischief wrote: post #854
insomagent wrote: post #851

I'm not super impressed with the performance, actually. I'm finding that it misunderstands me quite a bit. While it is definitely better at reading big codebases and finding a needle in a haystack, it's nowhere near as good as Opus 4.5 at reading between the lines and figuring out what I really want it to do, even with a pretty well defined issue. It also has a habit of "running wild". If I say "first, verify you und…

I am having trouble with 4.6 following the most basic of instructions. As an example, I asked it to commit everything in the worktree. I stressed everything and prompted it very explicitly, because even 4.5 sometimes likes to say, "I didn't do that other stuff, I'm only going to commit my stuff even though he said everything". It still only committed a few things. I had to ask again. And again. I had to ask four time…

Tell it what git commands to explicitly run and in what order for your desired outcome instead of “commit everything in the worktree”

This prompt will work better across any/all models.

Re: Claude Opus 4.6

#858

Post by lxgr »

MarcellusDrum wrote: post #825
grey-area wrote: post #818

Earlier quoted context omitted.

Surely the corpus Opus 4.6 ingested would include whatever reference you used to check the spells were there. I mean, there are probably dozens of pages on the internet like this: https://www.wizardemporium.com/blog/complete-list-of-harry-p... Why is this impressive? Do you think it's actually ingesting the books and only using those as a reference? Is that how LLMs work at all? It seems more likely it's predicting t…

So a good test would be replacing the spell names in the books with made-up spells. And if a "real" spell name was given, it also tests whether it "cheated".

It could still remember where each spell is mentioned. I think the only way to properly test this would be to run it against an unpublished manuscript.

Re: Claude Opus 4.6

#859

Post by app17 »

dahrkael wrote: post #838

I just tried it. designed a very detailed and reaaonable plan, made some amedments to it and wrote it down to a markdown file. i told it to implement it and it started implementing the original plan instead of the revised one, that was weird.

Did you use plan mode? Could it be that it used its original plan file (stored somewhere in ~/.claude) instead of your modified markdown? That’s unfortunately why I don’t use plan mode anymore. I wish I could just turn their plan files feature off.
Post reply on HN