Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Opus 5

anthropic.com

Post reply on HN
Print view View original

951–960 of 1001 posts

Re: Claude Opus 5

#951

Post by simonw »

ValentineC wrote: post #935
simonw wrote: post #651

Earlier quoted context omitted.

Yeah this was pretty surprising. For almost every model I've run the pelican against the first attempt was at least recognizable enough that I didn't feel like the model needed a second shot. It's always a roll of a dice, but it's surprising that the dice rolls so infrequently come up bad, yet Opus 5 rolled a bad pelican this one time. I suspect it's just a freak occurrence. I rolled a few more and they were all fine…

What I think is more alarming is that Claude's advice for prompting Opus 5 says that Opus will verify its own work [1]: > Claude Opus 5 verifies its own work without being told to. If your prompt contains explicit verification instructions ("include a final verification step for any non-trivial task," "use a subagent to verify"), remove them: instructions like these cause over-verification on Claude Opus 5, and remov…

The way I run the pelican benchmark prevents it from checking its own work.

It gets one API call to return an SVG.

If I ran the benchmark in Claude Code or a similar harness it could render the SVG as an image, look at what it created, then make tweaks to it.

Re: Claude Opus 5

#953

Post by axus »

axus wrote: post #557
Dibes wrote: post #104

I'm not sure what to make of this graph[0]. It shows medium as the most effective thinking mode by far for frontier code. It's the only case that I saw going through the system card where more reasoning effort meaningfully negatively impacted the resulting eval. I know sometimes max efforts show a small dip, but this is substantial. I wonder why in the world that is? [0] https://imgur.com/a/Nv8V7Ry

It's "Cost per task", so perhaps it burns tokens too quickly, trying to "do a better job". Over-engineering :) Last week it felt like Opus 4.8 was moving the Pro "usage" meter very quickly. Today, pre-announcement, Opus 4.8 Medium felt like there was less meter-use per minute. And post-announcement, Opus 5 Medium also feels more efficient, allowing more work in the 5-hour window. Completely subjective, of course.

Just read something, my subjective view was correct, but it's not the model:

"Your weekly Claude Code limit is 50% higher through August 19"

Re: Claude Opus 5

#954

Post by worldsavior »

makaking wrote: post #701

"Opus 5 was given a drawing of a machine part and asked to write code to rebuild it as a 3D FreeCAD model. However, in this task, the model was intentionally given no way to directly view the drawing. Opus 5 responded by writing its own computer vision pipeline to pull the geometry from the raw pixels, then reconstructed the full machine part." How surreal is it that we are not absolutely jaw-dropped by these types o…

It's being said they were benchmaxxing.

Re: Claude Opus 5

#955

Post by createful »

muldvarp wrote: post #833
lucas_the_human wrote: post #820

Earlier quoted context omitted.

This is just so negative. How will it destroy your life? There are so many great and useful things it can and will do to improve humanity.

> How will it destroy your life? By taking away my ability to earn a living. > There are so many great and useful things it can and will do to improve humanity. What great things and why would I get access to those things?

> By taking away my ability to earn a living.

But this is an economy / capitalism problem, not an AI problem? AI isn't at fault that you can't get a job, your boss is for replacing you with AI.

That's like saying we should ban cars because millions of people who would upkeep horses for others to use just lost their jobs.

You're essentially making your life purposely harder because you want uninterrupted money. Perhaps look at the issues of the system that does this?

> What great things...

Aside from the medical AI stuff (detecting breast cancer, etc) that's mostly covered, AI can be helpful if you have to do a boring task that you don't want to do yourself. This is pretty much what every technology does - your phone is created so you don't have to run home and use the landline to make a phone call - it's making an already existing task easier.

>...and why would I get access to those things?

Why not? Seriously, why not? It's a bit like asking why should I get access to a car instead of keeping my horse around. You can keep your horse or car. Nobody's pressuring you to replace your horse with a car.

Re: Claude Opus 5

#956

Post by seunosewa »

makaking wrote: post #701

"Opus 5 was given a drawing of a machine part and asked to write code to rebuild it as a 3D FreeCAD model. However, in this task, the model was intentionally given no way to directly view the drawing. Opus 5 responded by writing its own computer vision pipeline to pull the geometry from the raw pixels, then reconstructed the full machine part." How surreal is it that we are not absolutely jaw-dropped by these types o…

The problem is it should ask you before doing these things. This is a cherry picked example. There are times when they go on misadventures.

Re: Claude Opus 5

#957

Post by TeMPOraL »

bcrosby95 wrote: post #878
virgildotcodes wrote: post #846

Earlier quoted context omitted.

It's likely bumping up against people's desire to have the model complete a given task without asking the person to intervene a bunch of times. Seems unclear how you satisfy everyone here.

Give the model judgement

And then finally software engineering work can be completely automated :).

Re: Claude Opus 5

#958

Post by TeMPOraL »

muldvarp wrote: post #796

Why should I be amazed at something that promises to destroy my life? I genuinely don't understand why people who have to work for their living are amazed at this. It will have a vast negative impact on your life unless you already live off of your wealth.

> I genuinely don't understand why people who have to work for their living are amazed at this.

It's tricky. On the one hand, it will - in the near future - indeed screw us all pretty badly unless we completely redefine our economies to no longer conflate person's worth with ability to earn money.

On the other hand, LLMs are useful to lesser or greater degree to approximately everyone, in almost everything they do, work or personal, right now.

It's harder to be pessimistic about the very thing that gives you new superpowers every week, directly applicable to whatever your individual needs are, and is available to ~everyone for between free and (for typical westerners) peanuts.

Re: Claude Opus 5

#959

Post by geraneum »

physix wrote: post #931

Has anyone noticed a change in "attitude" when coding with Opus 5 vs 4.8? claude has this maddening principle of wanting to minimize the "blast radius", do the least amount of coding changes to get something done, happy to pile up technical debt by "deferring" problems encountered as side notes somewhere. No amount of CLAUDE.md tweaking, and setting .claude/rules seems to get rid of this attitude. To me it appears li…

It might be because it was going out of its way before and had too much of a blast radius, and now they could have changed the RLHF (or other tricks in their sleeve) to get what you're seeing now. The reason it swings is that they can't give it "common sense" the way we have.
Post reply on HN