Opus 4.5 is not the normal AI agent experience that I have had thus far
burkeholland.github.io
Opus 4.5 is not the normal AI agent experience that I have had thus far
1–10 of 1001 posts
Re: Opus 4.5 is not the normal AI agent experience that I have had thus far
#2Can it pre-emptively write the HN comment where someone says it utterly fails for them but no one else is able to reproduce?
Re: Opus 4.5 is not the normal AI agent experience that I have had thus far
#3[flagged]
Re: Opus 4.5 is not the normal AI agent experience that I have had thus far
#4The problem with this is none of this is production quality. You haven’t done edge case testing for user mistakes, a security audit, or even just maintainability.
Yes opus 4.5 seems great but most of the time it tries to vastly over complicate a solution. Its answer will be 10x harder to maintain and debug than the simpler solution a human would have created by thinking about the constraints of keeping code working.
Re: Opus 4.5 is not the normal AI agent experience that I have had thus far
#5I really wonder what means for software moving forward. In the last few months I've used Claude Code to build personalized versions of Superwhisper (voice-to-text), CleanShot X (screenshot and image markup), and TextSniper (image to text). The only cost was some time and my $20/month subscription.
Re: Opus 4.5 is not the normal AI agent experience that I have had thus far
#6Mm this is my experience as well, but I'm not particularly worried about software engineering a whole.
If anything this example shows that these cli tools give regular devs much higher leverage.
There's a lot of software labor that is like, go to the lowest cost country, hire some mediocre people there and then hire some US guy to manage them.
That's the biggest target of this stuff, because now that US guy can just get equal or hight code in both quality and output without the coordination cost.
But unless we get to the point where you can do what I call "hypercode" I don't think we'll see SWEs as a whole category die.
Just like we don't understand assembly but still need technical skills when things go wrong, there's always value in low level technical skills.
Re: Opus 4.5 is not the normal AI agent experience that I have had thus far
#7The problem with this is none of this is production quality. You haven’t done edge case testing for user mistakes, a security audit, or even just maintainability. Yes opus 4.5 seems great but most of the time it tries to vastly over complicate a solution. Its answer will be 10x harder to maintain and debug than the simpler solution a human would have created by thinking about the constraints of keeping code working.
Agree... but that is exactly what MVPs are. Humans have been shipping MVPs while calling them production-ready for decades.
Re: Opus 4.5 is not the normal AI agent experience that I have had thus far
#8Opus 4.5 has become really capable.
Not in terms of knowledge. That was already phenomenal. But in its ability to act independently: to make decisions, collaborate with me to solve problems, ask follow-up questions, write plans and actually execute them.
You have to experience it yourself on your own real problems and over the course of days or weeks.
Every coding problem I was able to define clearly enough within the limits of the context window, the chatbot could solve and these weren’t easy. It wasn’t just about writing and testing code. It also involved reverse engineering and cracking encoding-related problems. The most impressive part was how actively it worked on problems in a tight feedback loop.
In the traditional sense, I haven’t really coded privately at all in recent weeks. Instead, I’ve been guiding and directing, having it write specifications, and then refining and improving them.
Curious how this will perform in complex, large production environments.
Re: Opus 4.5 is not the normal AI agent experience that I have had thus far
#9Once you get your setup bulletproof such that you can have multiple agents running at the same time that can run unit tests and close their own loops things get even faster. However you accomplish that. Not as easy as it sounds mostly (and absurdly) due to port collision. E2E testing with playwright is another leap.
Re: Opus 4.5 is not the normal AI agent experience that I have had thus far
#10The problem with this is none of this is production quality. You haven’t done edge case testing for user mistakes, a security audit, or even just maintainability. Yes opus 4.5 seems great but most of the time it tries to vastly over complicate a solution. Its answer will be 10x harder to maintain and debug than the simpler solution a human would have created by thinking about the constraints of keeping code working.
That may be true now, but think about how far we've come in a year alone! This is really impressive, and even if the models don't improve, someone will build skills to attack these specific scenarios.
Over time, I imagine even cloud providers, app stores etc can start doing automated security scanning for these types of failure modes, or give a more restricted version of the experience to ensure safety too.