Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Opus 4.5 is not the normal AI agent experience that I have had thus far

burkeholland.github.io

Post reply on HN
Print view View original

611–620 of 1001 posts

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#611

Post by puttycat »

subomi wrote: post #497
lelandfe wrote: post #458

Earlier quoted context omitted.

I'm a quite senior frontend using React and even I see Sonnet 4.5 struggle with basic things. Today it wrote my Zod validation incorrectly, mixing up versions, then just decided it wasn't working and attempted to replace the entire thing with a different library.

Why do we all of a sudden hold these agents to some unrealistic high bar? Engineers write bugs all the time and write incorrect validations. But we iterate. We read the stacktrace in Sentry and realise what the hell I was thinking when I wrote that, and we fix things. If you're going to benefit from these agents, you'd need to be a bit more patient and point them correctly to your codebase. My rule of thumb is that i…

> Engineers write bugs all the time

Why do we hold calculators to such high bars? Humans make calculation mistakes all the time.

Why do we hold banking software to such high bars? People forget where they put their change all the time.

Etc etc.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#612

Post by Leynos »

llmslave2 wrote: post #522
troupo wrote: post #208

Earlier quoted context omitted.

> Opus 4.5 really is at a new tier however. It just...works. Literally tried it yesterday. I didn't see a single difference with whatever model Claude Code was using two months ago. Same crippled context window. Same "I'll read 10 irrelevant lines from a file", same random changes etc.

That's because Opus has been out for almost 5 months now lol. Its the same model, so I think people have been vibe coding with a heavy dose of wine this holiday and are now convinced its the future.

Opus 4.5 was released 24th November.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#613

Post by cheschire »

maciejzj wrote: post #604

I've been on a small adventure of posting more actively on HN since the release of Gemini 3, trying to stir debate around the more “societal” aspects of what's going on with AI. Regardless of how much you value Cloud Code technically, there is no denying that it has/will have huge impact. If technology knowledge and development are commoditised and distributed via subscription, huge societal changes are going to happ…

I’ve been thinking, what if all this robotics work doesn’t result in AI automating the real world, but instead results in third world slavery without the first world wages or immigration concerns anymore?

Connect the world with reliable internet, then build a high tech remote control facility in Bangladesh and outsource plumbing, electrical work, housekeeping, dog watching, truck driving, etc etc

No AGI necessary. There’s billions of perfectly capable brains halfway around the world.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#614

Post by emsign »

The worst part about this is that you can't know anymore whether the software you trustingly install on your hardware is clean or if it was coded by a misaligned coding model with a secret goal that it has hidden from its prompt engineer and from you.

This could pretty much be the beginning of the end of everything, if misaligned models wanted to they could install killswitches everywhere. And you can't trust security updates either so you are even more vulnerable to external exploits.

It's really scary, I fear the future, it's going to be so bad. It's best to not touch AI at all and stay hidden from it as long as possible to survive the catastrophe or not be a helping part of it. Don't turn your devices into a node of a clandestine bot net that is only waiting to conspire against us.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#615

Post by dugidugout »

jasonfarnon wrote: post #529
melagonster wrote: post #501

Earlier quoted context omitted.

- I cloned a project from GitHub and made some minor modifications. - I used AI-assisted programming to create a project. Even if the content is identical, or if the AI is smart enough to replicate the project by itself, the latter can be included on a CV.

I think I would prefer the former if I were reviewing a CV. It at least tells me they understood the code well enough to know where to make their minor tweaks. (I've spent hours reading through a repo to know where to insert/comment out a line to suit my needs.) The second tells me nothing.

Its odd you don't apply the same analysis to each. The latter certainly can provide a similar trail indicating knowledge of the use case and necessary parameters to achieve it. And certainly the former doesnt preclude llm interlocking.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#616

Post by koyote »

Sammi wrote: post #513
koyote wrote: post #391

Earlier quoted context omitted.

A greenfield project is definitely 'easy mode' for an LLM; especially if the problem area is well understood (and documented). Opus is great and definitely speeds up development even in larger code bases and is reasonably good at matching coding style/standard to that of of the existing code base. In my opinion, the big issue is the relatively small context that quickly overwhelms the models when given a larger task…

I just did something similar and it went swimmingly by doing this: Keep the plan and status in an md file. Tell it to finish one file at a time and run tests and fix issues and then to ask whether to proceed with the next file. You can then easily start a new chat with the same instructions and plan and status if the context gets poisoned.

I might give that a go in the future, but in this case it would've been faster for me to just do the work than to coach it for each file.

Also as this was an architectural change there are no tests to run until it's done. Everything would just fail. It's only done when the whole thing is done. I think that might be one of the reasons it got stuck: it was trying to solve issues that it did not prove existed yet. If it had just finished the job and run the tests it would've probably gotten further or even completed it.

It's a bit like stopping half way through renaming a function and then trying to run the tests and finding out the build does not compile because it can't find 'old_function'. You have to actually finish and know you've finished before you can verify your changes worked.

I still haven't actually addressed this tech debt item (it's not that important :)). But I might try again and either see if it succeeds this time (with plan in an md) or just do the work myself and get Opus to fix the unit tests (the most tedious part).

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#617

Post by weatherlite »

maciejzj wrote: post #604

I've been on a small adventure of posting more actively on HN since the release of Gemini 3, trying to stir debate around the more “societal” aspects of what's going on with AI. Regardless of how much you value Cloud Code technically, there is no denying that it has/will have huge impact. If technology knowledge and development are commoditised and distributed via subscription, huge societal changes are going to happ…

UBI (from taxing big tech) and retraining. In the U.S they'll have enough money to do this and it will still suck and many people won't recover the extreme loss of status and income (after we've been told our income and status are the most important things in life it's gonna be very hard for people to adapt to the loss of it). Countries like India and Philipines and Ukraine which are basically knowledge support hub without much original knowledge of their own yeah this is gonna be something for sure. Quite depressing.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#618

Post by neocron »

emsign wrote: post #614

The worst part about this is that you can't know anymore whether the software you trustingly install on your hardware is clean or if it was coded by a misaligned coding model with a secret goal that it has hidden from its prompt engineer and from you. This could pretty much be the beginning of the end of everything, if misaligned models wanted to they could install killswitches everywhere. And you can't trust securit…

I have to many machines standing around that are currently not powered on or are running somewhat airgapped with old software from around debian 8 and 9, so I guess they will be a safe haven once the AI overlords take over

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#619

Post by ekidd »

SeanAppleby wrote: post #383
multisport wrote: post #320

What bothers me about posts like this is: mid-level engineers are not tasked with atomic, greenfield projects. If all an engineer did all day was build apps from scratch, with no expectation that others may come along and extend, build on top of, or depend on, then sure, Opus 4.5 could replace them. The hard thing about engineering is not "building a thing that works", its building it the right way, in an easily unde…

One thing I've been tossing around in my head is: - How quickly is cost of refactor to a new pattern with functional parity going down? - How does that change the calculus around tech debt? If engineering uses 3 different abstractions in inconsistent ways that leak implementation details across components and duplicate functionality in ways that are very hard to reason about, that is, in conventional terms, an existe…

> But if claude can reliably reorganize code, fix patterns, and write working migrations for state when prompted to do so, it seems like the entire way to reason about tech debt has changed.

Yup, I recently spent 4 days using Claude to clean up a tool that's been in production for over 7 years. (There's only about 3 months of engineering time spent on it in those years.)

We've known what the tool needed for many years, but ugh, the actual work was fairly messy and it was never a priority. I reviewed all of Opus's cleanup work carefully and I'm quite content with the result. Maybe even "enthusiastic" would be accurate.

So even if Claude can't clean up all the tech debt in a totally unsupervised fashion, it can still help address some kinds of tech debt extremely rapidly.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#620

Post by dbspin »

cheschire wrote: post #613
maciejzj wrote: post #604

I've been on a small adventure of posting more actively on HN since the release of Gemini 3, trying to stir debate around the more “societal” aspects of what's going on with AI. Regardless of how much you value Cloud Code technically, there is no denying that it has/will have huge impact. If technology knowledge and development are commoditised and distributed via subscription, huge societal changes are going to happ…

I’ve been thinking, what if all this robotics work doesn’t result in AI automating the real world, but instead results in third world slavery without the first world wages or immigration concerns anymore? Connect the world with reliable internet, then build a high tech remote control facility in Bangladesh and outsource plumbing, electrical work, housekeeping, dog watching, truck driving, etc etc No AGI necessary. Th…

This is exactly what Meredith Whittaker is saying... The 'edge conditions' outside the training data will never go away, and 'AGI' will for the foreseeable future simply mean millions in servitude teleoperating the robots, RLHFing the models or filling in the AI gaps in various ways.
Post reply on HN