Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Opus 4.5 is not the normal AI agent experience that I have had thus far

burkeholland.github.io

Post reply on HN
Print view View original

551–560 of 1001 posts

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#551

Post by Davidzheng »

mcv wrote: post #133

Opus 4.5 ate through my Copilot quota last month, and it's already halfway through it for this month. I've used it a lot, for really complex code. And my conclusion is: it's still not as smart as a good human programmer. It frequently got stuck, went down wrong paths, ignored what I told it to do to do something wrong, or even repeat a previous mistake I had to correct. Yet in other ways, it's unbelievably good. I ca…

If it can consistently verify that the error persists after fix--you can run (ok maybe you can't budget wise but theoretically) 10000 parallel instances of fixer agents then verify afterwards (this is in line with how the imo/ioi models work according to rumors)

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#552

Post by seanmcdirmid »

tripledry wrote: post #549

Putting the performance aside for now as I just started trying out Opus 4.5, can't say too much yet, I don't hype or hate AI as of now, it's simply useful. Time will tell what happens, but if programming becomes "prompt engineering", I'm planning on quitting my job and pivoting to something else. It's nice to get stuff working fast, but AI just sucks the joy out of building for me. Trying to not feel the pressure/anx…

Pity, prompt engineering is just another kind of programming, I find it to be fun, but I guess lots of other people would see it differently.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#553

Post by suzzer99 »

sksishbs wrote: post #359
fenwick67 wrote: post #336

Earlier quoted context omitted.

Another thing that gets me with projects like this, there are already many examples of image converters, minesweeper clones etc that you can just fork on GitHub, the value of the LLM here is largely just stripping the copyright off

It’s kind of funny - there’s another thread up where a dev claimed a 20-50x speed up. To their credit they posted videos and links to the repo of their work. And when you check the work, a large portion of it was hand rolling an ORM (via an LLM). Relatively solved problem that an LLM would excel at, but also not meaningfully moving the needle when you could use an existing library. And likely just creating more debt…

I've hand-rolled my own ultra-light ORM because the off-the-shelf ones always do 100 things you don't need.*

And of course the open source ones get abandoned pretty regularly. Type ORM, which a 3rd party vendor used on an app we farmed out to them, mutates/garbles your input array on a multi-line insert. That was a fun one to debug. The issue has been open forever and no one cares. https://github.com/typeorm/typeorm/issues/9058

So yeah, if I ever need an ORM again, I'm probably rolling my own.

*(I know you weren't complaining about the idea of rolling your own ORM, I just wanted to vent about Type ORM. Thanks for listening.)

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#554

Post by sothatsit »

acosmism wrote: post #471
simonw wrote: post #289

Earlier quoted context omitted.

Using AI tools makes me think harder.

harder != better

My thinking is definitely better. I spend more time worrying about the specific architecture, memory layout, GPU features, etc. to come up with ideas for optimisations, and I think less about specific implementation details. I’ve gotten a better mental model of our code faster because of this. I have also found substantial speed ups by thinking about the problem at a higher level, while iterating on implementation details quickly using Opus.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#555

Post by puchatek »

zwnow wrote: post #542
Bridged7756 wrote: post #454

Earlier quoted context omitted.

Opus 4.5 is writing code that Opus 5.0 will refactor and extend. And Opus 5.5 will take that code and rewrite it in C from the ground up. And Opus 6.0 will take that code and make it assembly. And Opus 7.0 will design its own CPU. And Opus 8.0 will make a factory for its own CPUs. And Opus 9.0 will populate mars. And Opus 10.0 will be able to achieve AGI. And Opus 11.0 will find God. And Opus 12.0 will make us a time…

I also love how AI enthusiasts just ignore the issue of exhausted training data... You cant just magically create more training data. Also synthetic training data reduces the quality of models.

That's been my main argument for why LLMs might be at their zenith. But I recently started wondering whether all those codebases we expose to them are maybe good enough training data for the next generation. It's not high quality like accepted stackoverflow answers but it's working software for the most part.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#556

Post by tripledry »

seanmcdirmid wrote: post #552
tripledry wrote: post #549

Putting the performance aside for now as I just started trying out Opus 4.5, can't say too much yet, I don't hype or hate AI as of now, it's simply useful. Time will tell what happens, but if programming becomes "prompt engineering", I'm planning on quitting my job and pivoting to something else. It's nice to get stuff working fast, but AI just sucks the joy out of building for me. Trying to not feel the pressure/anx…

Pity, prompt engineering is just another kind of programming, I find it to be fun, but I guess lots of other people would see it differently.

Indeed it is another kind of programming, I simply don't enjoy it.

But it is also very early to say, maybe the next iteration of tools will completely change my perspective, I might enjoy it some day!

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#557

Post by TeMPOraL »

zwnow wrote: post #542
Bridged7756 wrote: post #454

Earlier quoted context omitted.

Opus 4.5 is writing code that Opus 5.0 will refactor and extend. And Opus 5.5 will take that code and rewrite it in C from the ground up. And Opus 6.0 will take that code and make it assembly. And Opus 7.0 will design its own CPU. And Opus 8.0 will make a factory for its own CPUs. And Opus 9.0 will populate mars. And Opus 10.0 will be able to achieve AGI. And Opus 11.0 will find God. And Opus 12.0 will make us a time…

I also love how AI enthusiasts just ignore the issue of exhausted training data... You cant just magically create more training data. Also synthetic training data reduces the quality of models.

They don't ignore it, they just know it's not an actual problem.

It saddens me to see AI detractors being stuck in 2022 and still thinking language models are just regurgitating bits of training data.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#558

Post by jjaksic »

maplethorpe wrote: post #421
whynotminot wrote: post #416

Earlier quoted context omitted.

> The hard thing about engineering is not "building a thing that works", its building it the right way, in an easily understood way, in a way that's easily extensible. You’re talking like in the year 2026 we’re still writing code for future humans to understand and improve. I fear we are not doing that. Right now, Opus 4.5 is writing code that later Opus 5.0 will refactor and extend. And so on.

Yeah I think it's a mistake to focus on writing "readable" or even "maintainable" code. We need to let go of these aging paradigms and be open to adopting a new one.

Do readability and maintainability not matter when AI "reads" and maintains the code? I'm pretty sure they do.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#559

Post by sothatsit »

pugworthy wrote: post #478
sothatsit wrote: post #321

Earlier quoted context omitted.

I like these examples that predictably show the weaknesses of current models. This reminds me of that example where someone asked an agent to improve a codebase in a loop overnight and they woke up to 100,000 lines of garbage [0]. Similarly you see people doing side-by-side of their implementation and what an AI did, which can also quite effectively show how AI can make quite poor architecture decisions. This is why…

To me, this doesn't show the weakness of current models, it shows the variability of prompts and the influence on responses. Because without the prompt it's hard to tell what influenced the outcome. I had this long discussion today with a co-worker about the merits of detailed queries with lots of guidance .md documents, vs just asking fairly open ended questions. Spelling out in great detail what you want, vs just g…

It is definitely a weakness of current models. The fact that people find ways around those weaknesses does not mean the weaknesses do not exist.

Your approach is also very similar to spec driven development. Your spec is just a conversation instead of a planning document. Both approaches get ideas from your brain into the context window.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#560

Post by [deleted] »

BatteryMountain wrote: post #504
OldGreenYodaGPT wrote: post #33

Most software engineers are seriously sleeping on how good LLM agents are right now, especially something like Claude Code. Once you’ve got Claude Code set up, you can point it at your codebase, have it learn your conventions, pull in best practices, and refine everything until it’s basically operating like a super-powered teammate. The real unlock is building a solid set of reusable “skills” plus a few agents for th…

The crazy part is, once you have it setup and adapted your workflow, you start to notice all sorts of other "small" things: claude can call ssh and do system admin tasks. It works amazingly well. I have 3 VM's, which depends on each other (proxmox with openwrt, adguard, unbound), and claude can prove to me that my dns chains works perfectly, my firewalls are perfect etc as claude can ssh into each. Setting up service…

[deleted]
Post reply on HN