Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Opus 4.5 is not the normal AI agent experience that I have had thus far

burkeholland.github.io

Post reply on HN
Print view View original

941–950 of 1001 posts

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#941

Post by neochief »

throw10920 wrote: post #803

Does anyone have a boring, multi-hour-long coding session with an agent that they've recorded and put on Vimeo or something? As many other commentators have said, individual results vary extremely widely. I'd love to be able to look at the footage of either someone who claims a 10x productivity increase, or someone who claims no productivity increase, to see what's happening.

I tried to make several, but they all end up prematurely when the agent hits a wall in an hour or so, unless you make trivial shit.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#942

Post by throw10920 »

neochief wrote: post #941
throw10920 wrote: post #803

Does anyone have a boring, multi-hour-long coding session with an agent that they've recorded and put on Vimeo or something? As many other commentators have said, individual results vary extremely widely. I'd love to be able to look at the footage of either someone who claims a 10x productivity increase, or someone who claims no productivity increase, to see what's happening.

I tried to make several, but they all end up prematurely when the agent hits a wall in an hour or so, unless you make trivial shit.

That sounds like genuinely useful data, though! Please reply if you end up posting them!

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#943

Post by ModernMech »

josu wrote: post #411
mcv wrote: post #133

Opus 4.5 ate through my Copilot quota last month, and it's already halfway through it for this month. I've used it a lot, for really complex code. And my conclusion is: it's still not as smart as a good human programmer. It frequently got stuck, went down wrong paths, ignored what I told it to do to do something wrong, or even repeat a previous mistake I had to correct. Yet in other ways, it's unbelievably good. I ca…

>So my verdict is that it's great for code analysis, and it's fantastic for injecting some book knowledge on complex topics into your programming, but it can't tackle those complex problems by itself. I don't think you've seen the full potential. I'm currently #1 on 5 different very complex computer engineering problems, and I can't even write a "hello world" in rust or cpp. You no longer need to know how to write co…

None of the problems you've shown there are anything close to "very complex computer engineering problems", they're more like "toy problems with widely-known solutions given to students to help them practice for when they encounter actually complex problems".

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#944

Post by ciupicri »

hollowturtle wrote: post #758
jwblackwell wrote: post #753

Earlier quoted context omitted.

With all due respect somebody could launch a version of Discord that's 10x faster tomorrow and nobody would know about it It's very difficult to unseat those incumbents, especially those with strong network effects. Plus the people that work in those larger companies are not at the edge of AI coding at all and not motivated to rock the boat

Totally disagree. One example is Zed which is very well known and it's faster than any other editor, wasn't built with AI though. > People on larger companies are not at the edge of AI coding False Microsoft is all in with Copilot, and I can't believe the company that created Copilot doesn't use it internally, I'd rather say they should be the ones that would know how to master it! Yet no better vscode, still bloated…

Zed fast? No way as I've mentioned last year https://news.ycombinator.com/item?id=42819817

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#945

Post by edg5000 »

gck1 wrote: post #809
edg5000 wrote: post #678

Earlier quoted context omitted.

This will work (if you add more details): "Have an agent investiate issue X in modules Y and Z. The agent should place a report at ./doc/rework-xyz-overview.md with all locations that need refactoring. Once you have the report, have agents refactor 5 classes each in parallel. Each agent writes a terse report in ./doc/rework-xyz/ When they are all done, have another agent check all the work. When that agent reports ev…

And you can automate all this so that it happens every time. I have an `/implement` command that is basically instructed to launch the agents and then do back and forth between them. Then there's a claude code hook that makes sure that all the agents, including the orchestrator and the agents spawned have respected their cycles - it's basically running `claude` with a prompt that tells it to read the plan file and se…

Interesting. Another thing I'll try is editing the system propmts. There are some projects floating around that can edit the minified JavaScript in the client. I also noticed that the "system tools" prompts take up ~5% context (10 ktok).

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#946

Post by edg5000 »

SeanAppleby wrote: post #383
multisport wrote: post #320

What bothers me about posts like this is: mid-level engineers are not tasked with atomic, greenfield projects. If all an engineer did all day was build apps from scratch, with no expectation that others may come along and extend, build on top of, or depend on, then sure, Opus 4.5 could replace them. The hard thing about engineering is not "building a thing that works", its building it the right way, in an easily unde…

One thing I've been tossing around in my head is: - How quickly is cost of refactor to a new pattern with functional parity going down? - How does that change the calculus around tech debt? If engineering uses 3 different abstractions in inconsistent ways that leak implementation details across components and duplicate functionality in ways that are very hard to reason about, that is, in conventional terms, an existe…

Good point. Most of the cost in dealing with tech debt is reading the code and noting the issues. I found that Claude can produce much better code when it has a functionally correct reference implementation. Also it's not needed to very specifically point out issues. I once mentioned "I see duplicate keys in X and Y, rework it to reduce repetition and verbosity". It came up with a much more elegant way to implement it.

So maybe doing 2-3 stages makes sense. First stage needs to be functionallty correct, but you accept code smells such as leaky abstractions, verbosity and repetition. In stage 2 and 3 you eliminate all this. You could integrate this all into the initial specification; you won't even see the smelly intermediate code; it only exists as a stepping stone for the model to iteratively refine the code!

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#949

Post by 3oil3 »

parliament32 wrote: post #867
JDye wrote: post #813

Earlier quoted context omitted.

We have an in-house, Rust-based proxy server. Claude is unable to contribute to it meaningfully outside of grunt work like minor refactors across many files. It doesn't seem to understand proxying and how it works on both a protocol level and business logic level. With some entirely novel work we're doing, it's actually a hindrance as it consistently tells us the approach isn't valid/won't work (it will) and then ent…

> I still believe those who rave about it are not writing anything I would consider "engineering". Correct. In fact, this is the entire reason for the disconnect, where it seems like half the people here think LLMs are the best thing ever and the other half are confused about where the value is in these slop generators. The key difference is (despite everyone calling themselves an SWE nowadays) there's a difference b…

When he said 'just look at what I'v been able to build', I was expecting anything but an 'image converter'

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#950

Post by Rapzid »

kypro wrote: post #310

It's been interesting watching HN shift in my direction on this in recent weeks... I had been saying since around summer of this year that coding agents were getting extremely good. The base model improvements were ok, but the agentic coding wrappers were basically game changers if you were using them right. Until recently they still felt very context limited, but the context problem increasingly feels like a solved…

> a steady stream of shippable products

Software/web meat shops have bean around since the dawn of the time.

I worked at McDonald's in my teens. One of the best managers I ever worked for was the manager at this store at this time(the owner rotated him between stores to help get things on track).

I'll never forget this one thing he said: "They have changed the Filet-O-Fish five times since I've been here, and each time it's become more profitable".

Congrats on making slop more profitable.

Post reply on HN