Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Opus 4.5 is not the normal AI agent experience that I have had thus far

burkeholland.github.io

Post reply on HN
Print view View original

721–730 of 1001 posts

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#721

Post by mjr00 »

redhale wrote: post #673
multisport wrote: post #320

What bothers me about posts like this is: mid-level engineers are not tasked with atomic, greenfield projects. If all an engineer did all day was build apps from scratch, with no expectation that others may come along and extend, build on top of, or depend on, then sure, Opus 4.5 could replace them. The hard thing about engineering is not "building a thing that works", its building it the right way, in an easily unde…

Not necessarily responding to you directly, but I find this take to be interesting, and I see it every time an article like this makes the rounds. Starting back in 2022/2023: - (~2022) It can auto-complete one line, but it can't write a full function. - (~2023) Ok, it can write a full function, but it can't write a full feature. - (~2024) Ok, it can write a full feature, but it can't write a simple application. - (~2…

This is disingenuous because LLMs were already writing full, simple applications in 2023.[0]

They're definitely better now, but it's not like ChatGPT 3.5 couldn't write a full simple todo list app in 2023. There were a billion blog posts talking about that and how it meant the death of the software industry.

Plus I'd actually argue more of the improvements have come from tooling around the models rather than what's in the models themselves.

[0] eg https://www.youtube.com/watch?v=GizsSo-EevA

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#722

Post by VMG »

zarzavat wrote: post #487
josu wrote: post #411

Earlier quoted context omitted.

>So my verdict is that it's great for code analysis, and it's fantastic for injecting some book knowledge on complex topics into your programming, but it can't tackle those complex problems by itself. I don't think you've seen the full potential. I'm currently #1 on 5 different very complex computer engineering problems, and I can't even write a "hello world" in rust or cpp. You no longer need to know how to write co…

How are you qualified to judge its performance on real code if you don't know how to write a hello world? Yes, LLMs are very good at writing code, they are so good at writing code that they often generate reams of unmaintainable spaghetti. When you submit to an informatics contest you don't have paying customers who depend on your code working every day. You can just throw away yesterday's code and start afresh. Clau…

> Claude is very useful but it's not yet anywhere near as good as a human software developer. Like an excitable puppy it needs to be kept on a short leash.

The skill of "a human software developer" is in fact a very wide distribution, and your statement is true for a ever shrinking tail end of that

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#723

Post by Scea91 »

redhale wrote: post #673
multisport wrote: post #320

What bothers me about posts like this is: mid-level engineers are not tasked with atomic, greenfield projects. If all an engineer did all day was build apps from scratch, with no expectation that others may come along and extend, build on top of, or depend on, then sure, Opus 4.5 could replace them. The hard thing about engineering is not "building a thing that works", its building it the right way, in an easily unde…

Not necessarily responding to you directly, but I find this take to be interesting, and I see it every time an article like this makes the rounds. Starting back in 2022/2023: - (~2022) It can auto-complete one line, but it can't write a full function. - (~2023) Ok, it can write a full function, but it can't write a full feature. - (~2024) Ok, it can write a full feature, but it can't write a simple application. - (~2…

> - (~2023) Ok, it can write a full function, but it can't write a full feature.

The trend is definitely here, but even today, heavily depends on the feature.

While extra useful, it requires intense iteration and human insight for > 90% of our backlog. We develop a cybersecurity product.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#724

Post by blitz_skull »

mjr00 wrote: post #721
redhale wrote: post #673

Earlier quoted context omitted.

Not necessarily responding to you directly, but I find this take to be interesting, and I see it every time an article like this makes the rounds. Starting back in 2022/2023: - (~2022) It can auto-complete one line, but it can't write a full function. - (~2023) Ok, it can write a full function, but it can't write a full feature. - (~2024) Ok, it can write a full feature, but it can't write a simple application. - (~2…

This is disingenuous because LLMs were already writing full, simple applications in 2023.[0] They're definitely better now, but it's not like ChatGPT 3.5 couldn't write a full simple todo list app in 2023. There were a billion blog posts talking about that and how it meant the death of the software industry. Plus I'd actually argue more of the improvements have come from tooling around the models rather than what's i…

What LLM were you using to build full applications in 2023? That certainly wasn’t my experience.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#725

Post by blitz_skull »

multisport wrote: post #320

What bothers me about posts like this is: mid-level engineers are not tasked with atomic, greenfield projects. If all an engineer did all day was build apps from scratch, with no expectation that others may come along and extend, build on top of, or depend on, then sure, Opus 4.5 could replace them. The hard thing about engineering is not "building a thing that works", its building it the right way, in an easily unde…

[flagged]

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#726

Post by rybosworld »

rat9988 wrote: post #681
bayindirh wrote: post #677

Earlier quoted context omitted.

Well, the first 90% is easy, the hard part is the second 90%. Case in point: Self driving cars. Also, consider that we need to pirate the whole internet to be able to do this, so these models are not creative. They are just directed blenders.

> Well, the first 90% is easy, the hard part is the second 90%. You'd need to prove that this assertion applies here. I understand that you can't deduce the future gains rate from the past, but you also can't state this as universal truth.

The saying is more or less treated as a truism at this point. OP isn't claiming something original and the onus of proving it isn't on them imo.

I've heard this same thing repeated dozens of times, and for different domains/industries.

It's really just a variation of the 80/20 rule.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#727

Post by mjr00 »

blitz_skull wrote: post #724
mjr00 wrote: post #721

Earlier quoted context omitted.

This is disingenuous because LLMs were already writing full, simple applications in 2023.[0] They're definitely better now, but it's not like ChatGPT 3.5 couldn't write a full simple todo list app in 2023. There were a billion blog posts talking about that and how it meant the death of the software industry. Plus I'd actually argue more of the improvements have come from tooling around the models rather than what's i…

What LLM were you using to build full applications in 2023? That certainly wasn’t my experience.

Just from googling, here's a video "Use ChatGPT to Code a Full Stack App" from May 18, 2023.[0]

There's a lot of non-ergonomic copy and pasting but it's definitely using an LLM to build a full application.

[0] https://www.youtube.com/watch?v=GizsSo-EevA

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#728

Post by herpdyderp »

multisport wrote: post #320

What bothers me about posts like this is: mid-level engineers are not tasked with atomic, greenfield projects. If all an engineer did all day was build apps from scratch, with no expectation that others may come along and extend, build on top of, or depend on, then sure, Opus 4.5 could replace them. The hard thing about engineering is not "building a thing that works", its building it the right way, in an easily unde…

On the contrary, Opus 4.5 is the best agent I’ve ever used for making cohesive changes across many files in a large, existing codebase. It maintains our patterns and looks like all the other code. Sometimes it hiccups for sure.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#729

Post by ozim »

mrheosuper wrote: post #669
ozim wrote: post #600

Earlier quoted context omitted.

*I've worked on teams where multiple engineers argued about the "right" way to build something. I remember thinking that they had biases based on past experiences and assumptions about what mattered. It usually took an outsider to proactively remind them what actually mattered to the business case.* Gosh I am so tired with that one - someone had a case that burned them in some previous project and now his life missio…

>someone had a case that burned them in some previous project and now his life mission is to prevent that from happening ever again Isn't that what makes them senior ? If you dont want that behaviour, just hire a bunch of fresh grad.

Nope, not realizing something doesn't apply and not being able to take in arguments is cargo culting not being a senior.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#730

Post by HarHarVeryFunny »

redhale wrote: post #673
multisport wrote: post #320

What bothers me about posts like this is: mid-level engineers are not tasked with atomic, greenfield projects. If all an engineer did all day was build apps from scratch, with no expectation that others may come along and extend, build on top of, or depend on, then sure, Opus 4.5 could replace them. The hard thing about engineering is not "building a thing that works", its building it the right way, in an easily unde…

Not necessarily responding to you directly, but I find this take to be interesting, and I see it every time an article like this makes the rounds. Starting back in 2022/2023: - (~2022) It can auto-complete one line, but it can't write a full function. - (~2023) Ok, it can write a full function, but it can't write a full feature. - (~2024) Ok, it can write a full feature, but it can't write a simple application. - (~2…

Sure, eventually we'll have AGI, then no worries, but in the meantime you can only use the tools that exist today, and dreaming about what should be available in the future doesn't help.

I suspect that the timeline from autocomplete-one-line to autocomplete-one-app, which was basically a matter of scaling and RL, may in retrospect turn out to have been a lot faster that the next LLM to AGI step where it becomes capable of using human level judgement and reasoning, etc, to become a developer, not just a coding tool.

Post reply on HN