Live data from Hacker News

Opus 4.5 is not the normal AI agent experience that I have had thus far

burkeholland.github.io

231–240 of 1001 posts

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#231
post #155
post #153

Opus 4.5 really is something else. I've been having a ton of fun throwing absurdly difficult problems at it recently and it keeps on surprising me. A JavaScript interpreter written in Python? How about a WebAssembly runtime in Python? How about porting BurntSushi's absurdly great Rust optimized string search routines to C and making them faster ? And these are mostly just casual experiments, often run from my phone!

One of my first tests with it was "Write a Python 3 interpreter in JavaScript." It produced tests, then wrote the interpreter, then ran the tests and worked until all of them passed. I was genuinely surprised that it worked.

There are multiple Python 3 interpreters written in JavaScript that were very likely included in the training data. For example [1] [2] [3]

I once gave Claude (Opus 3.5) a problem that I thought was for sure too difficult for an LLM, and much to my surprise it spat out a very convincing solution. The surprising part was I was already familiar with the solution - because it was almost a direct copy/paste (uncredited) from a blog post that I read only a few hours earlier. If I hadn't read that blog post, I would have been none the wiser that copy/pasting Claude's output would be potential IP theft. I would have to imagine that LLMs solve a lot of in-training-set problems this way and people never realize they are dealing with a copyright/licensing minefield.

A more interesting and convincing task would be to write a Python 3 interpeter in JavaScript that uses register based bytecode instead of stack based, supports optimizing the bytecode by inlining procedures and constant folding, and never allocates memory (all work is done in a single user provided preallocated buffer). This would require integrating multiple disparate coding concepts and not regurgitating prior art from the training data

[1] https://github.com/skulpt/skulpt

[2] https://github.com/brython-dev/brython

[3] https://github.com/yzyzsun/PyJS

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#232

That final line: "Disclaimer: This post was written by a human and edited for spelling, grammer by Haiku 4.5" Yeah, GRAMMAR For all the wonderment of the article, tripping up on a penultimate word that was supposedly checked by AI suddenly calls into question everything that went before...

Presumably that disclaimer was added manually after Haiku had run the checks.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#233
post #208

Earlier quoted context omitted.

> Opus 4.5 really is at a new tier however. It just...works. Literally tried it yesterday. I didn't see a single difference with whatever model Claude Code was using two months ago. Same crippled context window. Same "I'll read 10 irrelevant lines from a file", same random changes etc.

200k+ tokens is a pretty big context window if you are feeding it the right context. Editors like Cursor are really good at indexing and curating context for you; perhaps it'd be worth trying something that does that better than Claude CLI does?

> a pretty big context window if you are feeding it the right context.

Yup. There's some magical "right context" that will fix all the problems. What is that right context? No idea, I guess I need to read a yet-another 20 000-word post describing magical incantations that you should or shouldn't do in the context.

The "Opus 4.5 is something else/nex tier/just works" claims in my mind means that I wouldn't need to babysit its every decision, or that it would actually read relevant lines from relevant files etc. Nope. Exact same behaviors as whatever the previous model was.

Oh, and that "200k tokens context window"? It's a lie. The quality quickly degrades as soon as Claude reaches somewhere around 50% of the context window. At 80+% it's nearly indistinguishable from a model from two years ago. (BTW, same for Codex/GPT with it's "1 million token window")

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#234

Earlier quoted context omitted.

[flagged]

It's a little weird how defensive people are about these tools. Did everyone really think being able to import a few npm packages, string together a few APIs, and run npx create-react-app was something a large number of people could do forever? The vast majority of coders in employment barely write anything more complex than basic CRUD apps. These jobs were always going to be automated or abstracted away sooner or la…

They made the "abstracted away the CRUD app", it's called Salesforce. Hows that going?

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#235

Earlier quoted context omitted.

I really think a lof of people tried AI coding earlier, got frustrated at the errors and gave up. That's where the rejection of all these doomer predictions comes from. And I get it. Coding with Claude Code really was prompting something, getting errors, and asking it to fix it. Which was still useful but I could see why a skilled coder adding a feature to a complex codebase would just give up Opus 4.5 really is at a…

This was me. I was a huge AI coding detractor on here for a while (you can check my comment history). But, in order to stay informed and not just be that grouchy curmudgeon all the time, I kept up with the models and regularly tried them out. Opus 4.5 is so much better than anything I've tried before, I'm ready to change my mind about AI assistance. I even gave -True Vibe Coding- a whirl. Yesterday, from a blank dire…

> Oh, and I did this all without ever opening a single source file or even looking at the proposed code changes while Opus was doing its thing. I don't even know Kotlin and still don't know it.

... says it all.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#236

Yeah Opus 4.5 is a massive step change in my experience. I feel like I’m working with a peer, not a junior I’m having to direct. I can give it highly ambiguous and poorly specified tasks and it… just does it. I will note that my experience varies slightly by language though. I’ve found it’s not as good at typescript.

It’s excellent at typescript in my experience.

It’s also way better than I am at finding bits of code for reuse. I tell it, “I think I wrote this thing a while back, but it may never have been merged, so you may need to search git history.” And presto, it finds it.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#237
post #215

what strikes me about these posts is they praise models for apps | utilities commonly found on GitHub. ie well known paths based on training data. what's never posted is someone building something that solves a real problem in the real world - that deals with messy data | interfaces. I like a.i to do the common routine tasks that I don't like to do like apply tailwind styles but being renter and faking productivity t…

I used it with gemini 3 in tandem to build an app to simulate thermal bridges because I want to insulate a house. I explored this in various directions and there are some functionalities not completed or sound, but the main part is good and tested against ISO/DIN test cases for this kind of problem. You can try it here, although the numeric simulations take quite a while in the cloud app

https://thermal-bridge.streamlit.app/

Disclaimer: I'm not a programmer or software engineer. I have a background in physics and understand some scripting in python and basic git. The code is messy at the moment because I explored/am still exploring to port it to another framework/language

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#238
post #103

Earlier quoted context omitted.

I teach at a university, and spend plenty of time programming for research and for fun. Like many others, I spent some time on the holidays trying to push the current generation of Cursor, Claude Code, and Codex as far as I could. (They're all very good.) I had an idea for something that I wanted, and in five scattered hours, I got it good enough to use. I'm thinking about it in a few different ways: 1. I estimate I…

This is where the LLM coding shines in my opinion, there's a list of things they are doing very well: - single scripts. Anything which can be reduced to a single script. - starting greenfield projects from scratch - code maintenance (package upgrades, old code...) - tasks which have a very clear and single definition. This isn't linked to complexity, some tasks can be both very complex but with a single definition. I…

I'm trying to determine what programming tasks are not in this list. :) I think it is trying to exclude adding new features and fixing bugs in existing code. I've done enough of that with LLMs, though not in large codebases.

I should say I'm hardly ever vibe-coding, unlike the original article. If I think I want code that will last, I'll steer the models in ways that lean on years of non-LLM experience. E.g., I'll reject results that might work if they violate my taste in code.

It also helps that I can read code very fast. I estimate I can read code 100x faster than most students. I'm not sure there is any way to teach that other than the old-fashioned way, which involves reading (and writing) a lot of code.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#239
post #103

Most software engineers are seriously sleeping on how good LLM agents are right now, especially something like Claude Code. Once you’ve got Claude Code set up, you can point it at your codebase, have it learn your conventions, pull in best practices, and refine everything until it’s basically operating like a super-powered teammate. The real unlock is building a solid set of reusable “skills” plus a few agents for th…

I teach at a university, and spend plenty of time programming for research and for fun. Like many others, I spent some time on the holidays trying to push the current generation of Cursor, Claude Code, and Codex as far as I could. (They're all very good.) I had an idea for something that I wanted, and in five scattered hours, I got it good enough to use. I'm thinking about it in a few different ways: 1. I estimate I…

What did you build? I think people talk passed eachother when people don't share what exactly they were trying to do and achieving success/failure.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#240

Earlier quoted context omitted.

I really think a lof of people tried AI coding earlier, got frustrated at the errors and gave up. That's where the rejection of all these doomer predictions comes from. And I get it. Coding with Claude Code really was prompting something, getting errors, and asking it to fix it. Which was still useful but I could see why a skilled coder adding a feature to a complex codebase would just give up Opus 4.5 really is at a…

> I really think a lof of people tried AI coding earlier, got frustrated at the errors and gave up. That's where the rejection of all these doomer predictions comes from. It's not just the deficiencies of earlier versions, but the mismatch between the praise from AI enthusiasts and the reality. I mean maybe it is really different now and I should definitely try uploading all of my employer's IP on Claude's cloud and…

I feel similar, I'm not against the idea that maybe LLMs have gotten so much better... but I've been told this probably 10 times in the last few years working with AI daily.

The funny part about rapidly changing industries is that, despite the fomo, there's honestly not any reward to keeping up unless you want to be a consultant. Otherwise, wait and see what sticks. If this summer people are still citing the Opus 4.5 was a game changing moment and have solid, repeatable workflows, then I'll happily change up my workflow.

Someone could walk into the LLM space today and wouldn't be significantly at a loss for not having paid attention to anything that had happened in the last 4 years other than learning what has stuck since then.

Post reply on HN