Live data from Hacker News

Opus 4.5 is not the normal AI agent experience that I have had thus far

burkeholland.github.io

851–860 of 1001 posts

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#851

Every time I see a post like this on HN I try again and every time I come to the same conclusion. I have never see one agent managing to pull something off that I could instantly ship. It still ends up being very junior code. I just tried again and ask Opus to add custom video controls around ReactPlayer. I started in Plan mode which looked overal good (used our styling libs, existing components, icons and so on). I…

I have a vanilla JS project. I find that very small llms are able to work on it with no issue. (Including complete rewrites.) But I asked even large LLMs to port it to React and they all consistently fail. Basic functionality broken, rapid memory leaks.

So I just stuck with vanilla JS.

n = 1 but React might not be a great thing to test this stuff with. For the man and the machine! I tried and failed to learn React properly like 8 times but I've shipped multiple full stack things in like 5 other languages no problem.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#852

Most software engineers are seriously sleeping on how good LLM agents are right now, especially something like Claude Code. Once you’ve got Claude Code set up, you can point it at your codebase, have it learn your conventions, pull in best practices, and refine everything until it’s basically operating like a super-powered teammate. The real unlock is building a solid set of reusable “skills” plus a few agents for th…

I'm at the point where I say fuck it, let them sleep.

The tech industry just went through an insane hiring craze and is now thinning out. This will help to separate the chaff from the wheat.

I don't know why any company would want to hire "tech" people who are terrified of tech and completely obstinate when it comes to utilizing it. All the people I see downplaying it take a half-assed approach at using it then disparage it when it's not completely perfect.

I started tinkering with LLMs in 2022. First use case, speak in natural english to the llm, give it a json structure, have it decipher the natural language and fill in that json structure (vacation planning app, so you talk to it about where/how you want to vacation and it creates the structured data in the app). Sometimes I'd use it for minor coding fixes (copy and paste a block into chatgpt, fix errors or maybe just ideation). This was all personal project stuff.

At my job we got LLM access in mid/late 2023. Not crazy useful, but still was helpful. We got claude code in 2024. These days I only have an IDE open so I can make quick changes (like bumping up a config parameter, changing a config bool, etc.). I almost write ZERO code now. I usually have 3+ claude code sessions open.

On my personal projects I'm using Gemini + codex primarily (since I have a google account and chatgpt $20/month account). When I get throttled on those I go to claude and pay per token. I'll often rip through new features, projects, ideas with one agent, then I have another agent come through and clean things up, look for code smells, etc. I don't allow the agents to have full unfettered control, but I'd say 70%+ of the time I just blindly accept their changes. If there are problems I can catch them on the MR/PR.

I agree about the low hanging fruit and I'm constantly shocked at the sheer amount of FUD around LLMs. I want to generalize, like I feel like it's just the mid/jr level devs that speak poorly about it, but there's definitely senior/staff level people I see (rarely, mind you) that also don't like LLMs.

I do feel like the online sentiment is slowly starting to change though. One thing I've noticed a lot of is that when it's an anonymous post it's more likely to downplay LLMs. But if I go on linkedin and look at actual good engineers I see them praising LLMs. Someone speaking about how powerful the LLMs are - working on sophisticated projects at startups or FAANG. Someone with FUD when it comes to LLM - web dev out of Alabama.

I could go on and on but I'm just ranting/venting a little. I guess I can end this by saying that in my professional/personal life 9/10 of the top level best engineers I know are jumping on LLMs any chance they get. Only 1/10 talks about AI slop or bullshit like that.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#853
post #755

It’s a bit strange how anecdotes have become acceptable fuel for 1000 comment technical debates. I’ve always liked the quote that sufficiently advanced tech looks like magic, but its mistake to assume that things that look like magic also share other properties of magic. They don’t. Software engineering spans over several distinct skills: forming logical plans, encoding them in machine executable form(coding), making…

I can only speak for myself, but it feels like playing with fire to productize this stuff too quick.

Like, I woke up one day and a magical owl told me that I was a wizard. Now I control the elements with a flick of my wrist - which I love. I can weave the ether into databases, apps, scripts, tools, all by chanting a simple magical invocation. I create and destroy with a subtle murmur.

Do I want to share that power? Naturally, it would be lonely to hoard it and despite the troubles at the Unseen University, I think that schools of wizards sharing incantations can be a powerful good. But do I want to share it with everybody? That feels dangerous.

It's like the early internet - having a technical shelf to climb up before you can use the thing creates a kind of natural filter for at least the kinds of people that care enough to think about what they're doing and why. Selecting for curiosity at the very least.

That said, I'm also interested in more data from an engineering perspective. It's not a simple thing and my mind is very much straddling the crevasse here.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#854

I'm tired of constantly debating the same thing again and again. Where are the products? Where is some great performing software all LLM/agent crafted? All I see is software bloatness and decline. Where is Discord that uses just a bunch of hundreds megs of ram? Where is unbloated faster Slack? Where is the Excel killer? Fast mobile apps? Browsers and the web platform improved? Why Cursor team don't use Cursor to get…

With all due respect somebody could launch a version of Discord that's 10x faster tomorrow and nobody would know about it It's very difficult to unseat those incumbents, especially those with strong network effects. Plus the people that work in those larger companies are not at the edge of AI coding at all and not motivated to rock the boat

Get the feeling with the pending IPO, there might be some challengers to discord that get more traction due to the protracted enshittification of the platform (cf. bluesky)

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#855

Earlier quoted context omitted.

I know what you are talking about, but there is more to life than just product-market fit. Hardly any of us are working on Postgres, Photoshop, blender, etc. but it's not just cope to wish we were. It's good to think about the needs to business and the needs of society separately. Yes, the thing needs users, or no one is benefiting. But it also needs to do good for those users, and ultimately, at the highest caliber,…

Users will not care about the quality of your code, or the backed architecture, or your perfectly strongly typed language. They only care about their problems and treat their computers like an appliance. They don't care if it takes 10 seconds or 20 seconds. They don't even care if it has ads, popups, and junk. They are used to bloatware and will gladly open their wallets if the tool is helping them get by. It's an un…

I know the customer's couldn't care about the quality of the code they see. But the idea that they don't care about software being bad/laggy/bloated ever, because it "still solves problems", doesn't stand up to scrutiny as an immutable fact of the universe. Market conditions can change.

I'm banking on a future that if users feel they can (perhaps vibe) code their own solutions, they are far less likely to open their wallets for our bloatware solutions. Why pay exorbitant rents for shitty SaaS if you can make your own thing ad-free, exactly to your own mental spec?

I want the "computers are new, programmers are in short supply, customer is desperate" era we've had in my lifetime so far to come to a close.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#856

Every time I see a post like this on HN I try again and every time I come to the same conclusion. I have never see one agent managing to pull something off that I could instantly ship. It still ends up being very junior code. I just tried again and ask Opus to add custom video controls around ReactPlayer. I started in Plan mode which looked overal good (used our styling libs, existing components, icons and so on). I…

Have you tried Roo Code in "Orchestrator" mode? I find it generally "chews" the tasks I give it to then spoon feed into sub-tasks in "Code" (or others) mode, leaving less room to stray from very focused "bite-sized" changes.

I do need to steer it sometimes, but since it doesn't change a lot at a time, I can usually guide the agent and stop the disaster before it spreads.

A big caveat is I haven't tried heavy front-end stuff with it, more django stuff, and I'm pretty happy with the output.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#857
I'm always surprised to never see any comments in those discussions from people who just like coding, learning, solving problems… I mean, it's amazing that LLMs can build an image converter or whatever you dream of, in a language you don't know, in a field you are not familiar with, in 1 hour, for 30 cents… I'm sure your boss and shareholders love it. But where is the fun in that? For me it kills any interest in doing what I'm doing. I'm lucky enough to work in a place where using LLMs is not mandatory (yet), I don't know how people can make it through the day just writing prompts and reviewing AI slop.

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#858

I'm kind of surprised how many people are okay with deploying code that hasn't been audited. I read If Anyone Builds It Everyone Dies over the break. The basic premise was that we can't "align" AI so when we turn it loose in an agent loop what it produces isn't necessarily what we want. It may be on the surface, to appease us and pass a cursory inspection, but it could embed other stuff according to other goals. On t…

There's a really interesting story I read somewhere about some application which used neural nets to optimize for a goal (this was a while ago, it could have been merkel trees or something, who knows, not super important)

And everything worked really well until they switched chip set.

At which point the same model failed entirely. Upon inspection it turned out the AI model had learned that overloading particular registers would cause such an electrical charge buildup that transistors on other pathways would be flipped.

And it was doing this in a coordinated manner in order to get the results it wanted lol.

I can't find any references in my very cursory searches, but your comment reminded me of the story

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#859
post #848
post #702

Earlier quoted context omitted.

This deserves more upvotes. Even if there is a "fully vibe-coded" product that has real customers, the fact that it's vibe-coded means that others can do the same. Unless you have a secret LLM or some magical prompts that make the code better/more efficient than your competitions, your vibe coded product has no advantage over competition and no moat. What actually matters is everything else -- user experience (which…

I'm not sure what point you're making here. Tech is rarely the moat, you even get to that point at the end of your post. The "vibe coding" advantage is faster time to market, faster iterations, etc. These things will help you get that user experience, integrations, etc.

Faster, faster, faster. All to release something that is slower, by people that now know lesser, with bloat that explodes. All for a yet another useless saas that nobody or fee people wants and a chance to virtue signaling your vibe coded product on HN. Real world successfull products are orthogonal to this approach, it doesnt work anymore in today's world

Re: Opus 4.5 is not the normal AI agent experience that I have had thus far

#860
post #813

Earlier quoted context omitted.

I made a similar comment on a different thread, but I think it also fits here: I think the disconnect between engineers is due to their own context. If you work with frontend applications, specially React/React Native/HTML/Mobile, your experience with LLMs is completely different than the experience of someone working with OpenGL, io_uring, libev and other lower level stuff. Sure, Opus 4.5 can one shot Windows utilit…

We have an in-house, Rust-based proxy server. Claude is unable to contribute to it meaningfully outside of grunt work like minor refactors across many files. It doesn't seem to understand proxying and how it works on both a protocol level and business logic level. With some entirely novel work we're doing, it's actually a hindrance as it consistently tells us the approach isn't valid/won't work (it will) and then ent…

This isn't meant as a criticism, or to doubt your experience, but I've talked to a few people who had experiences like this. But, I helped them get Claude code setup, analyze the codebase and document the architecture into markdown (edit as needed after), create an agent for the architecture, and prompt it in an incremental way. Maybe 15-30 minutes of prep. Everyone I helped with this responded with things like "This is amazing", "Wow!", etc.

For some things you can fire up Claude and have it generate great code from scratch. But for bigger code bases and more complex architecture, you need to break it down ahead of time so it can just read about the architecture rather than analyze it every time.

Post reply on HN