Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agentic Engineering Patterns

simonwillison.net

Post reply on HN
Print view View original

271–280 of 341 posts

Re: Agentic Engineering Patterns

#271

Post by layer8 »

slaye wrote: post #168

Simon, if you're reading this, I'd be really curious to hear your thoughts on how to effectively conduct code reviews in a world where "code is cheap". One of the biggest struggles I have on my team is coworkers straight up vibing parts of the code and not understanding or guiding the architecture of subsystems. Or at least, not writing code in a way that is meant to be understood by others. Then when I go through th…

> I'm met with much pushback because "it works, why change it"?

This is an educational problem, and is unlikely to be easy to fix in your team (though I might be wrong). I would suggest to change to a team or company with a culture that values being able to reason about one’s software.

Re: Agentic Engineering Patterns

#272

Post by jerf »

simonw wrote: post #268
jerf wrote: post #264

Earlier quoted context omitted.

Worse yet, the problems are going to be real. There's a lifecycle to these hype runs, even when the thing behind the hype is plenty real. We're still in the phase where if you criticize AI you get told you don't "get it", so people are holding back some of their criticisms because they won't be received well. In this case, I'm not talking about the criticisms of the people standing back and taking shots at the tech,…

> And there are going to be problems that come from using vast quantities of AI on a code base, especially of the form "created so much code my AI couldn't handle it anymore and neither could any of the humans involved". There's going to need to be a discussion on techniques on how to handle this. There's going to be characteristic problems and solutions. That's essentially the thing we are calling "cognitive debt".…

Yeah, it's hard to even get started until we can go three months without a significant improvement in the AIs. Today's characteristic failures may not be 2027's characteristic failures. Example: Today I'm complaining that the AIs tend not to abstract as often as I'd like, but it's not hard to imagine it flipping until they're all architecture astronauts instead.

Re: Agentic Engineering Patterns

#273

Post by aprdm »

Balgair wrote: post #224
simonw wrote: post #199

Earlier quoted context omitted.

> Agree with everything else you said except this. Yeah, I'm not sure I agree with what I said there myself! > Incorrect, but not visibly incorrect, code will sit in place for years. If you let incorrect code sit in place for years I think that suggests a gap in your wider process somewhere. I'm still trying to figure out what closing those gaps looks like. The StrongDM pattern is interesting - having an ongoing swar…

This kinda gets into the idea of AIs as droids right? So, you have a code writing droid that is aligned towards writing good clean code that humans can read. Then you have an implementation droid that goes into actually launching and running the code and is aligned with business needs and expenses. And you have a QA droid that stress tests the code and is aligned with the hacker mindset and is just slightly evil, so…

These are just agents with a different name ? People have been working like that today.

Re: Agentic Engineering Patterns

#274

Post by keeda »

lbreakjai wrote: post #109

We're going to do it again, aren't we? We're going to take something simple and sensible ("write tests first", "small composable modules", etc.), give it a fancy complicated name ("Behavior-Constrained Implementation Lifecycle pattern", "Boundary-Scoped Processing Constructs pattern", etc.), and create an entire industry of consultants and experts selling books and enterprise coaching around it, each swearing they ha…

I'm not sure what this comment is addressing, I didn't find any fancy terms in TFA? If it's the title of the article itself, it seems simpler than "Things that help writing code effectively with AI agents."

> You can just ask it to do what you want.

Yes, but very clearly, as any HN thread on AI shows, different people are having VERY different outcomes with it. And I suspect it is largely the misconception that it will magically "just do what you want" that leads to poor outcomes.

The techniques mentioned -- coding, docs, modularity etc. -- may seem obvious now, but only recently did we realize that the primary principle emerging is "what's good for humans is good for agents." That was not at all obvious when we started off. It is doubly counter-intuitive given the foremost caveat has been "Don't anthropomorphize AI." I'm finding that is actually a decent way to understand these models. They are unnervingly like us, yet not like us.

All that to say, AI is essentially black magic and it is not yet obvious how to use it well for all people and all use-cases, so yes, more exposition is warranted.

Re: Agentic Engineering Patterns

#275

Post by dgunay »

A lot of this is just things that high-functioning human teams were already doing: automate testing, explain your PRs to guide reviewers, demoing work, not just throwing bad code over the wall during code review, etc.

Re: Agentic Engineering Patterns

#276

Post by luca-ctx »

> I don't let LLMs write text for my blog.

Thank you Simon and I'm sure you would quickly fall off from #1 blogger on HN if you did. I insist on this for myself as well.

Somehow we are all getting really good at detecting "written by AI" with primal intuition.

Re: Agentic Engineering Patterns

#277

Post by tshaddox »

simonw wrote: post #205
jihadjihad wrote: post #197

Earlier quoted context omitted.

Among many other possible examples, here are a few [0] from Ruby that I've seen in the wild before LLMs, and still see today spat out by LLMs. 0: https://www.codewithjason.com/examples-pointless-rspec-tests...

I do see agents pop out tests that look like this occasionally: it { expect(classroom).to have_many(:students) } If I catch them I tell them not to and they remove it again, but a few do end up slipping through. I'm not sure that they're particularly harmful any more though. It used to be that they added extra weight to your test suite, meaning when you make changes you have to update pointless tests. But if the agen…

I don’t love tests like that either, but I’ve seen a lot of them (long before the generative AI era) and heard reasonable people make arguments in favor of them.

Admittedly, in the absence of halfway competent static type checking, it does seem like a good way to prevent what would be a very bad regression. It doesn’t seem worse than tests which check that a certain property is non-null (when that’s a vital business requirement and you’re using a language without a competent type system).

Re: Agentic Engineering Patterns

#278

Post by jgraettinger1 »

ramoz wrote: post #261
jgraettinger1 wrote: post #257

Earlier quoted context omitted.

Maintaining a high-quality requirements / specification document for large features prior to implementation, and then referencing it in "plan mode" prompts, feels like consensus best practice at this stage. However a thing I'm finding quite valuable in my own workflows, but haven't seen much discussion of, is spending meaningful time with AI doing meta-planning of that document. For example, I'll spend many sessions…

> consensus best practice I'm not sure I agree with this. I don't think there needs to be a whole spec & documentation process before plan mode. There is alternative thought leadership that the waterfall approach for building out projects is not the right Agentic pattern[1]. Planning itself can be such an intensive process where you're designing and figuring out the specs on the fly in a focused manner for the thing…

I think there's a Danger Zone when planning is light-weight and iterative, and code is cheap, but reviewing code is expensive: it leads to a kind of local hill-climbing.

Suppose you iterate through many sessions of lightweight planning, implementation, and code review. It _feels_ high velocity, you're cranking through the feature, but you've also invested a lot of your time and energy (planning isn't free, and code review and fit-for-purpose checks, in particular, are expensive). As often happens -- with or without AI -- you get towards the end and realize: there might have been a fundamentally better approach to take.

The tradeoff of that apparent velocity is that _now_ course correction is much more challenging. Those ephemeral plans are now gone. The effort you put into providing context within those plans is gone. You have an locally optimal solution, but you don't have a great way of expressing how to start over from scratch pointed in a slightly different direction.

I think that part can be really valuable, because given a sufficiently specific arrow, the AI can just rip.

Whether it's worth the effort, I suppose, depends on how high-conviction you are on your original chosen approach.

Re: Agentic Engineering Patterns

#279

Post by mexicocitinluez »

nightski wrote: post #209
mexicocitinluez wrote: post #170

Earlier quoted context omitted.

> The problem is that the average person doesn't know how what their actual problems are in sufficient detail to get a working solution. When you get down to breaking down that problem... you become a programmer. Agreed. I've spent the last few years building an EMR at an actual agency and the idea that users know what they want and can articulate it to a degree that won't require ANY technical decisions is pure fant…

Right now with agents this is definitely going to continue to be the case. That said, at the end of the day engineers work with stakeholders to come up with a solution. I see no reason why an agent couldn't perform this role in the future. I say this as someone who is excited but at the same time terrified of this future and what it means to our field. I don't think we'll get their by scaling current techniques (Dari…

> I see no reason why an agent couldn't perform this role in the future.

Yea, we'll see. I didn't think they'd come this far, but they have. Though, the cracks I still see seem to be more or less just how LLMs work.

It's really hard to accurately assess this given how much I have at stake.

> and he's far more qualified albeit biased

Yea, I think biased is an understatement. And he's working on a very specific product. How much can any one person really understand the entire industry or the scope of all it's work? He's worked at Google and OpenAi. Not exactly examples of your standard line-of-business software building.

Re: Agentic Engineering Patterns

#280

Post by pixl97 »

devin wrote: post #226
Terr_ wrote: post #219

Earlier quoted context omitted.

I predict the main democratization change is going to be how easy people can make plumbing that doesn't require--or at least not obviously require--such specificity or mental-modeling of the business domain. For example, "Generate me some repeatable code to ask system X for data about Y, pull out value Z, and submit it to system W."

What happens when value Z is not >= X? What happens when value Z doesn't exist, but values J and K do? What should be done when... I hear what you're saying, but I think it's going to be entertaining watching people go "I guess this is why we paid Bob all of that money all those years".

> when value Z is not >= X?

Is your AI not even doing try/catch statements, what century are you in?

Post reply on HN