Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agentic Engineering Patterns

simonwillison.net

Post reply on HN
Print view View original

151–160 of 341 posts

Re: Agentic Engineering Patterns

#151

Post by logicprog »

Madmallard wrote: post #143
logicprog wrote: post #69

Earlier quoted context omitted.

I think it's nonsensical to insist that it would only be a subjective improvement. The tests either exist and ensure that there aren't bugs in certain areas, or they don't. The agent is either in a feedback loop with those tests and continues to work until it has satisfied them or it doesn't.

That sounds like a very specific implementation strategy related to TDD

Red-Green TDD is one of the main "agent patterns" Simon proposes, so it seemed relevant.

Also, the same thing applies to feedback loops with compilers and linters as well: they provide objective feedback that then the AI goes and fixes, verifiably resolving the feedback.

Even with less verifiable things like using specifications, the fact that it relies on less objective grounding metrics doesn't mean there's no change in the model's behavior. I'm sure if you looked at the code that a model produced and the amount of intervention necessary to get there for a model that was asked to produce something without a specification versus with one, you would definitely see an objective difference on average. We're already getting objective studies regarding AGENTS.MD

Re: Agentic Engineering Patterns

#152

Post by logicprog »

lbreakjai wrote: post #109

We're going to do it again, aren't we? We're going to take something simple and sensible ("write tests first", "small composable modules", etc.), give it a fancy complicated name ("Behavior-Constrained Implementation Lifecycle pattern", "Boundary-Scoped Processing Constructs pattern", etc.), and create an entire industry of consultants and experts selling books and enterprise coaching around it, each swearing they ha…

I think the problem is that because it talks and understands English and more or less does whatever you ask, the affordences aren't particularly clear. That's actually one of the biggest problems with the chatbot model of AI — it has the same problems as the CLI, in that it's extremely flexible and powerful and you can do a lot with it and add a lot to it, but it's really not clear what way of interacting with it is more or less effective than any other, or what it can or can't do well.

I think attempts to document the most effective things to ask it to do in order to get to your overall goal, as well as what it is and is not good for, is probably worth doing. It would be bad if it turned into a whole consultant marketing OOP coaching clusterfuck. Yeah, but building some kind of community knowledge that these things aren't like, demigods, they have limitations and during things one way or the other with them can be better is probably a good thing. At the very least in theory would cut down some of the hype?

Re: Agentic Engineering Patterns

#153

Post by Thews »

There was a mention of using agents to build projects into WASM. I've had the best luck telling it to use zig to compile to webassembly. It shortens the time to completion by a significant amount.

Re: Agentic Engineering Patterns

#154

Post by ljlolel »

storus wrote: post #126

These lessons get obliterated with every new LLM generation. Like how LangChain started on stupid models with small context, creating some crazy architecture around it to bypass their limitations that got completely obliterated when GPT-3.5 was released, yet people still use it and overcomplicate things. Rather look at where the puck is going, we might soon not need more than a single agent to do everything given con…

It’ll all be a ClaudeVM. No code. https://jperla.com/blog/claude-electron-not-claudevm

Re: Agentic Engineering Patterns

#155

Post by ljlolel »

pkorzeniewski wrote: post #52
benrutter wrote: post #10

I use AI in my workflow mostly for simple boilerplate, or to troubleshoot issues/docs. I've dipped into agentic work now and again, but never been very impressed with the output (well, that there is any functioning output is insanely impressive, but it isn't code I want to be on the hook for complaining). I hear a lot of people saying the same, but similarly a bunch of people I respect saying they barely write code a…

One thing I rarely see mentioned is that often creating code by hand is simply faster (at least for me) than using AI. Creating a plan for AI, waiting for execution, verifying, prompting again etc. can take more time than just doing it on my own with a plan in my head (and maybe some notes). Creating something from scratch or doing advanced refactoring is almost always faster with AI, but most of my daily tasks are b…

That’s why we won’t plan anymore or compile it’ll just execute https://jperla.com/blog/claude-electron-not-claudevm

Re: Agentic Engineering Patterns

#156

Post by echelon »

63stack wrote: post #140
lbreakjai wrote: post #109

We're going to do it again, aren't we? We're going to take something simple and sensible ("write tests first", "small composable modules", etc.), give it a fancy complicated name ("Behavior-Constrained Implementation Lifecycle pattern", "Boundary-Scoped Processing Constructs pattern", etc.), and create an entire industry of consultants and experts selling books and enterprise coaching around it, each swearing they ha…

People are rushing to be the first one to coin something and hit it big. Imagine the amount of $$$ you could get for being an "expert ai consultant" in this space. There was already another attempt at agentic patterns earlier: https://agentic-patterns.com/ Absolute hot air garbage.

It's not hot air garbage.

Secondly: this is a temporary vacuum state. We're only needed to bridge the gap.

I wouldn't be trying to be a consultant, I would be scurrying to ensure we have access to these tools once they're industrial. A "$5M button" to create any business function won't be within the reach of labor capital, but it will be for financial capital. That's the world we're headed to.

Re: Agentic Engineering Patterns

#157

Post by simonw »

Thews wrote: post #153

There was a mention of using agents to build projects into WASM. I've had the best luck telling it to use zig to compile to webassembly. It shortens the time to completion by a significant amount.

That's a great tip, thanks! I did not know Zig could do this.

You can "pip install ziglang" and get the right version for different platforms too.

Re: Agentic Engineering Patterns

#158

Post by jihadjihad »

I wish there was a little more color in the Testing and QA section. While I agree with this:

  > A comprehensive test suite is by far the most effective way to keep those features working.
there is no mention at all about LLMs' tendency to write tautological tests--tests that pass because they are defined to pass. Or, tests that are not at all relevant or useful, and are ultimately noise in the codebase wasting cycles on every CI run. Sometimes to pass the tests the model might even hardcode a value in a unit test itself!

IMO this section is a great place to show how we as humans can guide the LLM toward a rigorous test suite, rather than one that has a lot of "coverage" but doesn't actually provide sound guarantees about behavior.

Re: Agentic Engineering Patterns

#159

Post by Rohunyyy »

joelthelion wrote: post #123
flir wrote: post #117

Earlier quoted context omitted.

Has anyone staked a claim to "Agile AI" yet?

I've seen several already. There's a huge business opportunity (at our expense, of course).

At this point what is happening that is not at our expense? Hell if I could be a grifter and start another .ai company honestly I would. I guess I am just not that talented.
Post reply on HN