> reliable LLM applications
add that to the list of contradictory phrases (jumbo shrimp, etc.)
12-factor Agents: Patterns of reliable LLM applications
41–50 of 84 posts
Re: 12-factor Agents: Patterns of reliable LLM applications
#42This old obscure blog post about framework patterns has resonated with me throughout my career and I think it applies here too. LLMs are best used as "libraries" rather than "frameworks", for all the reasons described in the article and more, especially now while everything is in such flux. "Frameworks" are sexier and easier to sell though, and lead to lock-in and add-on services, so that's what gets promoted. https:…
This is so good… “… you can find frameworks not just in software, but also in ordinary life. If you buy package holidays, you're buying a framework - they transport you to some place, put you in a hotel, feed you and your activities have to fit into the shape provided by the framework (say, go into the pool and swim there). If you travel independently, you are composing libraries. You have to book your flights, find…
My favorite blog post / presentation is Sandi Metz "The Wrong Abstraction", but this one is up there. Definitely punches above its weight for a small obscure post.
Re: 12-factor Agents: Patterns of reliable LLM applications
#43Re: 12-factor Agents: Patterns of reliable LLM applications
#44Earlier quoted context omitted.
This is so good… “… you can find frameworks not just in software, but also in ordinary life. If you buy package holidays, you're buying a framework - they transport you to some place, put you in a hotel, feed you and your activities have to fit into the shape provided by the framework (say, go into the pool and swim there). If you travel independently, you are composing libraries. You have to book your flights, find…
My favorite blog post / presentation is Sandi Metz "The Wrong Abstraction", but this one is up there. Definitely punches above its weight for a small obscure post.
Yeah duplication way better than the wrong abstraction.
Just write the dang switch statement
Re: 12-factor Agents: Patterns of reliable LLM applications
#45These are great. I had my own list of takeaways [0] after doing this for a couple years, though I wouldn't go so far as calling mine factors.
Like you, biggest one I didn't include but would now is to own the lowest level planning loop. It's fine to have some dynamic planning, but you should own an OODA loop (observe, orient, decide, act) and have heuristics for determining if you're converging on a solution (e.g. scoring), or else breaking out (e.g. max loops).
I would also potentially bake in a workflow engine. Then, have your model build a workflow specification that runs on that engine (where workflow steps may call back to the model) instead of trying to keep an implicit workflow valid/progressing through multiple turns in the model.
Re: 12-factor Agents: Patterns of reliable LLM applications
#46Earlier quoted context omitted.
My favorite blog post / presentation is Sandi Metz "The Wrong Abstraction", but this one is up there. Definitely punches above its weight for a small obscure post.
Yeah duplication way better than the wrong abstraction. Just write the dang switch statement
> switch
Hmm, that hit a bit of a nerve. My experience with switch blocks is it can be a gateway drug for teams A, B, C to add their special-case code to team D's repo within a `switch(calling_service)` block. My read of the presentation is more, factor your stuff so that any "switch" is a higher level concern that consumers can do in their own services. Then if you start to see all your consumers write very similar consumption logic, then start thinking about how to pull that down into the library/service itself.
But beyond that trigger nerve, agreed.
Re: 12-factor Agents: Patterns of reliable LLM applications
#47Earlier quoted context omitted.
Yeah duplication way better than the wrong abstraction. Just write the dang switch statement
> switch Hmm, that hit a bit of a nerve. My experience with switch blocks is it can be a gateway drug for teams A, B, C to add their special-case code to team D's repo within a `switch(calling_service)` block. My read of the presentation is more, factor your stuff so that any "switch" is a higher level concern that consumers can do in their own services. Then if you start to see all your consumers write very similar…
fair enough. I think switch statement is a broader category for "basic programming primitives that you should just do yourself" -
agree big switch statements can be an anti-pattern, e.g. when an interface is clearly better suited
Re: 12-factor Agents: Patterns of reliable LLM applications
#48I totally agree with this. Most, if not all, frameworks or building agents are a waste of time
Re: 12-factor Agents: Patterns of reliable LLM applications
#49[dead]
Re: 12-factor Agents: Patterns of reliable LLM applications
#50I totally agree with this. Most, if not all, frameworks or building agents are a waste of time
this guy gets it