> reliable LLM applications add that to the list of contradictory phrases (jumbo shrimp, etc.)
it can be done! I believe!
12-factor Agents: Patterns of reliable LLM applications
51–60 of 84 posts
Re: 12-factor Agents: Patterns of reliable LLM applications
#52Really curious and excited to know the experience you faces at Heroku that led to the formulation of these 12 principles
Re: 12-factor Agents: Patterns of reliable LLM applications
#53> reliable LLM applications add that to the list of contradictory phrases (jumbo shrimp, etc.)
Can you successfully transfer data over unreliable connections? LLM is just a misbehaving DB, once you pin it down the right way and lower your expectations, then "reliable LLM applications" are definitely possible. But if we go yolo with regexp-like-intelligence, then...
Re: 12-factor Agents: Patterns of reliable LLM applications
#54really cool to see BAML on here :) 100% align on so much of what you've said here. its really about treating LLMs as functions.
Re: 12-factor Agents: Patterns of reliable LLM applications
#55I am wondering how libraries like DSPY [0] fits in your factor-2 [1]
As I was reading, I saw mention of BAML > (the above example uses BAML to generate the prompt ...
Personally, in my experience hand-writing prompts for extracting structured information from unstructured data has never been easy. With DSPY, my experience has been quite good so far.
As you have used raw prompt from BAML, what do you think of using the raw prompts from DSPY [2]?
[0] https://dspy.ai/
[1] https://github.com/humanlayer/12-factor-agents/blob/main/con...
[2] https://dspy.ai/tutorials/observability/#using-inspect_histo...
Re: 12-factor Agents: Patterns of reliable LLM applications
#56Very informative wiki, thank you, I will definitely use it. So Ive made my own "AI Agents framework" [0] based on actor model, state machines and aspect oriented programming (released just yesterday, no HN post yet) and I really like points 5 and 7: 5: Unify execution state and business state 8. Own your control flow That is exactly what SecAI does, as it's a graph control flow library at it's core (multigraph instea…
This is great, thank you so much for sharing!
Re: 12-factor Agents: Patterns of reliable LLM applications
#57I believe the principles would be easier to follow if there is a consistent narrative through the factors, why which I mean using potentially real-world example for such a system.
This is a great bit of feedback - what kinda of use cases do you think would make sense? Definitely wanna evolve this in the open with the community
May be if you pick a real-world agent workflow (toy from your production experience, trim it down), and showcase how all these factors will come along in a project.
I am inspired by the simplicity of these 12 factors and definitely want to learn more with an example that embraces these factors.
Re: 12-factor Agents: Patterns of reliable LLM applications
#58This could not have come at a better time for me, thank you!
I've been tinkering with an idea for an audiovisual sandbox[1] (like vvvv[2] but much simpler of course, barebones).
Idea is to have a way to insert LM (or some simple locally run neural net) "nodes" which are given specific tasks and whose output is expected to be very constrained. Hence your example:
"question -> answer: float"
[1]: loose set of bulletpoints brainstorming the idea if curious, not organised: https://kfs.mkj.lt/#audiovisllm (click to expand description)
[2]: https://vvvv.org/
Re: 12-factor Agents: Patterns of reliable LLM applications
#59Very informative wiki, thank you, I will definitely use it. So Ive made my own "AI Agents framework" [0] based on actor model, state machines and aspect oriented programming (released just yesterday, no HN post yet) and I really like points 5 and 7: 5: Unify execution state and business state 8. Own your control flow That is exactly what SecAI does, as it's a graph control flow library at it's core (multigraph instea…
"Another thing often missed by other frameworks are dedicated devtools"
From my experience, PydanticAI really nailed it with Logfire—debugging[0] agents was significantly easier and more effective compared to the other frameworks and libraries I tested.
Re: 12-factor Agents: Patterns of reliable LLM applications
#60I've noticed some of these factors myself as well. I'd love to build more AI applications like this. Currently I'm a data analyst and they don't fully appreciate that I can build stuff like this as it is not a technology oriented company.
I'd love to work on stuff like this full-time. If anyone is interested in a chat, my email is on my profile (US/EU).