Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

12-factor Agents: Patterns of reliable LLM applications

github.com

Post reply on HN
Print view View original

31–40 of 84 posts

Re: 12-factor Agents: Patterns of reliable LLM applications

#31

Post by pancsta »

dhorthy wrote: post #5
pancsta wrote: post #4

Very informative wiki, thank you, I will definitely use it. So Ive made my own "AI Agents framework" [0] based on actor model, state machines and aspect oriented programming (released just yesterday, no HN post yet) and I really like points 5 and 7: 5: Unify execution state and business state 8. Own your control flow That is exactly what SecAI does, as it's a graph control flow library at it's core (multigraph instea…

i like the terminal UI and otel integrations - what tasks are you using this for today?

Thanks, terminal UI is an important design choice - it's fast, cheap, and runs everywhere (like the web via wasm / ssh, or on iphones with touch). The LLM layer is still fresh, and I personally use it for web scraping, but the underlying workflow engine is quite mature and ubiquitous - it was used for sync engines, UIs, daemons, network services. It shines when faces complexity, nondeterminism, and retry logic - the more chaotic the flow is, the bigger the gains.

The approach is to shape behavior from chaos by exclusion, instead of defining all possible transitions. With LLMs, this process could be automated and effectively an agent would be dynamically creating itself using a DSL (state schema and predefined states). The great thing about LLMs is being charged by tokens instead of a number of requests. We can just interrogate them about every detail separately and build a flow graph with transparent (and debuggable) reasoning. I also have API sketches for proactive scenarios (originally made for an ML prototype) [0].

[0] https://github.com/pancsta/secai/blob/474433796c5ffbc7ec5744...

Re: 12-factor Agents: Patterns of reliable LLM applications

#32

Post by dhorthy »

daxfohl wrote: post #30
dhorthy wrote: post #28

Earlier quoted context omitted.

oh heck yeah this rocks. I'm gonna add to the links section

Additionally in terms of career development, you're going to be a lot better off learning the low level LLM interfaces rather than being dependent on a framework (or their even more evil cousin, platforms). Once you learn those, jumping to a platform is usually trivial, whereas the reverse can be more challenging. Junior devs often think that the more frameworks they have on their resume the better, but it often pige…

exactly - we keep trying to figure out the right interfaces, but we jump to assume that we know what they are.

If we had the right interface, we would set up the black box, and then put holes/knobs on the box to allow anyone to change the things they should actually need to change.

if we have the wrong interface, then the knobs aren't interesting, and instead we keep end up opening the box, or reaching into the holes at weird angles to do things that nobody knew we'd want to, but that are obviously the right things to do to maximize performance

someday we'll have the right interface, but for now, better to skip the box and do the extra cycles. You're an engineer, you can write a for loop and a switch statement. don't outsource your prompts and give up control flow to save a few hundred lines that will eventually become pretty customized anyway

Re: 12-factor Agents: Patterns of reliable LLM applications

#34

Post by dhorthy »

glial wrote: post #33

This is great -- and I have learned 80% the hard way. The other 20% will be valuable reading! Personally I've had success with LangGraph + pydantic schemas. Curious to know what others have found useful.

funny you say

> I have learned 80% the hard way

because the other working title for this was "Agents the Hard Way" (in the spirit of https://github.com/kelseyhightower/kubernetes-the-hard-way)

Re: 12-factor Agents: Patterns of reliable LLM applications

#35

Post by daxfohl »

DebtDeflation wrote: post #6

> most "AI Agents" that make it to production aren't actually that agentic. The best ones are mostly just well-engineered software with LLMs sprinkled in at key points I've been saying that forever, and I think that anyone who actually implements AI in an enterprise context has come to the same conclusion. Using the Anthropic vernacular, AI "workflows" are the solution 90% of the time and AI "agents" maybe 10%. But e…

I think it got started as AI tools for things like cancer detection based purely on deep learning started to outperform tools where humans guide the models what to look for. The expectation became that eventually this will happen for LLM agents too if only we can add more horsepower. But it seems like we've hit a bit of a ceiling there. The latest releases from OpenAI and Meta were largely duds despite their size, still very far from anything you'd trust for anything important, and there's nothing left to add to their training corpus that isn't already there.

Of course a new breakthrough could happen any day and get through that ceiling. Or "common sense" may be something that's out of reach for a machine without life experience. Until that shakes out, I'd be reluctant to make any big bets on any AI-for-everything solutions.

Re: 12-factor Agents: Patterns of reliable LLM applications

#38

Post by saadatq »

daxfohl wrote: post #24

This old obscure blog post about framework patterns has resonated with me throughout my career and I think it applies here too. LLMs are best used as "libraries" rather than "frameworks", for all the reasons described in the article and more, especially now while everything is in such flux. "Frameworks" are sexier and easier to sell though, and lead to lock-in and add-on services, so that's what gets promoted. https:…

This is so good…

“… you can find frameworks not just in software, but also in ordinary life. If you buy package holidays, you're buying a framework - they transport you to some place, put you in a hotel, feed you and your activities have to fit into the shape provided by the framework (say, go into the pool and swim there). If you travel independently, you are composing libraries. You have to book your flights, find your accommodation and arrange your program (all using different libraries). It is more work, but you are in control - and you can arrange things exactly the way you need.”

Re: 12-factor Agents: Patterns of reliable LLM applications

#40

Post by mikedelfino »

dhorthy wrote: post #18
deadbabe wrote: post #8

With all this AI-agent bullshit out there these days, the most useful AI-agent I still use in daily life is the humble floor vacuum/mopping robot.

They kept telling me automation would do my chores so we could spend more time on writing and art. I write less and still have to do my own laundry

The irony is that much of the writing and art have indeed been automated.
Post reply on HN