Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

12-factor Agents: Patterns of reliable LLM applications

github.com

Post reply on HN
Print view View original

71–80 of 84 posts

Re: 12-factor Agents: Patterns of reliable LLM applications

#71

Post by chrismdp »

dhorthy wrote: post #70
chrismdp wrote: post #68

Earlier quoted context omitted.

It's always true that you need to drop down a level of abstraction in order to extract the ultimate performance. (eg I wrote a decent-sized game + engine entirely in C about 10 years ago and played with SIMD vectors to optimise the render loop) However, I think the vast majority of use cases will not require this level of control, and we will abandon prompts once the tools improve. Langchain and DSPY are also not the…

i'd be interested to check it out here's a take, I adapted this from someone on the notebookLM team on swyx's podcast > the only way to build really impressive experiences in AI, is to find something right at the edge of the model's capability, and to get it right consistently. So in order to build something very good / better than the rest, you will always benefit from being able to bring in every optimization you c…

I think the building blocks of the most impressive experiences will come from choosing the exact right point to involve an LLM, the orchestration of the component pieces, and the user experience.

That's certainly what I found in games. The games which felt magic to play were never the ones with the best hand rolled engine.

The tools aren't there yet to ignore prompts, and you'll always need to drop down to raw prompting sometimes. I'm looking forward to a future where wrangling prompts is only needed for 1% of my system.

Re: 12-factor Agents: Patterns of reliable LLM applications

#72

Post by dhorthy »

chrismdp wrote: post #71
dhorthy wrote: post #70

Earlier quoted context omitted.

i'd be interested to check it out here's a take, I adapted this from someone on the notebookLM team on swyx's podcast > the only way to build really impressive experiences in AI, is to find something right at the edge of the model's capability, and to get it right consistently. So in order to build something very good / better than the rest, you will always benefit from being able to bring in every optimization you c…

I think the building blocks of the most impressive experiences will come from choosing the exact right point to involve an LLM, the orchestration of the component pieces, and the user experience. That's certainly what I found in games. The games which felt magic to play were never the ones with the best hand rolled engine. The tools aren't there yet to ignore prompts, and you'll always need to drop down to raw prompt…

yeah. the issue is when you're baked into a tool stack/framework where you cant go customize in that 1% of cases. A lot of tools try to get the right abstractions where you can "customize everything you would want to" but they miss the mark in some cases

Re: 12-factor Agents: Patterns of reliable LLM applications

#73

Post by chrismdp »

dhorthy wrote: post #72
chrismdp wrote: post #71

Earlier quoted context omitted.

I think the building blocks of the most impressive experiences will come from choosing the exact right point to involve an LLM, the orchestration of the component pieces, and the user experience. That's certainly what I found in games. The games which felt magic to play were never the ones with the best hand rolled engine. The tools aren't there yet to ignore prompts, and you'll always need to drop down to raw prompt…

yeah. the issue is when you're baked into a tool stack/framework where you cant go customize in that 1% of cases. A lot of tools try to get the right abstractions where you can "customize everything you would want to" but they miss the mark in some cases

100%. You can't and shouldn't wrap every interaction. We need a new approach.

Re: 12-factor Agents: Patterns of reliable LLM applications

#74

Post by hhimanshu »

dhorthy wrote: post #61
hhimanshu wrote: post #57

Earlier quoted context omitted.

May be if you pick a real-world agent workflow (toy from your production experience, trim it down), and showcase how all these factors will come along in a project. I am inspired by the simplicity of these 12 factors and definitely want to learn more with an example that embraces these factors.

I link in a few places to https://github.com/got-agents/agents where I have a few of these real agents

Thank you, I will take a look

Re: 12-factor Agents: Patterns of reliable LLM applications

#75

Post by darepublic »

I didn't really read this extensively but to me I would want to use as much deterministic code as possible and leverage the llm as little as possible. That to me is a better portend of predictable result, lower operational costs and is a signal that nobody could just quickly reproduce the same app. I would tend to roll my own tools and not use out of the box buzz word glue to integrate my llm with other systems. And if these conditions aren't met or aren't necessary I'd figure someone else could just vibe code the same solution in no time anyway. Keep control I say! Die on the hill of control! That's not to say I'm not impressed by LLMs.. quite the opposite

Re: 12-factor Agents: Patterns of reliable LLM applications

#76

Post by dhorthy »

darepublic wrote: post #75

I didn't really read this extensively but to me I would want to use as much deterministic code as possible and leverage the llm as little as possible. That to me is a better portend of predictable result, lower operational costs and is a signal that nobody could just quickly reproduce the same app. I would tend to roll my own tools and not use out of the box buzz word glue to integrate my llm with other systems. And…

control is good, and determinism is good - while the primary goal is to convince people "don't give up too much control" - there is a secondary which is: THESE are the places where it makes sense to give up some control

Re: 12-factor Agents: Patterns of reliable LLM applications

#78

Post by musicale »

pancsta wrote: post #53
musicale wrote: post #41

> reliable LLM applications add that to the list of contradictory phrases (jumbo shrimp, etc.)

Can you successfully transfer data over unreliable connections? LLM is just a misbehaving DB, once you pin it down the right way and lower your expectations, then "reliable LLM applications" are definitely possible. But if we go yolo with regexp-like-intelligence, then...

> Can you successfully transfer data over unreliable connections?

Validating LLM output is probably not as easy as computing a checksum or CRC.

Re: 12-factor Agents: Patterns of reliable LLM applications

#79

Post by musicale »

daxfohl wrote: post #35
DebtDeflation wrote: post #6

> most "AI Agents" that make it to production aren't actually that agentic. The best ones are mostly just well-engineered software with LLMs sprinkled in at key points I've been saying that forever, and I think that anyone who actually implements AI in an enterprise context has come to the same conclusion. Using the Anthropic vernacular, AI "workflows" are the solution 90% of the time and AI "agents" maybe 10%. But e…

I think it got started as AI tools for things like cancer detection based purely on deep learning started to outperform tools where humans guide the models what to look for. The expectation became that eventually this will happen for LLM agents too if only we can add more horsepower. But it seems like we've hit a bit of a ceiling there. The latest releases from OpenAI and Meta were largely duds despite their size, st…

> Or "common sense" may be something that's out of reach for a machine without life experience

Maybe Doug Lenat's idea of a common sense knowledge base wasn't such a bad one.

Re: 12-factor Agents: Patterns of reliable LLM applications

#80

Post by dhorthy »

musicale wrote: post #78
pancsta wrote: post #53

Earlier quoted context omitted.

Can you successfully transfer data over unreliable connections? LLM is just a misbehaving DB, once you pin it down the right way and lower your expectations, then "reliable LLM applications" are definitely possible. But if we go yolo with regexp-like-intelligence, then...

> Can you successfully transfer data over unreliable connections? Validating LLM output is probably not as easy as computing a checksum or CRC.

*probably :)
Post reply on HN