11–20 of 20 posts
[flagged]
Hey! Ben here (one of the engineers who built this). This is a reason why we made our http framework (@outputai/http) a first class citizen for the greater framework and our claude code plugins. As you pointed out at this moment in time theres a Cambrian explosion both in new tools/libraries and the willingness to use them, which poses a systemic security threat when combined with how LLMs function. So while you're f…
Hey HN! I'm Daniel, cofounder of GrowthX and Ben's colleague (who posted it). We have about 20 engineers building AI agents and workflows for companies like Lovable, Webflow, Airbyte. Output is the framework we extracted from that work. It runs our AI infrastructure and we open-sourced it. We kept hitting the same problems: writing and iterating on prompts at scale, orchestrating API calls that fail unpredictably, tr…
Hey HN! I'm Daniel, cofounder of GrowthX and Ben's colleague (who posted it). We have about 20 engineers building AI agents and workflows for companies like Lovable, Webflow, Airbyte. Output is the framework we extracted from that work. It runs our AI infrastructure and we open-sourced it. We kept hitting the same problems: writing and iterating on prompts at scale, orchestrating API calls that fail unpredictably, tr…
This looks really interesting - appreciate you sharing. Is it only API key driven or is there a way to try out with a Claude/Anthropic subscription?
So the API keys during setup are entirely optional. They're used in the example workflow that evaluates blog posts for clarity and provides feedback on how to improve.
Youre more than free to ignore/delete the example workflow and create your own that doesn't make use of an LLM 1. Fetching trending hn posts 2. Pulling reddit posts that match keywords 3. Transforming Daily calendar events into an html page etc..
And the claude code plugins (that are installed for you) all work with you Anthropic subscription no problem
Interesting that this came out of 500 agents in production. The hardest part I've seen with agent tool calls is handling partial failures gracefully — the tool returns something but it's incomplete or stale. Do you bake retry/fallback logic into the framework itself or leave that to individual tool implementations?
So we had a few goals here
1. Be opinionated on best practises, tools and libraries
2. Not get in the way of what the developer wants to do
To that end the core is built on top of Temporal, and our llm package is a thin wrapper around ai-sdk that provides QoL enhancements (Prompt files, tracing, cost tracking etc..)
So for failures in general, and tool calling specifically there are two levels of retries.
1. ai-sdk level tool retries: The library by default handles tool call failures and will retry if the LLM deems it a transient issue, and will never hard fail if one of its tool calls in unsuccessful (unless perhaps you instruct it to).
2. Temporal level activity failures: Our workflows and steps are all configured with a base line affordance to reattempt steps that have failed. You as the developer are able to change this, you can make it so a step is never retried, or retried say 100 times with exponential backoff.
Hope that helps!
Curious what the three failure modes were that caused the most incidents before you extracted the framework, those tend to reveal the assumptions baked into the design.