Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apache Burr: Build reliable AI agents and applications

burr.apache.org

Post reply on HN
Print view View original

51–60 of 125 posts

Re: Apache Burr: Build reliable AI agents and applications

#51

Post by mbreese »

brotchie wrote: post #6

I'm still on the fence about agent frameworks, they have their place, and it depends on the nature of the agent: e.g. "Low latency, return a good enough response in 3 seconds, vs. working for 3 hours on a problem." BUT, if you boil it down, an agent really is context building, making an LLM call, executing requested tool calls, parsing the final model output, returning it to some frontend. There's extensions like mem…

I'm somewhat in agreement. I like building 1:1 code for that specific agent.

Where I'm starting to question this is maintainability. When I come up with a new technique or way of doing something in my new agent, how can I update an older agent. Do I want to update the older agent?

But, I get what you're talking about w.r.t. building for the exact problem at hand. For example, I'm guessing that Apache Burr has support for a plugin-able vector RAG system (or at least it will if it doesn't now). That's great, but I want my RAG system to add documents to the context and keep them as part of an updated system prompt with some very specific tweaks that happen as part of that process. This is a bespoke way of working with an existing concept (RAG) that doesn't lend itself to using any specific framework.

In my use-case, bespoke is the way to go. But then I'm still stuck with having to make engineering choices for updating older agents. So, I see your point.

Re: Apache Burr: Build reliable AI agents and applications

#52

Post by krawczstef »

brotchie wrote: post #6

I'm still on the fence about agent frameworks, they have their place, and it depends on the nature of the agent: e.g. "Low latency, return a good enough response in 3 seconds, vs. working for 3 hours on a problem." BUT, if you boil it down, an agent really is context building, making an LLM call, executing requested tool calls, parsing the final model output, returning it to some frontend. There's extensions like mem…

yep - here's something cool an end user wrote with Burr - https://github.com/msradam/phoebe

Burr just helps you, the engineer, to really control the primitives. Then adds some cool features you don't have to think about -- like observability :)

Re: Apache Burr: Build reliable AI agents and applications

#53

Post by peterbell_nyc »

brotchie wrote: post #6

I'm still on the fence about agent frameworks, they have their place, and it depends on the nature of the agent: e.g. "Low latency, return a good enough response in 3 seconds, vs. working for 3 hours on a problem." BUT, if you boil it down, an agent really is context building, making an LLM call, executing requested tool calls, parsing the final model output, returning it to some frontend. There's extensions like mem…

For me the heart of an agentic system is NOT using agents (except when you really have to). Components of a working system include: - Pipelines/recipes to describe multi-step flows (deterministic, agentic and HiTL steps), loops, conditionals, exit-on's for max loop iteractions, etc - The logistics to actually run the model and HiTL steps reliably across multiple agent worker pools - Management and delivery (and security/governance and permissioning) of thick skills with code to do as much as possible - Context management so the right agents have the right context for the right sessions at the right time - Project management - ability to store and access tickets, dependencies, track progress, restart stuck ticket claims, etc - Transcript saving, memory features and dreaming/compounding capabilities so the agents continue to learn from each session - o11y for understanding whats happening, tracking costs and usages, etc - Evals and auto-tuning of prompts so you can go cross model provider and also lock to a model version so you can do an ROI on each model version upgrade - Sandboxes for running the actual model sessions

Don't need to get it all from one vendor, but that feels to me like the toolkit and for most use cases I'd argue: - Don't limit yourself to a single model provider (anthropic, openai, etc) - Own your context - Own your compounding

Re: Apache Burr: Build reliable AI agents and applications

#54

Post by peterbell_nyc »

tomrod wrote: post #38
throw1234567891 wrote: post #29

Earlier quoted context omitted.

Have a look at pi.

Whst about pi do you like?

Pi is a nice multi-agent wrapper. I use it to wrap my OpenAI max plan calls and my API calls. It takes care of some of the agent plumbing - still need sandbox, orchestrator, compounding, context, evals, etc but it's a nice component.

Re: Apache Burr: Build reliable AI agents and applications

#55

Post by peterbell_nyc »

brotchie wrote: post #28
vanuatu wrote: post #23

Earlier quoted context omitted.

my job rn is just building agents the hard part about building agents isnt the framework it's discovery, context, traditional engineering, handling the last mile there are some invariants like the loop, tools, observability, guardrails, monitors etc...

100% agreed, the "this is what an agent looks like to write" is the wrong pitch for a new agent framework. The better pitch would be, "this is how easy observability, guardrails, monitoring, deployment, evals, versioning, A/B testing are with our framework." What the agent code looks like is somewhat incidental.

This this this!

Anyone have something they genuinely like for all of this? For now I'm rolling my own, but I can't believe I won't find a better OSS alternative soon...

Re: Apache Burr: Build reliable AI agents and applications

#56

Post by elijahbenizzy »

tcdent wrote: post #13

A builder pattern and decorators. Yes, Python has decorators, but they're best used as "filters" that apply to functions or methods. Cache this, serialize the output of this function always, prepare this function to be used as a tool by an agentic harness. Not registration, not flow control. You may disagree but someone has to say it; FastAPI influenced the modern use of decorators far too much in the wrong direction…

So decorators here specifically attach metadata to make a function a reusable component. Builder makes a workflow. In Hamilton it's all decorators because it's purely declarative construction (sans reusability, really).

Re: Apache Burr: Build reliable AI agents and applications

#57

Post by peterbell_nyc »

trollbridge wrote: post #19
brotchie wrote: post #6

I'm still on the fence about agent frameworks, they have their place, and it depends on the nature of the agent: e.g. "Low latency, return a good enough response in 3 seconds, vs. working for 3 hours on a problem." BUT, if you boil it down, an agent really is context building, making an LLM call, executing requested tool calls, parsing the final model output, returning it to some frontend. There's extensions like mem…

It’s painfully obvious that you can just open your coding harness and… tell it you’d like to make an agent. They’re simple to write.

The agent isn't the hard part - it's the orchestration, skills, research systems, adversarial reviews, dreaming/compounding, context management and all the rest. Plus all the annoying hygiene tools to "poke the agent that got a clear prompt and decided to just sit there and wait for no good reason" and "delete the remote branches that the prompt told all the agents to delete but some of them forgot to":)

Re: Apache Burr: Build reliable AI agents and applications

#58

Post by aplomb1026 »

brotchie wrote: post #6

I'm still on the fence about agent frameworks, they have their place, and it depends on the nature of the agent: e.g. "Low latency, return a good enough response in 3 seconds, vs. working for 3 hours on a problem." BUT, if you boil it down, an agent really is context building, making an LLM call, executing requested tool calls, parsing the final model output, returning it to some frontend. There's extensions like mem…

[flagged]

Re: Apache Burr: Build reliable AI agents and applications

#59

Post by agentifysh »

brotchie wrote: post #6

I'm still on the fence about agent frameworks, they have their place, and it depends on the nature of the agent: e.g. "Low latency, return a good enough response in 3 seconds, vs. working for 3 hours on a problem." BUT, if you boil it down, an agent really is context building, making an LLM call, executing requested tool calls, parsing the final model output, returning it to some frontend. There's extensions like mem…

100% this

you dont need a framework

Re: Apache Burr: Build reliable AI agents and applications

#60

Post by agentdev001 »

peterbell_nyc wrote: post #55
brotchie wrote: post #28

Earlier quoted context omitted.

100% agreed, the "this is what an agent looks like to write" is the wrong pitch for a new agent framework. The better pitch would be, "this is how easy observability, guardrails, monitoring, deployment, evals, versioning, A/B testing are with our framework." What the agent code looks like is somewhat incidental.

This this this! Anyone have something they genuinely like for all of this? For now I'm rolling my own, but I can't believe I won't find a better OSS alternative soon...

Nvidia Openshell solves most of the hard problems I've run into while building stuff in this space.

Observability is, for my purposes, solved by a given framework supporting OpenTelemetry.

Guardrails is where I've gotten the most value of openshell being a neat package. Agent workload scope is written as policy in openshell, and capability is backed by openshell handling all execution.

Monitoring/deployment/versioning is helped as well, depending on how agents/runners are slotted into the system. Deployment namely is quite well supported- openshell has kube/helm bits that are experimental atm, but seem like a logical approach imho.

Evals and a/b testing isnt something ive explored in depth, considering that agents with composable tool sets + frontier models are beyond my expectations already.

Post reply on HN