Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building AI Products–Part I: Back-End Architecture

philcalcado.com

Post reply on HN
Print view View original

11–20 of 25 posts

Re: Building AI Products–Part I: Back-End Architecture

#11

Post by bsenftner »

xrd wrote: post #4

This article is written by an engineer, first and foremost. Many of the APIs or LLM extensions provided by AI companies are written by ML engineers that do not have Phil's decades of experience in distributed systems, databases and networking. That is evident after reading this article; the first time I've seen a coherent discussion of the tools and tradeoffs when building agentic systems. I've struggled to actually…

Yeah, it's really clear it's an engineer, what the software does is never even mentioned, the purposes and tasks that it performs: what are they? And how is hallucination managed? This reads to me like a complexity soup, where they just started without a clear idea of purpose or goal. Perhaps if the article mentioned what the software does, the purpose, it might be more clear. It sounds like a replacement for the entire management layer of a company...

Re: Building AI Products–Part I: Back-End Architecture

#16

Post by theptip »

Great breakdown of the "architectural decision log" for the evolution of this system.

> This model broke down when we added backpressure and resilience patterns to our agents. We faced new challenges: what happens when the third of five LLM calls fails during an agent’s decision process? Should we retry everything? Save partial results and retry just the failed call? When do we give up and error out?”

> We first looked at ETL tools like Apache Airflow. While great for data engineering, Airflow’s focus on stateless, scheduled tasks wasn’t a good fit for our agents’ stateful, event-driven operations.

> I’d heard great things about Temporal from my previous teams at DigitalOcean. It’s built for long-running, stateful workflows, offering the durability and resilience we needed out of the box.

I would also have reached for workflow engines here. But I wonder if Actor frameworks might actually be the sweet spot; something like Erlang's distributed actor model could be a good fit. I'm not familiar with a good distributed Actor framework for Python but there's of course Elixir, Actix, Akka in other stacks.

Coming from the other direction, I'm not surprised that Airflow isn't fit for this purpose, but I wonder if one of the newer generation of ETL engines like Dagster would work? Maybe the workflow here just involves too many pipelines (one per customer per Agent, I suppose), and too many Sensor events (each Slack message would get materialized, not sure if that's excessive). Could be a fairly substantial overhaul to the architecture vs. Temporal, but I'd be interested to know if anyone has experimented with this option for AI workflows.

Re: Building AI Products–Part I: Back-End Architecture

#18

Post by pixelsort »

bsenftner wrote: post #11
xrd wrote: post #4

This article is written by an engineer, first and foremost. Many of the APIs or LLM extensions provided by AI companies are written by ML engineers that do not have Phil's decades of experience in distributed systems, databases and networking. That is evident after reading this article; the first time I've seen a coherent discussion of the tools and tradeoffs when building agentic systems. I've struggled to actually…

Yeah, it's really clear it's an engineer, what the software does is never even mentioned, the purposes and tasks that it performs: what are they? And how is hallucination managed? This reads to me like a complexity soup, where they just started without a clear idea of purpose or goal. Perhaps if the article mentioned what the software does, the purpose, it might be more clear. It sounds like a replacement for the ent…

Manager need to replace engineers with AI faster than engineers use AI to replace their managers. In the end, nobody wins except OpenAI.

Re: Building AI Products–Part I: Back-End Architecture

#19

Post by NeutralCrane »

bsenftner wrote: post #11
xrd wrote: post #4

This article is written by an engineer, first and foremost. Many of the APIs or LLM extensions provided by AI companies are written by ML engineers that do not have Phil's decades of experience in distributed systems, databases and networking. That is evident after reading this article; the first time I've seen a coherent discussion of the tools and tradeoffs when building agentic systems. I've struggled to actually…

Yeah, it's really clear it's an engineer, what the software does is never even mentioned, the purposes and tasks that it performs: what are they? And how is hallucination managed? This reads to me like a complexity soup, where they just started without a clear idea of purpose or goal. Perhaps if the article mentioned what the software does, the purpose, it might be more clear. It sounds like a replacement for the ent…

Agreed. This could be a very intelligent implementation, or it could be an over-engineered mess. It certainly seems like overkill for my experiences with agents, but problem applications can vary wildly. It is impossible to tell how to evaluate these design choices without more concrete details.

Re: Building AI Products–Part I: Back-End Architecture

#20

Post by pcalcado »

xrd wrote: post #4

This article is written by an engineer, first and foremost. Many of the APIs or LLM extensions provided by AI companies are written by ML engineers that do not have Phil's decades of experience in distributed systems, databases and networking. That is evident after reading this article; the first time I've seen a coherent discussion of the tools and tradeoffs when building agentic systems. I've struggled to actually…

Thanks! Honestly, I feel like my first year was a lot of just translating what those papers were trying to say—especially because often they talk a lot but don't say much. I am lucky that my cofounder has a background in ML/AI and could help me understand, but something else that helped me was to ask Claude/GPT to explain something I don't understand: "using analogies an experience back end developer understands".
Post reply on HN