Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What years of production-grade concurrency teaches us about building AI agents

georgeguimaraes.com

Post reply on HN
Print view View original

41–50 of 53 posts

Re: What years of production-grade concurrency teaches us about building AI agents

#41

Post by znnajdla »

randomtoast wrote: post #8

I’ve built fairly large OTP systems in the past, and I think the core claim is directionally right: long lived, stateful, failure prone "conversations" map very naturally to Erlang processes plus supervision trees. An agent session is basically a call session with worse latency and more nondeterminism. That said, a lot of current agent workloads are I/O bound around external APIs. If 95% of the time is waiting on Ope…

If you are just “gluing together API calls” that’s exactly where Elixir state machines and supervisors make your agentic code so much easier to write. API calls are constantly failing, timing out, retrying, and being pre/post processed, and you need to keep track of the state of your agent across multiple failure modes, recover gracefully, and coordinate between processes. In most other languages like Python or TypeScript that’s a hell of a lot of process orchestration code (Bull/Celery), try/catches, health checks, retry logic, and global state management in a a database of some sort. Compare that to Elixir processes where you get most of that for free because the language was designed exactly for that. It’s just a state machine that can crash and recover gracefully in the right order with all related processes. The BEAM is not just about running millions of processes at scale. It’s also about simplifying how you reason about a single process lifecycle, and long running agent lifecycles can get really complex.

When you are just “gluing together API calls” surrounding tooling doesn’t matter as much. I don’t care that Elixir doesn’t have such a large community as Python, I’m just gluing together API calls, I don’t have dependencies.

Re: What years of production-grade concurrency teaches us about building AI agents

#42

Post by znnajdla »

mccoyb wrote: post #2

Broadly agree with the author's points, except for this one: > TypeScript/Node.js: Better concurrency story thanks to the event loop, but still fundamentally single-threaded. Worker threads exist but they're heavyweight OS threads, not 2KB processes. There's no preemptive scheduling: one CPU-bound operation blocks everything. This cannot be a real protest: 100% of the time spent in agent frameworks is spent ... waiti…

> 100% of the time spent in agent frameworks is spent ... waiting for the agent to respond, or waiting for a tool call to execute. Almost no time is spent in the logic of the framework itself.

But that’s exactly where multi threaded Elixir is better! You want a single thread like Node for CPU bound work, you want extreme multi threading for I/O bound work like AI agents. In Elixir you can do both: heavy CPU work without worrying about stopping the world, and heavy concurrency across millions of threads where work is I/O bound and you want to saturate your network connection. In Node you can’t do either of those things easily - it’s just a single thread.

Re: What years of production-grade concurrency teaches us about building AI agents

#43

Post by znnajdla »

simianwords wrote: post #6

I don’t see the point of agent frameworks. Other than durability and checkpoints how does it help me? Claude code already works as an agent that calls tools when necessary so it’s not clear how an abstraction helps here. I have been really confused by langchain and related tech because they seem so bloated without offering me any advantages? I genuinely would like to know what I’m missing.

Even if you’re just running Claude Code as an external CLI then Elixir makes this easier because it supervises and handles external processes gracefully. Starting, checking, stopping processes needs to be done manually in most languages.

Re: What years of production-grade concurrency teaches us about building AI agents

#44

Post by znnajdla »

quadruple wrote: post #27

> The BEAM's "let it crash" philosophy takes the opposite approach. Instead of anticipating every failure mode, you write the happy path and let processes crash. The supervisor detects the crash and restarts the process in a clean state. The rest of the system continues unaffected. Do I want this? If my request fails because the tool doesn't have a DB connection, I want the model to receive information about that err…

“Let it crash” doesn’t mean keep bashing your head against the wall. Elixir makes it easy to write state machines which reason about different types of failures, but it’s more declarative (this process requires X and Y preconditions, otherwise do Z) rather than imperative (I have to try/catch failures due to X and Y, now do Z). With Elixir you can actually specify that the process doesn’t start until the DB connection is ready, if that was the cause of the failure, it won’t start again (something else can take care of the DB). When the LLM API returns an error you can put the agent in a paused “errored” state and then you can have a different process decide what to do with the error, and pass it back to the main agent when it’s done. This is all really elegant functional code in Elixir compared to try/catches and if statements.

Re: What years of production-grade concurrency teaches us about building AI agents

#45

Post by manojlds »

igravious wrote: post #18
manojlds wrote: post #12

Surely they mean Erlang not Elixir

addressed at the very top of the article A note on terminology: Throughout this post I refer to "the BEAM." BEAM is the virtual machine that runs both Erlang and Elixir code, similar to how the JVM runs both Java and Kotlin. Erlang (1986) created the VM and the concurrency model. Elixir (2012) is a modern language built on top of it with better ergonomics. When I say "BEAM," I mean the runtime and its properties. Whe…

How is that addressing the title

Re: What years of production-grade concurrency teaches us about building AI agents

#46

Post by Reefersleep »

cyberpunk wrote: post #17
christophilus wrote: post #14

Earlier quoted context omitted.

Sounds to me like they mean “BEAM” rather than a specific language. But BEAM means Elixir for most newcomers.

Which is a real shame as if you actually spend some time with both you’ll probably eventually realise erlang is the nicer language. Elixir just feels… Like it’s a load of pre-compile macros. There’s not even a debugger.

Do you have other examples of how it's nicer? I've only ever heard of Elixir being the nicer alternative.

Re: What years of production-grade concurrency teaches us about building AI agents

#47

Post by simianwords »

spoiler wrote: post #9
simianwords wrote: post #6

I don’t see the point of agent frameworks. Other than durability and checkpoints how does it help me? Claude code already works as an agent that calls tools when necessary so it’s not clear how an abstraction helps here. I have been really confused by langchain and related tech because they seem so bloated without offering me any advantages? I genuinely would like to know what I’m missing.

There's lots of things you could do. Imagine you're making a group chat bot (way more difficult than a 1-1 chat) where people can play social games by giving the LLM game rules. You can have an agent that only manages game state using natural language (controlled by the main LLM). You could have an agent dedicated to remembering important conversation, while not paying attention to chit-chatting

What more functionality do you need than the system prompt and list of tools?

Re: What years of production-grade concurrency teaches us about building AI agents

#48

Post by asa400 »

Reefersleep wrote: post #46
cyberpunk wrote: post #17

Earlier quoted context omitted.

Which is a real shame as if you actually spend some time with both you’ll probably eventually realise erlang is the nicer language. Elixir just feels… Like it’s a load of pre-compile macros. There’s not even a debugger.

Do you have other examples of how it's nicer? I've only ever heard of Elixir being the nicer alternative.

This is gonna rankle folks who like one or the other, but they're basically the same language. When it comes to languages that run on the same VM, Erlang and Elixir are very close together. They aren't nearly as far apart as say, Java and Clojure.

Elixir adds a few things (a lisp-style macro system, protocols, UTF-8 as the default string type, a builtin build tool, streams) but Elixir is not a huge departure from Erlang in the way that Clojure is a huge departure from Java.

By far the biggest things you're going to learn when you learn either one are going to be the BEAM runtime itself and the OTP libraries, which both Elixir and Erlang have in common.

Re: What years of production-grade concurrency teaches us about building AI agents

#49

Post by fud101 »

asa400 wrote: post #38
fud101 wrote: post #26

Earlier quoted context omitted.

I'd use something like a lightweight python framework (take your pick) and pair it with htmx. You can run that on low powered hardware or a cheap VPS. I can't even dev elixir on my N100 minipc, it's too demanding. Otherwise Python and SolidJS or Preact will work perfectly for a SPA.

> I can't even dev elixir on my N100 minipc, it's too demanding. Isn't the N100 a quad core machine? It can't run Elixir?

It can but it feels very sluggish developing in my ide of choice (vscode). I also miss having a debugger which is nonnegotiable for me.

Re: What years of production-grade concurrency teaches us about building AI agents

#50

Post by walletdrainer »

Every single post on this blog is AI generated spam, all commenters seem completely oblivious.

Are you guys okay? WTF is going on with HN?

There’s one interesting detail about this blog though, you can see how the LLM-generated spam improves over the years as models get better.

Post reply on HN