Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building your own CLI coding agent with Pydantic-AI

martinfowler.com

Post reply on HN
Print view View original

1–10 of 43 posts

Re: Building your own CLI coding agent with Pydantic-AI

#3

Post by binalpatel »

Pydantic-AI is lovely - I've been working on a forever, fun project to build a coding agent CLI for a year plus now. IMO it does make constructing any given agent very easy, though the lower level APIs are a little painful to use but they seem to be aware of that.

https://github.com/caesarnine/rune-code

Part of the reason I switched to it initially wasn't so much it's niceties versus just being disgusted at how poor the documentation/experience of using LiteLLM was and I thought the folks who make Pydantic would do a better job of the "universal" interface.

Re: Building your own CLI coding agent with Pydantic-AI

#4

Post by bluecoconut »

After maintaining my own agents library for a while, I’ve switched over to pydantic ai recently. I have some minor nits, but overall it's been working great for me. I’ve especially liked combining it with langfuse.

Towards coding agents, I wonder if there are any good / efficient ways to measure how much different implementations work on coding? SWE-bench seems good, but expensive to run. Effectively I’m curious for things like: given tool definition X vs Y (eg. diff vs full file edit), prompt for tool X vs Y (how it’s described, does it use examples), model choice (eg. MCP with Claude, but python-exec inline with GPT-5), sub-agents, todo lists, etc. how much across each ablation, does it matter? And measure not just success, but cost to success too (efficiency).

Overall, it seems like in the phase space of options, everything “kinda works” but I’m very curious if there are any major lifts, big gotchas, etc.

I ask, because it feels like the Claude code cli always does a little bit better, subjectively for me, but I haven’t seen a LLMarena or clear A vs B, comparison or measure.

Re: Building your own CLI coding agent with Pydantic-AI

#5

Post by ziftface »

binalpatel wrote: post #3

Pydantic-AI is lovely - I've been working on a forever, fun project to build a coding agent CLI for a year plus now. IMO it does make constructing any given agent very easy, though the lower level APIs are a little painful to use but they seem to be aware of that. https://github.com/caesarnine/rune-code Part of the reason I switched to it initially wasn't so much it's niceties versus just being disgusted at how poor…

I had the opposite experience. I liked the niceties of Pydantic AI, but had trouble with it that I found difficult to deal with. For example, some of the models wouldn't stream, but the OpenAI models did. It took months to resolve, and well before that I switched to LiteLLM and just hand-rolled the agentic logic stuff. LiteLLM's docs were simple and everything worked as expected. The agentic code is simple enough that I'm not sure what the value-add for some of these libraries is besides adding complexity and the opportunity for more bugs. I'm sure for more complex use cases they can be useful, but for most of the applications I've seen, a simple translation layer like LiteLLM or maybe OpenRouter is more than enough.

Re: Building your own CLI coding agent with Pydantic-AI

#7

Post by drdeafenshmirtz »

iLoveOncall wrote: post #6

I really wish Pydantic invested in... Pydantic, instead of some AI API wrapper garbage. I've been using it a lot lately and anything beyond basic usage is an absolute chore.

how so? Was thinking of using it for my next project - would love to hear some of the risks

Re: Building your own CLI coding agent with Pydantic-AI

#8

Post by siva7 »

These abstractions are nice to not get locked in with one llm provider - but like with langchain - once you use some more niche feature the bugs do shine through. I tried it out with structured output for azure openai but had to give up since somewhere somewhat was broken and it's difficult to figure out if it's the abstraction or the library of the llm provider which the abstraction uses.

Nevertheless i would strongly recommend to not use directly the libraries of the ai providers as you get quickly locked in a extremely fast paced market where today's king can change weekly.

Re: Building your own CLI coding agent with Pydantic-AI

#9

Post by mritchie712 »

we[0] have a pretty complex agent[1] running on Pydantic AI. The team is very responsive to bugs / feature requests. If I had to do it over again, I'd pick Pydantic AI again.

0 - https://www.definite.app/

1 - https://pydantic.dev/articles/building-data-team-with-pydant...

Re: Building your own CLI coding agent with Pydantic-AI

#10

Post by dcreater »

siva7 wrote: post #8

These abstractions are nice to not get locked in with one llm provider - but like with langchain - once you use some more niche feature the bugs do shine through. I tried it out with structured output for azure openai but had to give up since somewhere somewhat was broken and it's difficult to figure out if it's the abstraction or the library of the llm provider which the abstraction uses. Nevertheless i would strong…

In this example, you get locked into pydantic_ai, another proprietary provider.
Post reply on HN