What's the best programming language for coding agents?

> In our results, there was little sign of inter-language differences in solve rates, for any model (Figure 5b). This suggests that AI models have learned generalized programming skills, rather than pattern-matching syntax. This does not mean that implementation language is irrelevant. Conditional on solving a target, we found a small effect on token usage: successful Python solutions tended to use fewer tokens than average, while successful Ada solutions tended to use more (Appendix C). We consider these to be small differences, given that these six programming languages vary widely in how concise they are, and in how much functionality is provided by their standard library (recall that agents cannot download dependencies in MirrorCode, they must solve the task using only the standard library).

In Appendix C, Ada tended to use only about 25% more tokens than the average language. Ada is a language used mainly in safety-critical aerospace and defense systems, which has ~200x less pre-training data available than C or Python.

We're also comparing more recent language models (on just Go vs Ada, for cost reasons), on our leaderboard [2].

[1] https://arxiv.org/pdf/2606.30182

[2] https://epoch.ai/MirrorCode#leaderboard

It's like they haven't maintained any actual software, because the criteria they choose is... Completely irrelevant?

The things that matter are tooling, orchestration and ecosystem - as well as how the LLM will actually implement the solution for a task

LLMs constantly do idiotic things. If you have good libraries to utilize, the likelihood of the solution actually working goes up because they no longer need to implement the hard part.

If you have orchestration for dependency injection, code generation, meta analysis etc

Tooling like the way otel tracing is integrated, openapi generation etc is also invaluable because every time the LLM does something the likelihood of it being hallucinated/wrong increases etc

You'd need to implement a nontrivial project in different languages, then add nontrivial features across them and only then start by rating eg correctness and incident occurrence after the final output

But token use on a one shot? Completely irrelevant as far as I see it.

- https://ai-sdk.dev/

I have used it, and I can say it is really nicely written.

Matt Pocock has created a good tutorial on it:

- https://www.youtube.com/watch?v=mojZpktAiYQ