LLM Economist – Mechanism Design for Simulated Agent Societies
1–10 of 10 posts
Re: LLM Economist – Mechanism Design for Simulated Agent Societies
#2We simulate large-scale agent societies where heterogeneous personas work, adapt, and vote—governed by an in-context planner optimizing social welfare.
The system models decentralized governance, dynamic tax policy, and institutional evolution—entirely via in-context reinforcement learning, no fine-tuning required.
Full paper (arXiv): https://arxiv.org/abs/2507.15815
Re: LLM Economist – Mechanism Design for Simulated Agent Societies
#3We simulate large-scale agent societies where heterogeneous personas work, adapt, and vote—governed by an in-context planner optimizing social welfare. The system models decentralized governance, dynamic tax policy, and institutional evolution—entirely via in-context reinforcement learning, no fine-tuning required. Full paper (arXiv): https://arxiv.org/abs/2507.15815
I like the idea of simulating a society! I don't pretend to understand everything that you're doing, so please correct me where I'm wrong below.
The right side of Fig 5a shows that your LLM tool has 80% tax for people making between 0 and $11.6k/year, then drops to about 30% for the next tax bracket, with other tax brackets moving around all over the place. This seems to be designed to induce people to NOT pay taxes.For all its faults, I think the US progressive system is fairly rational and does a pretty good job of inducing people to actually pay taxes [1]; specifically the (effectively) negative tax rate in the US for low-income people gets them in the habit of paying taxes. I.e. whatever underlying model of social welfare you are assuming to get the great social welfare on the right side of Fig 5a seems to not model real people. I wonder if some LLM hallucinations are going on under the hood to create the strange behavior in Fig 5a.
Some questions: You don't seem to model the US system of tax credits; is that right? Also, is there a Saez tax below $47.2k in Fig 5a? What about between $244k and $609k? I.e. is the Saez tax ever under the LLM tax?
[1] https://blogs.worldbank.org/en/governance/why-does-progressi...
Re: LLM Economist – Mechanism Design for Simulated Agent Societies
#4We simulate large-scale agent societies where heterogeneous personas work, adapt, and vote—governed by an in-context planner optimizing social welfare. The system models decentralized governance, dynamic tax policy, and institutional evolution—entirely via in-context reinforcement learning, no fine-tuning required. Full paper (arXiv): https://arxiv.org/abs/2507.15815
I like the idea of simulating a society! I don't pretend to understand everything that you're doing, so please correct me where I'm wrong below. The right side of Fig 5a shows that your LLM tool has 80% tax for people making between 0 and $11.6k/year, then drops to about 30% for the next tax bracket, with other tax brackets moving around all over the place. This seems to be designed to induce people to NOT pay taxes.…
These are the marginal tax rates not the effective tax rate (e.g. 80% of first $10k, 30% of $10k-20k). We do not model tax credits here. We try to keep the system as simple as possible so that we can effectively evaluate changes. As is, the Economic theory is intractable once we introduce bounded rationality from purely rational. We do think in future work we can potentially work out some smoothness in the overall tax rate but try to let the LLM planner try what it thinks is best in order to help test the in-context optimization capabilities.
Also, while there is a complicated tax code in the US, in our simulation there is no way for agents to avoid paying taxes :)
The Saez tax rates are perturbed from the LLM Economist's tax rates to find the theoretically optimal values according to the economic theory.
Thanks for the interest and I hope that this helps clarify some of the details.
Re: LLM Economist – Mechanism Design for Simulated Agent Societies
#5Earlier quoted context omitted.
I like the idea of simulating a society! I don't pretend to understand everything that you're doing, so please correct me where I'm wrong below. The right side of Fig 5a shows that your LLM tool has 80% tax for people making between 0 and $11.6k/year, then drops to about 30% for the next tax bracket, with other tax brackets moving around all over the place. This seems to be designed to induce people to NOT pay taxes.…
These are the marginal tax rates not the effective tax rate (e.g. 80% of first $10k, 30% of $10k-20k). We do not model tax credits here. We try to keep the system as simple as possible so that we can effectively evaluate changes. As is, the Economic theory is intractable once we introduce bounded rationality from purely rational. We do think in future work we can potentially work out some smoothness in the overall ta…
Thanks for the further details!
Ah, the fact that they are marginal rates makes marginally more sense, but it still seems to me that the SWF in fig 5a has very little relation to the real world.
> Also, while there is a complicated tax code in the US, in our simulation there is no way for agents to avoid paying taxes :)
Seems like an obvious thing to add. I.e. if you believe the World Bank when they say "People are more willing to pay tax when taxes are progressive" [1], then it seems worthwhile to update your model to include this.
[1] https://blogs.worldbank.org/en/governance/why-does-progressi...
Re: LLM Economist – Mechanism Design for Simulated Agent Societies
#6Interesting approach, but I'm curious about the practical cost considerations. A 1,000-agent simulation could easily be hundreds of thousands of API calls. The repo recommends gpt-4o-mini over gpt-4 and supports local Llama models, but there's no guidance on the performance trade-offs.
Would love to see cost-per-experiment breakdowns and quality benchmarks across model tiers. Does a local Llama 3.1 8B produce meaningful economic simulations or do you need the reasoning power of frontier models? This could be the difference between $5 and $500 experiments.
Re: LLM Economist – Mechanism Design for Simulated Agent Societies
#7Interesting approach, but I'm curious about the practical cost considerations. A 1,000-agent simulation could easily be hundreds of thousands of API calls. The repo recommends gpt-4o-mini over gpt-4 and supports local Llama models, but there's no guidance on the performance trade-offs. Would love to see cost-per-experiment breakdowns and quality benchmarks across model tiers. Does a local Llama 3.1 8B produce meaning…
Using smaller, cheaper agents is one of the goals of the work. There is a Pareto frontier though: by using smaller, faster, cheaper agents, the number of steps required to converge increases. We touch upon this briefly in the paper
Re: LLM Economist – Mechanism Design for Simulated Agent Societies
#8Interesting approach, but I'm curious about the practical cost considerations. A 1,000-agent simulation could easily be hundreds of thousands of API calls. The repo recommends gpt-4o-mini over gpt-4 and supports local Llama models, but there's no guidance on the performance trade-offs. Would love to see cost-per-experiment breakdowns and quality benchmarks across model tiers. Does a local Llama 3.1 8B produce meaning…
Using smaller, cheaper agents is one of the goals of the work. There is a Pareto frontier though: by using smaller, faster, cheaper agents, the number of steps required to converge increases. We touch upon this briefly in the paper
Thanks. That Pareto trade-off is exactly what I'm trying to quantify not just qualify. For example, if I've got a $50 budget, what's the sweet spot?
Scenario A: 100 agents × GPT-4o-mini × 500 steps Scenario B: 500 agents × local Llama 3-8B × 1,000+ steps
A quick table like "X agents × Y model × Z steps → tokens, $, convergence score" in the README would let new users budget experiments without having to read the whole paper plus run expensive experiments just to discover basic resource planning.
Re: LLM Economist – Mechanism Design for Simulated Agent Societies
#9Earlier quoted context omitted.
Using smaller, cheaper agents is one of the goals of the work. There is a Pareto frontier though: by using smaller, faster, cheaper agents, the number of steps required to converge increases. We touch upon this briefly in the paper
Thanks. That Pareto trade-off is exactly what I'm trying to quantify not just qualify. For example, if I've got a $50 budget, what's the sweet spot? Scenario A: 100 agents × GPT-4o-mini × 500 steps Scenario B: 500 agents × local Llama 3-8B × 1,000+ steps A quick table like "X agents × Y model × Z steps → tokens, $, convergence score" in the README would let new users budget experiments without having to read the whol…
We ran each method in under 24 hours on a singular H100. I understand your point and think we will include this in future iterations of our work since this is very interesting from the user perspective. Though, in the paper we focus more on algorithmic concerns.
Re: LLM Economist – Mechanism Design for Simulated Agent Societies
#10Earlier quoted context omitted.
Thanks. That Pareto trade-off is exactly what I'm trying to quantify not just qualify. For example, if I've got a $50 budget, what's the sweet spot? Scenario A: 100 agents × GPT-4o-mini × 500 steps Scenario B: 500 agents × local Llama 3-8B × 1,000+ steps A quick table like "X agents × Y model × Z steps → tokens, $, convergence score" in the README would let new users budget experiments without having to read the whol…
We ran each method in under 24 hours on a singular H100. I understand your point and think we will include this in future iterations of our work since this is very interesting from the user perspective. Though, in the paper we focus more on algorithmic concerns.
I'll look out for future iterations. Thanks and good luck with the paper.