Canaries in the Coal Mine? Recent Employment Effects of AI [pdf]

LLMs are very useful tools for software development, but focusing on employment does not appear to really dig into if it will automate or augment labor (to use their words). Behaviors are changing not just because of outcomes but because of hype and expectations and b2b sales. You'd expect the initial corporate behaviors to look much the same whether or not LLMs turn into fully-fire-and-forget employee-replacement tools.

Some nits I'd pick along those lines:

>For instance, according to the most recent AI Index Report, AI systems could solve just 4.4% of coding problems on SWE-Bench, a widely used benchmark for software engineering, in 2023, but performance increased to 71.7% in 2024 (Maslej et al., 2025).

Something like this should have the context of SWE-Bench not existing before November, 2023.

Pre-2023 systems were flying blind with regard to what they were going to be tested with. Post-2023 systems have been created in a world where this test exists. Hard to generalize from before/after performance.

> The patterns we observe in the data appear most acutely starting in late 2022, around the time of rapid proliferation of generative AI tools.

This is quite early for "replacement" of software development jobs as by their own prior statement/citation the tools even a year later, when SWE-Bench was introduced, were only hitting that 4.4% task success rate.

It's timing lines up more neatly with the post-COVID-bubble tech industry slowdown. Or with the start of hype about AI productivity vs actual replaced employee productivity.

To me it seems that LLMs are a tool that only increase productivity for given headcount in dimensions that were neglected in the past.

For example, everyone now writes emails with perfect grammar in a fraction of a time. So now the expectation for emails is that they will have perfect grammar.

Or one can build an interactive dashboard to visualize their spreadsheet and make it pleasing. Again the expectation just changed. The bar is higher.

So far I have not seen productivity increase in dimensions with direct sight to revenue. (Of course there is the niche of customer service, translation services etc that already were in the process of being automated)

I'm sad to see this for several reasons because I do not expect or want everyone up use a LLM to converse with me via mail, the whole point is to exchange information, with everyone using a LLM as output and input, now the whole thing becomes a game of telephone.

You do not need to build a spreadsheet visualiser tool there are plenty of options that exist and are free and open source.

I'm not against advances, I'm just really failing to see what problem was in need of solving here.

The only use I can get behind is the translation, which admittedly works relatively well with LLMs in general due to the nature of the work.

One would expect that if such studies indeed indicate that AI has an effect on early-career workers in AI-exposed occupations, that this would be a global effect. I wonder if there are good comparable non-US studies available.

It's an interesting dilemma, since if I know that an email was written mostly with AI, it feels to me like the author didn't put effort in, and thus I won't put much effort into reading the email.

I had a conversation with my manager about the implications of everyone using AI to write/summarise everything. The end result will most likely be staff getting Copilot to generate a report, then their manager uses Copilot to summarise the report and generate a new report for their manager, ad inifinitum.

Eventually all context is lost, busywork is amplified, and nobody gains anything.