Slow down people. Let's stop jumping to biases and see what we have here.
Note upfront: I'm not suggesting AI is not having an impact. That would be foolish. But I will say there's *a lot* less to the conclusion of this study, simply because the data is questionable. It's not that they did anything wrong per se. I won't say that here because it'll end up a HN cluster fuck. Cluster fuck aside, the caveats and associated doubt are enough to say, "Don't bet the farm on this study." Great bander for the bar? Sure.
It's an interesting study but I've seen it called "absolute proof" and other type things. Don't be fooled, it's not that.
https://digitaleconomy.stanford.edu/wp-content/uploads/2025/...
From the original study:
> "This study uses data from ADP, the largest payroll processing firm in America. The company provides payroll services for firms employing over 25 million workers in the US. We use this information to track employment changes for workers in occupations measured as more or less exposed to artificial intelligence"
a) I'm calling this out because I've seen posts on LinkedIn saying it was a sample of 25M. Nope! ADP simply does payroll for that many.
b) The size of the US workforce is ~165M, making ADP's coverage ~15% of the workforce.
https://www.statista.com/statistics/191750/civilian-labor-fo...
c) Do the business ADP server come from particular industries, are of a particular size, in particular geographic locations? etc.? It's not only about the size of the sample - which we'll get to shortly - but the nature of the companies - which we'll also get to shortly.
> "We make several sample restrictions for our main analysis sample."
d) It's great that they say this, but it should raise an eyebrow.
> "We include only workers employed by firms that use ADP’s payroll product to maintain worker earnings records. We also exclude employees classified by firms as part-time from the analysis and subset to people between the age of 18 and 70."
e) Translation: we did a slight bit of pruning (read: cherry-picking).
> "The set of firms using payroll services changes over time as companies join or leave ADP’s platform. We maintain a consistent set of firms across our main sample period by keeping only companies that have employee earnings records for each month from January 2021 through July 2025."
f) Translation: More cherry-picking.
> "In addition, ADP observes job titles for about 70% of workers in its system. We exclude workers who do not have a recorded job title."
g) Translation: More cherry-picking.
> "After these restrictions we have records on between 3.5 and 5 million workers each month for our main analysis sample, though we consider robustness to alternative analyses such as allowing for firms to enter and leave the sample."
h) 3.5M to 5.0M feels like a large enough sample... if it wasn't so "restricted." Furthermore, there's no explanation on the 1.5M delta, and how adding or removing that much impacts the analysis.
i) And they considered that why? And did what they did why? It's a significant assumpt that gets nothing more than a hand wave?
> "While the ADP data include millions of workers in each month, the distribution of firms using ADP services does not exactly match the distribution of firms across the broader US economy."
j) Translation: as mentioned above ADP !== a representation of the broader economy.
> "Further details on differences in firm composition can be found in Cajner et al. (2018) and ADP Reserch (2025)."
j) Great there's a citation, but given the acknowledgement of the delta isn't at least a line or two in order? Something about the nature of the delta, and THEN mention the citation?
k) Editorial: You might think this hand-wave is ok, but to me it's usually indicative of a tell and a smell.
l) Finally, do understand the nature of academia and null research (which has been mentioned on HN). In short, there is a (career / financial) incentive to find something novel (read: worth publishing). You advance your career by doing not-null
research.
Again, I'm not suggesting anything nefarious per se. But this study is getting A LOT of attention. All things considered, more than it objectively deserves.
__Again: I'm not suggesting AI is not having an impact. That would be foolish.__