I also looked at the study and noticed a few aspects that were surprising:
(a) some of the 95% CIs crossed zero, meaning no benefit is a possible interpretation in figures 6 and 7
(b) did anyone account for what happens when two workers are in different experimental groups and sit next to each other? I imagine it was likely common people in the experimental group to use co-pilot queries for their friends.
(c) for experienced workers, the mean value is actually negative (lol) in the unweighted data in Figure 7
There are a lot of other subtleties in the interpretation of this paper.
"...developers who were less productive before the experiment are significantly more likely to accept any given suggestion...."
...curious lol