Earlier quoted context omitted.
Conversely, I think you're overestimating the technical capailities of Canadian federal or provincial regulatory organizations. Government departments notoriously have serious technical debt, so while they might entitled to demand this kind of data, I sincerely doubt they are doing it with anything resembling modern data analysis. These orgs aren't going to have working data lakes, spark clusters or data warehouses o…
Canadian economics departments are full of really smart economists. They are happy to help. The issue here has nothing to do with the size of the data. A grocery store will have 100,000 products at most, probably sets prices about weekly and maybe you have ten years of data. The market is an oligopoly so you have 10-15 store chains. That fits in memory. The issue is not at all the mechanics of running the analysis bu…
The appropriate incentive is lacking.