One thing I cannot understand is why we have both: (a) a long history of research proving to us that principles like "High-Quality Work Produced = (Time Spent) x (Intensity of Focus)" are accurate and highly related to extracting the most economically valuable outputs from knowledge workers. (b) open plan offices. Maybe the tech industry is daunting for newcomers because we cram people into a sardine can, give them a…
The reason is because management is almost never driven by science or productivity maximization but maximization of either narrow subdivided metrics (like costs for a particular function) or absolute cargo cult conventional wisdom. Even when there is an empirical, data driven culture somewhere in an organization, it rarely extends to the board, often doesn't extend to the executive suite, and almost never extends int…
I think the answer is more plain: shared offices cut cost, and and the resulting immediate an ongoing increase in profit is 'the most important of all benchmarks' for most companies, in fact it's not even a 'benchmark' ... it's 'the point'.
So - essentially, 'private offices' are an 'investment' and an investment has to be clearly and obviously justified, otherwise the default is to go to open office plans.
The vast majority of work simply is not deep work - and yes, even things like 'Marketing' - although most operational marketing people have a 'creative bone' - and brilliant marketers are as useful and brilliant as equivalent Engineer, the actual creative/deep thinking Marketing stuff is usually done in agencies, which are like architecture firms. Similarly I don't doubt creative financial structuring or deal making, 'deep time' is just not as consistent for them as it is for most Engineers.
There is actually rather scarce 'deep work' to be done, and so, as so few companies willingly require people to be 'learning very important new skills' ... offices become less common due to the simple economics of it all.