Having done both sides of the work, I totally understand the author’s perspective. Modelling and coding production are certainly two different skillsets, and many people will prefer one or the other, but not both. Now, there is one thing I guess the author doesn’t get (or he gets but doesn’t like) and a thing that most companies don’t get.
The former is that companies don’t care enough about what the individuals they hire prefer - especially not if it doesn’t address their need. They don’t need a beautifully crafted model that can’t be run - they want actionable results that hit the bottom line, and in the energy forecasting model that means generating new forecasts at every cycle (months, days, hours, minutes - whatever). A great model that can’t be put in production and run efficiently has about the same value as no model, and an average model in production will have much more impact.
The latter is that companies don’t usually understand that ML is not software development. Putting ML in production is, but finding the right model is research work, and code is mostly a discardable tool for research. Its goal is not to go live, it is to validate a hypothesis (in this case, that algorithm X, when presented with data Y, generates a model with enough predicting power to be useful to the business). This validation requires code to gather Y, clean/join/analyze/reshape/featurize it into a more informative and clean representation (clean from the algorithm’s perspective, not necessarily a human’s), run X, run inference with the generated model and run some test of the results against additional data.
If this test is negative, some or all the code written above is useless, and we go back to the drawing board. Given the very coupled nature of this process (a new data source has to be joined to the rest - coupling; a new data transformation changing a feature changes the data schema downstream - coupling; a new algorithm needs a different data input - coupling, and so it goes). If you have an experience like mine, you may actually be able to write in a way where you can reuse some of it, but I have 28 years of experience with data, there are simply not enough people in the market with that level of background or the interest in learning all this. Companies must accept that they will not always get this perfect candidate with all the skills they want, and start thinking of pairing the right people in teams.
Some who have been around for a while may remember the Venn diagram of the perfect Data Scientists - it usually was an intersection of business, math and programming skills (also often communication skills and a few others). My thoughts since I first saw this diagram were: “Even if there are people out there with all these skills, why would they want to work for others?”
This, more than anything else, is my guess at the core reason why so few companies are successful in putting ML in production.