Why Are Data Science Leaders Running for the Exit?

Until recently I led a Data Science team (manager and IC) at a non-software Fortune 10. I, and everybody else on the team, left within a year of joining. The division where I was in hired me to grow both Data Science practice and the team. The company was enormous, so there were data science teams all across the company, all operating in silos.

The biggest problem can be summarized like this: If you want to run your business differently, let the new people you hire do something different. If you want to do the same old thing and just act you are doing something differently, just teach the old people to talk like the new people.

I've got plenty of anecdotes from the year, but here's a simple one. The division had an existing "Research and Analytics" group that used linear regression in Excel to develop models. They would develop models on awful (read: highly-multicollinear) data with 100 rows and get "R-squared of .98." Large long-term business decisions were made on these models.

Over the course of the year I fought this constantly. Why do you need more/better data when the other team is okay with the current data? Why are you struggling to get your R-squared to >.90 (output of any type of model had to be R-squared) when the other team can do it so easily? Why do you need to speak to so many people to understand the data better? What I realized over the course of the year was that nobody actually cared about the models unless they matched the "gut" of senior management. So it was easy for the other team to p-hack until they got a slidedeck people would like.

I think there are plenty of problems that lend themselves to good Data Science. These are ones where you can properly identify what a prediction or better understanding of data (e.g. variable importance, coefficient confidence intervals) means in terms of resources (e.g. people, money, time). You can also identify what it means to not have that data insight or model.

There are also plenty of business areas where the link between resource investment and long term success is less clear (e.g. Marketing, Sales Incentives) and more gut and smart work than science. A lot of people are trying to clear up that grey with data science, but I go back to what I said before: you need to let people think and try different things over time to see the value of those experiments.

Comically enough, the Research and Analysis team picked up all the lingo we used and became a huge success: work the old way, talk in the new way. Management decided their Data Science adventure was a failed experiment and we just hadn't lived up to the hype.

It's a bit of both.

You are unlikely to ever achieve a position in industry that has the same freedom of investigation and time that you experienced as a graduate student, let alone the holy grail of a self-funded post-doc or the like. On the other hand, you won't achieve that as faculty either. In industry you're going to end up spending a lot of time on activities and meetings that don't relate to what you now consider "what you do" (e.g. ETL, domain interview, co-ordination etc.). Your success and happiness in this will depend a lot on whether you consider those things a waste of time taking you away from what you "really do", or an important part of how you do such activities in larger groups.

On the other hand, data science has a similar problem as software programming; failing to know how to find or produce good managers for it, companies tend to promote from the ranks of their best individual contributors. This can work well with enough support and mentorship (assuming the candidate wants to do it) but it can be a disaster without a plan and that support.

Maybe these people want to leave because they realize they aren't adding value?

I've worked in many Fortune 500's with "data science" or "big data" teams. These are well staffed, very expensive teams that have large budgets for pricey hardware (sometimes on-prem, sometimes in the cloud)

I have never seen one of these teams produce insights or actionable intelligence valued anywhere near their cost. I mean not even close. Usually it is a tremendous money fire. (Also, before the pitchforks come out, I'm sure there are places where the data science team is a profitable department, but it's not the norm, not by any stretch)

Part of the problem is the business doesn't know what questions to ask. Part of the problem is the technology itself. Spark streaming, Hadoop, and all the other tools really aren't very good (very good being defined by helping businesses answer burning questions in a reliable and timely manner)

The most valuable data insights I've seen come from purpose built analytics tools using simple storage backends (RDBMS, elasticsearch etc) where the person running the team is a domain expert, not a "data scientist".

> in Silicon Valley

Ironically, I assumed he was talking about NYC. Apparently he's actually in Minneapolis

Thank you for this comment.

I joined a non-tech company as their first data scientist, and am now tasked with building up the data science efforts of the company. Your comment is really ringing true, though I've found the company here to be generally great to work with in terms of their flexibility. Funny enough, it took me months of working with the IT department to get a VM with 8-cores and 64 GB of RAM, or the ability to spin up AWS instances (none of the analysts at the company had ever asked for that). I'm happy with it though, it's a fascinating challenge and I'll admit at times I feel a bit in over my head.

In my experience, many companies are wanting to hire a lot more data scientists (and paying them six-figure salaries to boot), when what they really are looking for are what has traditionally been called a "data analyst" position. I.E., the entire sum of the job is Excel reporting and basic statistics, but instead wrapped in some "Machine Learning" marketing language.

As opposed to more long-term focus on what business problems can be solved by carefully cleaning data, building models, testing predictive power, and then deploying these models into production. Which obviously takes a lot more time, a lot more effort focusing on a single problem, and doesn't have nearly as many immediate deliverables that higher-ups can open their Outlook inbox and see. Naturally they then wonder why they are paying such a premium for someone who doesn't bring as many deliverables to the table as their 60-70k analysts. And then the data scientist wonders why he's spending most of his time sending Excel attachments. Definitely more of a problem in non-tech companies.