Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Are Data Science Leaders Running for the Exit?

linkedin.com

Post reply on HN
Print view View original

81–90 of 102 posts

Re: Why Are Data Science Leaders Running for the Exit?

#81

Post by apohn »

wakkaflokka wrote: post #40
apohn wrote: post #32

Earlier quoted context omitted.

Until recently I led a Data Science team (manager and IC) at a non-software Fortune 10. I, and everybody else on the team, left within a year of joining. The division where I was in hired me to grow both Data Science practice and the team. The company was enormous, so there were data science teams all across the company, all operating in silos. The biggest problem can be summarized like this: If you want to run your…

Thank you for this comment. I joined a non-tech company as their first data scientist, and am now tasked with building up the data science efforts of the company. Your comment is really ringing true, though I've found the company here to be generally great to work with in terms of their flexibility. Funny enough, it took me months of working with the IT department to get a VM with 8-cores and 64 GB of RAM, or the abi…

In my experience, many companies are wanting to hire a lot more data scientists (and paying them six-figure salaries to boot), when what they really are looking for are what has traditionally been called a "data analyst" position. I.E., the entire sum of the job is Excel reporting and basic statistics, but instead wrapped in some "Machine Learning" marketing language.

100% Agree.

Some of the execs I reported into were obsessed with the idea of being a startup within a large organization, and part of this was the assumption that one of the things startups did was use data better than big companies. To them, there was some hidden value in their data that their Business Analysts did not have the skills to unlock. Data Scientists are perceived as the magic key to unlock this value, even when what the business is asking for doesn't make sense or isn't feasible in the short-term.

One of the small ways we delivered value was to write small ETL scripts that fed into Tableau dashboards. People loved the Dashboards because it cut down on so much Excel stuff that was done weekly/monthly/quarterly, etc. The company did have a large IT org with a BI team, but they were always busy and took months to do anything. So for us to do ETL+Tableau in a few days was seen as a miracle. Plus, Tableau has an Excel export :)

Of course, ETL+Tableau wasn't enough to retain the team.

Re: Why Are Data Science Leaders Running for the Exit?

#82

Post by megaman22 »

pseudonymGuy wrote: post #77

As someone (a Director) running for the exit... lack of control over: taking on new projects, staffing, infrastructure, meeting schedules, deadlines. Complete lack of ability to say "no" to our biggest client. Add in conflicting priorities from leadership, month long delays in compensation adjustment, lack of clarity around valuation status, and having to focus primarily on new and maintenance ETL work. I got frustra…

I feel ya; this is almost exactly what I've felt this year. It's just incompetent management and leadership, and it's all too common. But it's incredibly demoralizing to be on the front lines of it. When you can't control anything, and are reduced to reacting on ever more tactical issues, it burns you out hard and fast. Indecisiveness is killer; better to pick something and do it, than waffle around forever spinning the wheels.

Re: Why Are Data Science Leaders Running for the Exit?

#83

Post by thousandautumns »

steve_gh wrote: post #58
em500 wrote: post #54

Earlier quoted context omitted.

I'm sorry but a standard t-test is like the most basic thing in statistics after means and variances. It's not like they want you prove the CLT or something. To me it's more like a statistics fizz-buzz.

To which, of course, the correct answer is "Unless you have good reason to believe the data is normally distributed, you should be using Mann-Whitney", then explain that :-)

Thats not really true though. The vast majority of t-tests are done using a sample average as a test statistic, and by the CLT, as the sample size goes up, the distribution of a sample average becomes approximately normal. So unless you have a really weak sample size, the t-test assumption holds even in the absence of normally distributed data.

Re: Why Are Data Science Leaders Running for the Exit?

#84

Post by apohn »

kthejoker2 wrote: post #78
wakkaflokka wrote: post #40

Earlier quoted context omitted.

Thank you for this comment. I joined a non-tech company as their first data scientist, and am now tasked with building up the data science efforts of the company. Your comment is really ringing true, though I've found the company here to be generally great to work with in terms of their flexibility. Funny enough, it took me months of working with the IT department to get a VM with 8-cores and 64 GB of RAM, or the abi…

I work for a large consulting firm that does both of these activities (data engineer/analyst delivering rapid value + long-term data scientist / architects engineering fully integrated solutions connecting ML to business process) and I can say without a doubt we get much higher feedback from the the former than the latter. Most businesses think small. I'm not even sure it's about risk/reward, either. I think people r…

IMO every Data Scientist needs to be able to quickly identify a win and deliver quick value, even in a longer term ML project. Iterate and get something out. Most people in large businesses cannot do any data ETL or build a dashboard, so if you can do that in the first few weeks of a project you've already got a deliverable/win.

Having some expertise with BI applications (e.g. Tableau) has been a major strength for showing value.

Re: Why Are Data Science Leaders Running for the Exit?

#85

Post by Terr_ »

zebrafish wrote: post #53
eric_b wrote: post #36

Maybe these people want to leave because they realize they aren't adding value? I've worked in many Fortune 500's with "data science" or "big data" teams. These are well staffed, very expensive teams that have large budgets for pricey hardware (sometimes on-prem, sometimes in the cloud) I have never seen one of these teams produce insights or actionable intelligence valued anywhere near their cost. I mean not even cl…

I'm a firm believer that data science should be pulled out of IT and put into the business. However, storage is so cheap that you should never NOT collect data if you can. It's better to dump it into a data warehouse and wait for somebody who can use it to come along than it is to never collect it. Then, and only then, should you look at your service levels and determine if there is a need for some sort of Hadoop or…

> However, storage is so cheap that you should never NOT collect data if you can.

Unless it involves customers and clients, in which case it's actually a sneaky liability... At least in terms of PR, if not legality.

Re: Why Are Data Science Leaders Running for the Exit?

#86

Post by acdha »

zebrafish wrote: post #53
eric_b wrote: post #36

Maybe these people want to leave because they realize they aren't adding value? I've worked in many Fortune 500's with "data science" or "big data" teams. These are well staffed, very expensive teams that have large budgets for pricey hardware (sometimes on-prem, sometimes in the cloud) I have never seen one of these teams produce insights or actionable intelligence valued anywhere near their cost. I mean not even cl…

I'm a firm believer that data science should be pulled out of IT and put into the business. However, storage is so cheap that you should never NOT collect data if you can. It's better to dump it into a data warehouse and wait for somebody who can use it to come along than it is to never collect it. Then, and only then, should you look at your service levels and determine if there is a need for some sort of Hadoop or…

I like data too but that approach has several hazards. The most obvious is liability: if you store it, you have to be responsible for protecting against misuse.

The second, however, is more subtle but often more damaging: people often assume that the data they have is the right data or complete so the drunkard's lampost problem is easy to fall into and people might not realize it as quickly because, hey, those numbers were based on so much data it took a day to run the query! Web analytics is notorious for that — people would make statements about performance, browser support, etc. from a bunch of log files and miss that this was skewed by bot traffic, confuse their server's response time as being a reasonable proxy for the user's perceived load-time, etc.

Re: Why Are Data Science Leaders Running for the Exit?

#87

Post by apohn »

zebrafish wrote: post #53
eric_b wrote: post #36

Maybe these people want to leave because they realize they aren't adding value? I've worked in many Fortune 500's with "data science" or "big data" teams. These are well staffed, very expensive teams that have large budgets for pricey hardware (sometimes on-prem, sometimes in the cloud) I have never seen one of these teams produce insights or actionable intelligence valued anywhere near their cost. I mean not even cl…

I'm a firm believer that data science should be pulled out of IT and put into the business. However, storage is so cheap that you should never NOT collect data if you can. It's better to dump it into a data warehouse and wait for somebody who can use it to come along than it is to never collect it. Then, and only then, should you look at your service levels and determine if there is a need for some sort of Hadoop or…

>I'm a firm believer that data science should be pulled out of IT and put into the business.

I say this from personal experience. The major risk here is that Data Scientists will constantly get pulled in as resources to support business fires and high visibility projects that should be handled by other people.

Otherwise what happens is this: "I know you are working on that Data Science project that is scoped for 4 weeks, but can you do this Business Analyst thing the insert exec name here asked for by this Friday?"

Data Scientists need to be shielded from that day to day analyst stuff, otherwise they'll never be able to do their jobs. One of the ways to do this is to put them in IT in a business facing consulting role. IT is typically (not always) is more focused on a proper solution, not day to day triage. So they can filter out projects that aren't appropriate.

Re: Why Are Data Science Leaders Running for the Exit?

#88

Post by RHSman2 »

pseudonymGuy wrote: post #77

As someone (a Director) running for the exit... lack of control over: taking on new projects, staffing, infrastructure, meeting schedules, deadlines. Complete lack of ability to say "no" to our biggest client. Add in conflicting priorities from leadership, month long delays in compensation adjustment, lack of clarity around valuation status, and having to focus primarily on new and maintenance ETL work. I got frustra…

Keep at it. Find tour support or do it yourself.

Re: Why Are Data Science Leaders Running for the Exit?

#89

Post by RHSman2 »

zebrafish wrote: post #53
eric_b wrote: post #36

Maybe these people want to leave because they realize they aren't adding value? I've worked in many Fortune 500's with "data science" or "big data" teams. These are well staffed, very expensive teams that have large budgets for pricey hardware (sometimes on-prem, sometimes in the cloud) I have never seen one of these teams produce insights or actionable intelligence valued anywhere near their cost. I mean not even cl…

I'm a firm believer that data science should be pulled out of IT and put into the business. However, storage is so cheap that you should never NOT collect data if you can. It's better to dump it into a data warehouse and wait for somebody who can use it to come along than it is to never collect it. Then, and only then, should you look at your service levels and determine if there is a need for some sort of Hadoop or…

Data should be moved out of IT and put into Data. Data headed up by a Chief Data Officer should be a first class citizen of every company. Not f’in IT, not Buisness. Data.

Re: Why Are Data Science Leaders Running for the Exit?

#90

Post by RHSman2 »

chestervonwinch wrote: post #20

As someone near graduation with a PhD in mathematics with research focusing on machine learning / image processing and beginning the job search, this didn't leave me feeling very optimistic. Is this a problem of not setting proper expectations for a role on the company's part or not fully understanding the expected duties of the position before hired on the data scientist's part?

Get in with a small, clever company. Do, fail and repeat but better. Just like data science. Large companies. Meh.
Post reply on HN