Why Are Data Science Leaders Running for the Exit? • Hacker News Board

Re: Why Are Data Science Leaders Running for the Exit?

#51

Post by wakkaflokka » Fri, Dec 08, 2017, 4:18 PM UTC

Asdfbla wrote: post #25

shas3 wrote: post #18

Earlier quoted context omitted.

They’re probably in the wrong company. Are you suggesting there are isn’t an explosion of jobs and applications requiring advance math and machine learning or statistical inference? If so you are off the mark.

You're right of course, but still there is a point: For 1 'intellectually' interesting data science job created there's probably 10 that have more to do with data massaging and all the relatively boring logistical stuff that come with data science. Doesn't invalidate your point, but the majority of jobs under the very broad 'data science' label just aren't super interesting after all. Guess you just have to be carefu…

I've found this to be very true. I've got friends who were hired as data scientists, paid a great salary, and are finding themselves doing basic reporting and SQL querying all day. No statistical modeling, and certainly no building anything.

Re: Why Are Data Science Leaders Running for the Exit?

#52

Post by sonabinu » Fri, Dec 08, 2017, 4:19 PM UTC

Xcelerate wrote: post #47

Speak for yourself. I work as a data scientist at a large (non-tech) company. Our team of data scientists is directly responsible for massive amounts of profit that the company would not otherwise have. This article seems like a baseless generalization.

A lot of value from Data Science comes in large non-tech companies. It's not the super glamorous type of DS. It is about using data to drive optimizations, process improvements, manage supply chains, instrumenting alert mechanisms to monitor key mechanical and software components etc.

Re: Why Are Data Science Leaders Running for the Exit?

#53

Post by zebrafish » Fri, Dec 08, 2017, 4:23 PM UTC

eric_b wrote: post #36

Maybe these people want to leave because they realize they aren't adding value? I've worked in many Fortune 500's with "data science" or "big data" teams. These are well staffed, very expensive teams that have large budgets for pricey hardware (sometimes on-prem, sometimes in the cloud) I have never seen one of these teams produce insights or actionable intelligence valued anywhere near their cost. I mean not even cl…

I'm a firm believer that data science should be pulled out of IT and put into the business. However, storage is so cheap that you should never NOT collect data if you can. It's better to dump it into a data warehouse and wait for somebody who can use it to come along than it is to never collect it.

Then, and only then, should you look at your service levels and determine if there is a need for some sort of Hadoop or Spark infrastructure.

Re: Why Are Data Science Leaders Running for the Exit?

#54

Post by em500 » Fri, Dec 08, 2017, 4:31 PM UTC

autokad wrote: post #22

ataturk wrote: post #12

It was all hype. All of it. I enjoyed learning about data science, but in the end, there were no jobs I could actually apply for and realistically get. I also don't think they were providing the big wins for the company that would justify what they were getting paid. Again, all hype.

same here. i work great in teams, masters in computer science, do really well in kaggle competitions, I understand when to use the algorithms, how they work, etc. but in the very few ds interviews I had, I was tanked as soon I got asked questions like: whats the formula for a T test. I know what the test is, and when its not appropriate to apply it, but I don't memorize those kinds of formulas. the field is just too…

I'm sorry but a standard t-test is like the most basic thing in statistics after means and variances. It's not like they want you prove the CLT or something. To me it's more like a statistics fizz-buzz.

Re: Why Are Data Science Leaders Running for the Exit?

#55

Post by knownothing » Fri, Dec 08, 2017, 4:43 PM UTC

reureu wrote: post #15

This is an article making baseless generalizations that all data science is failing in Silicon Valley, and that the true need of a company is to hire a "data strategist." Unsurprisingly, the article is written by a self-proclaimed data strategist. This blog post would be more compelling if there were citations or even general context backing up the authors pretty wild claims ("the vast majority (we are talking 80-90%…

> There are many under-/un-qualified managers running around (I don't know if that's a data science specific problem, or if it also applies to much of engineering as well).

This is a problem no matter what line of work you're in.

Re: Why Are Data Science Leaders Running for the Exit?

#56

Post by chestervonwinch » Fri, Dec 08, 2017, 5:05 PM UTC

ska wrote: post #34

chestervonwinch wrote: post #20

As someone near graduation with a PhD in mathematics with research focusing on machine learning / image processing and beginning the job search, this didn't leave me feeling very optimistic. Is this a problem of not setting proper expectations for a role on the company's part or not fully understanding the expected duties of the position before hired on the data scientist's part?

It's a bit of both. You are unlikely to ever achieve a position in industry that has the same freedom of investigation and time that you experienced as a graduate student, let alone the holy grail of a self-funded post-doc or the like. On the other hand, you won't achieve that as faculty either. In industry you're going to end up spending a lot of time on activities and meetings that don't relate to what you now cons…

> You are unlikely to ever achieve a position in industry that has the same freedom of investigation and time that you experienced as a graduate student, let alone the holy grail of a self-funded post-doc or the like. On the other hand, you won't achieve that as faculty either.

which explains my advisor's "enjoy it while you still can!" attitude whenever we talk of such things :)

Re: Why Are Data Science Leaders Running for the Exit?

#57

Post by ska » Fri, Dec 08, 2017, 5:07 PM UTC

chestervonwinch wrote: post #56

ska wrote: post #34

Earlier quoted context omitted.

It's a bit of both. You are unlikely to ever achieve a position in industry that has the same freedom of investigation and time that you experienced as a graduate student, let alone the holy grail of a self-funded post-doc or the like. On the other hand, you won't achieve that as faculty either. In industry you're going to end up spending a lot of time on activities and meetings that don't relate to what you now cons…

> You are unlikely to ever achieve a position in industry that has the same freedom of investigation and time that you experienced as a graduate student, let alone the holy grail of a self-funded post-doc or the like. On the other hand, you won't achieve that as faculty either. which explains my advisor's "enjoy it while you still can!" attitude whenever we talk of such things :)

They speak truth :)

Re: Why Are Data Science Leaders Running for the Exit?

#58

Post by steve_gh » Fri, Dec 08, 2017, 5:07 PM UTC

em500 wrote: post #54

autokad wrote: post #22

Earlier quoted context omitted.

same here. i work great in teams, masters in computer science, do really well in kaggle competitions, I understand when to use the algorithms, how they work, etc. but in the very few ds interviews I had, I was tanked as soon I got asked questions like: whats the formula for a T test. I know what the test is, and when its not appropriate to apply it, but I don't memorize those kinds of formulas. the field is just too…

I'm sorry but a standard t-test is like the most basic thing in statistics after means and variances. It's not like they want you prove the CLT or something. To me it's more like a statistics fizz-buzz.

To which, of course, the correct answer is "Unless you have good reason to believe the data is normally distributed, you should be using Mann-Whitney", then explain that :-)

Re: Why Are Data Science Leaders Running for the Exit?

#59

Post by rockinghigh » Fri, Dec 08, 2017, 5:09 PM UTC

autokad wrote: post #22

ataturk wrote: post #12

It was all hype. All of it. I enjoyed learning about data science, but in the end, there were no jobs I could actually apply for and realistically get. I also don't think they were providing the big wins for the company that would justify what they were getting paid. Again, all hype.

same here. i work great in teams, masters in computer science, do really well in kaggle competitions, I understand when to use the algorithms, how they work, etc. but in the very few ds interviews I had, I was tanked as soon I got asked questions like: whats the formula for a T test. I know what the test is, and when its not appropriate to apply it, but I don't memorize those kinds of formulas. the field is just too…

For many companies, data science means statistics. Statistics are used for experiments and to analyze data. If you don't know what a t-test is, you cannot be hired as a statistician. However, you may be hired as a machine learning engineer.

Re: Why Are Data Science Leaders Running for the Exit?

#60

Post by ska » Fri, Dec 08, 2017, 5:12 PM UTC

soVeryTired wrote: post #44

chestervonwinch wrote: post #20

As someone near graduation with a PhD in mathematics with research focusing on machine learning / image processing and beginning the job search, this didn't leave me feeling very optimistic. Is this a problem of not setting proper expectations for a role on the company's part or not fully understanding the expected duties of the position before hired on the data scientist's part?

A major point that's underappreciated in academia is quality of execution. The more complicated a model is, the less well you can execute it. Academics are rewarded for building complicated things, but it often takes graduate PhDs entering industry a long time to learn about quality of execution. I think that's the fundamental tension between business and academia.

I wanted to say that I disagreed with you, but then I realized I don't know what you mean by "execution" here, though.

This statement "The more complicated a model is, the less well you can execute it" is fishy though. The typical problem in practice with complicated models is the data quality and quantity to support it, not the implementation or interpretation. I suppose here we also need to define "complexity" - I am using this to mean roughly number of parameters.