Data Science: Reality Doesn't Meet Expectations • Hacker News Board

Re: Data Science: Reality Doesn't Meet Expectations

#31

Post by beckingz » Tue, Apr 07, 2020, 9:17 PM UTC

dtjones wrote: post #10

stuxnet79 wrote: post #6

I've been a data person for the past year and a half and I'm very disappointed with the bewildering array of titles out there and the rather vague meanings behind them (Data Analyst, Data Scientist, Data Engineer, ML Engineer). It's overall hurting my ability to build my personal brand and seek roles that are a fit for my existing skillset and aspirations. What exactly does 'ML Engineer' communicate to employers in t…

I've been working in data roles for 10 years and hold a masters in ML. I've hired and managed each of the roles you mentioned. I think of the responsibilities of each of those roles as: -ML Engineers as building software infrastructure to scale machine learning inference and training. -Data engineers focusing on data infrastructure and pipelining into either model inference, training, or other business intelligence p…

How hard could it be to find one person who can do all that?

Re: Data Science: Reality Doesn't Meet Expectations

#32

Post by beckingz » Tue, Apr 07, 2020, 9:18 PM UTC

_jal wrote: post #21

worik wrote: post #11

I do not understand. Have never understood. "Data Science" is, surly, newspeak. The appropriate term, surly, is "statistics".

A new title means a new opportunity to ask for more money and influence. See also, "devops".

Going from IT to devops is a great way to double your salary.

Re: Data Science: Reality Doesn't Meet Expectations

#33

Post by kqr » Tue, Apr 07, 2020, 9:19 PM UTC

hectormalot wrote: post #5

> Moreover, you may quickly realize much of this work is repetitive and while time-consuming, is “easy”. In fact, most analyses involve a great deal of time to understand the data, clean it and organize it. You may spend a minimal amount of time doing the “fun” parts that data scientists think of: complex statistics, machine learning and experimentation with tangible results. This. Universities and online challenges…

I took a data visualisation class in uni that handled this really cleverly. The second assignment sounded very easy. The teacher provided links to the sources where we could find data.

Most people figured that with such a simple assignment (not significantly harder than the first one, which was also easy-ish) they could put off doing it until the last moment.

Most people failed.

This real world data needed hours upon hours of cleaning before it was in any way useable. Of course, the teacher knew this, gave bonus points to the ones who did start in time, and then extended the deadline as he had expected to from the start.

Never again will I underestimate the dirtiness of real world data. One of the best teachers I had.

Re: Data Science: Reality Doesn't Meet Expectations

#34

Post by codingslave » Tue, Apr 07, 2020, 9:20 PM UTC

The real issue with data science, from the perspective of ML pipelines/using ml in products, is most people are straight up not smart enough for it. The second the problem falls outside the bounds of a commonly used model, 90% of data scientists are ill equipped to come up with a profitable solution. So they stumble around in the dark, producing nothing of real value. People underestimate the degree to which extreme mathematical maturity and skill can bend the results of commonly used ml models.

Re: Data Science: Reality Doesn't Meet Expectations

#35

Post by starpilot » Tue, Apr 07, 2020, 9:20 PM UTC

stuxnet79 wrote: post #6

I've been a data person for the past year and a half and I'm very disappointed with the bewildering array of titles out there and the rather vague meanings behind them (Data Analyst, Data Scientist, Data Engineer, ML Engineer). It's overall hurting my ability to build my personal brand and seek roles that are a fit for my existing skillset and aspirations. What exactly does 'ML Engineer' communicate to employers in t…

This reminds of the latter days of the LAMP stack. A "web developer" might do front/backend and sysadmin work. I think some people see "data scientist" similarly, wearing many (all) hats, which can work for some environments, but not most corporate ones.

Re: Data Science: Reality Doesn't Meet Expectations

#36

Post by rafiki6 » Tue, Apr 07, 2020, 9:23 PM UTC

antipaul wrote: post #29

rafiki6 wrote: post #23

I've seen a few similar articles now. Does this represent the general view of folks working in data science? "Data Science" is such as meaningless catch all term. The reality is in many organizations it's simply advanced business intelligence or advanced business analytics. There are some industries that lend themselves well to this whole practice, and they tend to be industries that have been borne out of the intern…

100% agree with the article. The top misconceptions are spot on. I’m at a big place where data science hype among leadership couldn’t be bigger.

If I may ask, what were you told when you interviewed that convinced you to join?

Re: Data Science: Reality Doesn't Meet Expectations

#37

Post by smeeth » Tue, Apr 07, 2020, 9:24 PM UTC

rafiki6 wrote: post #23

I've seen a few similar articles now. Does this represent the general view of folks working in data science? "Data Science" is such as meaningless catch all term. The reality is in many organizations it's simply advanced business intelligence or advanced business analytics. There are some industries that lend themselves well to this whole practice, and they tend to be industries that have been borne out of the intern…

I always thought the non-specificity of the term Data Science was a strange criticism for those in the tech industry to make. How many types of SWE are there? Front-end, back-end, full-stack, devops, security, QA...

I agree wholeheartedly with your recommendation. Like any other job, each company has different needs and expectations and if you want something else out of the role you'd best avoid that company.

Re: Data Science: Reality Doesn't Meet Expectations

#38

Post by s1t5 » Tue, Apr 07, 2020, 9:26 PM UTC

I work as a data scientist. Some of the author's points are workplace-specific: lack of leadership, being the only data person, ethical concerns. The others are just aspects of the job - communicating about your job and impact, dealing with vague specs or managing low-quality datasets.

Neither of those quite match the articles title, perhaps it just refers to the author's personal expectations. Neither of them seem that specific to data science, or without parallels in other software jobs. And neither of the points read like a slight towards data science to me, like some of the other commenters here suggest.

Re: Data Science: Reality Doesn't Meet Expectations

#39

Post by tqi » Tue, Apr 07, 2020, 9:30 PM UTC

kristjansson wrote: post #4

Teams being small, data being crummy, infra being hard, and yet expectations being high aren't so much complaints as the they are the job description. The point of data scientists and the related roles listed in the article are not to just churn out the fun stuff, but to wade through the institutional and technical muck and mire it takes to bring the fun stuff to bear on a relevant business problem and to communicate…

Yeah this guy seems to think Data Science work should be like doing a problem set for CS class. Sorry that you have to deal with messy data, fragile infra, and limited resources - I know it's not "fun", but frankly that's what the money is for.

Re: Data Science: Reality Doesn't Meet Expectations

#40

Post by rafiki6 » Tue, Apr 07, 2020, 9:31 PM UTC

smeeth wrote: post #37

rafiki6 wrote: post #23

I've seen a few similar articles now. Does this represent the general view of folks working in data science? "Data Science" is such as meaningless catch all term. The reality is in many organizations it's simply advanced business intelligence or advanced business analytics. There are some industries that lend themselves well to this whole practice, and they tend to be industries that have been borne out of the intern…

I always thought the non-specificity of the term Data Science was a strange criticism for those in the tech industry to make. How many types of SWE are there? Front-end, back-end, full-stack, devops, security, QA... I agree wholeheartedly with your recommendation. Like any other job, each company has different needs and expectations and if you want something else out of the role you'd best avoid that company.

Frankly I have the same criticism of those who use the term software engineer. Engineering is a pretty established profession with a set of standards, ethics and practices. Most of us who work in software are not engineers. We are developers. Similarly, a scientist is one who follows the scientific method to do research. So by that logic a data scientist should be a person who uses the scientific method to do research on data. Does that make any sense? And let's be serious, is that what most data scientists are being hired to do?