I do not understand. Have never understood. "Data Science" is, surly, newspeak. The appropriate term, surly, is "statistics".
Data Science: Reality Doesn't Meet Expectations
11–20 of 168 posts
Re: Data Science: Reality Doesn't Meet Expectations
#12Great read. A lot of those problems are real, and some of those I’ve experienced myself. But I think at least some of them are related to the immaturity of the field. We’re only at the beginning of creating the tools and platforms to facilitate DS, making it more reproducible and easier to measure.
For example, I’m working on the tool to make data management easier and convert datasets into a structured representation. If you have experienced that you spend a lot of time on preparing and analyzing data, and it is tedious, please reach out to me michael at heartex.net, would love to get your feedback on the product we have built so far.
Re: Data Science: Reality Doesn't Meet Expectations
#13I do not understand. Have never understood. "Data Science" is, surly, newspeak. The appropriate term, surly, is "statistics".
Indeed, perhaps applied statistics or even data analysis.
It has always felt stupid calling myself a data scientist, but the term statistician has certain connotations that are not always relevant for the corporate context.
Re: Data Science: Reality Doesn't Meet Expectations
#14Teams being small, data being crummy, infra being hard, and yet expectations being high aren't so much complaints as the they are the job description. The point of data scientists and the related roles listed in the article are not to just churn out the fun stuff, but to wade through the institutional and technical muck and mire it takes to bring the fun stuff to bear on a relevant business problem and to communicate…
As somebody in an ML Engineering role, i.e. somebody who could be asked to either fix the logging infrastructure or build some models, I would have agreed with this.
But even in this day and age with ML being the new hotness, you will find people who are quite happy to work on infrastructure and don't have a huge amount of interest in training models themselves, and it is probably a lot easier to hire them than people who can do both, and you may get better results from actual specialists.
Re: Data Science: Reality Doesn't Meet Expectations
#15One issue might be that organizations subconsciously resist the data scientist, or more generally, the nerd in his/her attempt to take over decisions. If these decisions are invariably tied to the goals and careers of managers, how can the data scientist have a "seat at the table" in all but the most enlightened and technical companies? The disorganized state of data and infrastructure suits the ambitious manager well, who can just put in enough effort to find data to have their project greenlightened or to answer one specific question.
Progress may only come slowly, ideally through products bought from 3rd parties whose results are understood and controlled by management.
Re: Data Science: Reality Doesn't Meet Expectations
#16Great read. A lot of those problems are real, and some of those I’ve experienced myself. But I think at least some of them are related to the immaturity of the field. We’re only at the beginning of creating the tools and platforms to facilitate DS, making it more reproducible and easier to measure. For example, I’m working on the tool to make data management easier and convert datasets into a structured representatio…
> But I think at least some of them are related to the immaturity of the field.
I agree. More so, I sometimes feel that in the end the field will break up once things start settling down. Some roles will migrate more towards engineering, some will go back towards data analysis.
The expectation that a data scientist is a funnel that can turn anything into magical insights and tools can't last forever.
Re: Data Science: Reality Doesn't Meet Expectations
#17It really depends on the niche or the industry considered, though: I can happily say that I can do materials informatics from my basement at home now and much faster and better than as a cog in any lab anywhere in the world. Same for a great number of STEM applications, if you ask or follow high-level practitioners through conferences, journals and social media. The elephant in the room is the Intellectual Property generated through STEM applied data science, which is hot and even dangerous as you can see from superstars like OpenAI, DeepMind or politically-motivated aggregations.
Re: Data Science: Reality Doesn't Meet Expectations
#18it's a problem with tech in general. some things come over-hyped. and in the process people forget what's the actual problem to be solved because they fell in love with tools | tech. maybe the solution could easily be done in excel but then that's not sexy. I personally prefer to handle most parts in Python because of automation. writing functions in python is easier than writing functions in SQL or Excel(macros)
Re: Data Science: Reality Doesn't Meet Expectations
#19I wrote a blog post along similar lines in 2018 (https://minimaxir.com/2018/10/data-science-protips/ ); unfortunately, the industry hasn't changed much since then.
As noted in the submission, there's a lot of flexibility in what a "data scientist" is. Normally that's good and healthy for the industry. However, it contradicts a lot of optimistic bootcamps/Medium/YouTube videos, and many won't be prepared for the difference.
Re: Data Science: Reality Doesn't Meet Expectations
#20Coase: "If you torture the data long enough, it will confess to anything."
via https://www.reddit.com/r/QuotesPorn/comments/b76ujr/if_you_t...