Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The State of Data Science and Machine Learning

kaggle.com

Post reply on HN
Print view View original

21–30 of 43 posts

Re: The State of Data Science and Machine Learning

#21

Post by followmeon »

Next it'd be interesting to see Python 2k vs. Python 3+. My own experience tells me that the majority of top Kagglers still use Python 2k, despite Kaggle Kernels being Python 3+ exclusively.

I also am quite amazed with the predominant use of Logistic Regression. I wonder if that is less about interpretability / ease of engineering, and more about the barriers that data scientists face when using more complex methods: lack of data science talent, lack of management support, results not used by decision makers, limitations of tools.

If Kaggle results are anything to go by, all businesses that care about best performance on structured data, should be using a form of gradient boosting.

Re: The State of Data Science and Machine Learning

#22

Post by followmeon »

kmax12 wrote: post #13

In "What barriers are faced at work?", I really wish they broke down the "dirty data" response into more categories. In particular, I'd love to know if people are dealing with data quality issues, feature engineering issues, or something else all together. In my opinion, this is representative of the problems with data science tools today. There is so much focus on the machine learning algorithms rather than getting…

Dirty data is not as much as a problem for me than human-biased data. Dirty data engineering, like modeling, will soon be largely automated.

Let's say you are predicting store sales. You create a feature that holds the store sales of one year back. The feature works really well and you are happy with your evaluation. But you captured bias: The previous model the store used was "predict today's sales by looking at last year's sales". Store managers fitted their sales tactics to this model (when the model predicted too much sales, the store managers do their best to get rid of the surplus inventory, for instance: by adding discounts or moving the products to a more prominent spot in the store).

So in the end you end up with a model with good evaluation, but you actually have (over)fitted to previous policies/models. You have not created the best possible sales predictor. How to ever find this out, without a costly intimate deep-dive in the data and data generation processes?

Re: The State of Data Science and Machine Learning

#23

Post by apohn »

zebrafish wrote: post #17
apohn wrote: post #12

Earlier quoted context omitted.

>40% of people survey spend >1-2 hours per week searching for another job. Surprising given all companies complain about the difficulty in finding data scientists/machine learners. I've hired data scientists in the past. One thing I found is that a lot of interviewees want to talk about all the algorithms (e.g. Gradient Boosting) they've used and are not able to describe how they thought through the problems before t…

I agree with your assessment that a lot of times the business problems have been put into a form that lends itself to exploitation by machine learning. Sometimes a company has a lot of data that's actually useless. Most of the time, I've found that business people do not understand the value of data. Often I hear, "we have this data set, let's unleash the data scientist on this to tell us something." or "we have this…

>I spend a lot of my time explaining that there must first be a business objective, a key question, or hypothesis that can then be understood through data. I cannot take a haystack and find the needle that is interesting to you. And if I do find that needle, many times there are no resulting changes made to our strategy.

IMO a number of data science positions should be considered partly research positions. You are hiring somebody think critically about how to generate high value/impact from data. This includes exploring if there is a different way to think about a business problem than it has been formulated in the past. This may include defining and collecting data when you discover the existing (or non-existent) data isn't appropriate. As with any research, you'll sometimes realize the path you are on is wrong and a correction is needed.

The "find all the needles in this haystack" is a totally different worldview and throws a lot of critical thinking out the window. I think this really plays into the idea that an organization can hire a person who is going to do immediate "magic" with algorithms and zero effort beyond that. You can slice/dice and p-hack your way into a million thoughtless and useless "insights."

Re: The State of Data Science and Machine Learning

#24

Post by kevinnk »

followmeon wrote: post #22
kmax12 wrote: post #13

In "What barriers are faced at work?", I really wish they broke down the "dirty data" response into more categories. In particular, I'd love to know if people are dealing with data quality issues, feature engineering issues, or something else all together. In my opinion, this is representative of the problems with data science tools today. There is so much focus on the machine learning algorithms rather than getting…

Dirty data is not as much as a problem for me than human-biased data. Dirty data engineering, like modeling, will soon be largely automated. Let's say you are predicting store sales. You create a feature that holds the store sales of one year back. The feature works really well and you are happy with your evaluation. But you captured bias: The previous model the store used was "predict today's sales by looking at las…

If I understand you correctly, the way you'd address this is by using counterfactuals. See this course[1] for an overview and this paper[2] which talks about the bias problem in the context of movie recommendations.

[1] http://www.cs.cornell.edu/courses/cs7792/2016fa/ [2] http://www.cs.cornell.edu/people/tj/publications/schnabel_et...

Re: The State of Data Science and Machine Learning

#25

Post by apohn »

followmeon wrote: post #22
kmax12 wrote: post #13

In "What barriers are faced at work?", I really wish they broke down the "dirty data" response into more categories. In particular, I'd love to know if people are dealing with data quality issues, feature engineering issues, or something else all together. In my opinion, this is representative of the problems with data science tools today. There is so much focus on the machine learning algorithms rather than getting…

Dirty data is not as much as a problem for me than human-biased data. Dirty data engineering, like modeling, will soon be largely automated. Let's say you are predicting store sales. You create a feature that holds the store sales of one year back. The feature works really well and you are happy with your evaluation. But you captured bias: The previous model the store used was "predict today's sales by looking at las…

>Dirty data engineering, like modeling, will soon be largely automated.

I don't agree. For every modern tech company that collects data that lends itself to automated data cleaning, there's a 40+ year old company that defined what data to be collected in 1990, designed an "automated system" in 1995 and has been shoehorning improvements on that system since then.

At my last job I was given access to a database with 150+ tables with no data dictionary. The person who wrote the load process and ETL (the output was a lot of summaries) had left 10 years before and nobody truly understood how anything actually worked or the downstream dependencies. It took me a week of digging just to find out which of those 150 tables were just temp tables for one of the many queries that executed on that system.

It's going to be a while before somebody figures out how to clean that data automatically, or even find issues in that data. That's the reality of the world of data for many organizations.

Re: The State of Data Science and Machine Learning

#26

Post by antgoldbloom »

kmax12 wrote: post #13

In "What barriers are faced at work?", I really wish they broke down the "dirty data" response into more categories. In particular, I'd love to know if people are dealing with data quality issues, feature engineering issues, or something else all together. In my opinion, this is representative of the problems with data science tools today. There is so much focus on the machine learning algorithms rather than getting…

As the saying goes, 80% of data science is cleaning the data and 20% is complaining about cleaning the data.

Re: The State of Data Science and Machine Learning

#27

Post by godelski »

Anyone else find it weird that when you click "other" for gender that the data looks more like garbage?

I was trying to actually compare male and female salaries out of interest but have a hard time believing so many people earn Has anyone sorted through this data and tried to account for these factors? I'd be interested at the uncertainty and how the information was gathered.

Re: The State of Data Science and Machine Learning

#28

Post by radarsat1 »

kmax12 wrote: post #13

In "What barriers are faced at work?", I really wish they broke down the "dirty data" response into more categories. In particular, I'd love to know if people are dealing with data quality issues, feature engineering issues, or something else all together. In my opinion, this is representative of the problems with data science tools today. There is so much focus on the machine learning algorithms rather than getting…

Of course everyone agrees that "cleaning data" is difficult and boring, and it's always mentioned, but what I don't really understand is what kind of tools people expect for this beyond what are already available. E.g. pandas is pretty good at merging tables, re-ordering, finding doubles, filling or dropping unknowns etc. There are also tools for visualizing large amounts of data, look for outliers, etc. Beyond the basic tools it seems to me that each dataset requires decisions to be made that can't be automated. (e.g. do I drop or fill the unknowns?) I don't see how this could be improved, as every decision has a solid, semantic implication related to whatever is the overarching research question.

So statements like "getting data ready for the algorithms" seem kind of meaningless to me, in the sense of general methodologies. How could you possibly "get the data ready" without considering what it is, how it will be used, etc. How can it possibly be generalized to anything beyond the specific requirements of each problem instance?

I'm just really curious what you are imagining when you say that better tools are needed here.

Re: The State of Data Science and Machine Learning

#29

Post by kmax12 »

radarsat1 wrote: post #28
kmax12 wrote: post #13

In "What barriers are faced at work?", I really wish they broke down the "dirty data" response into more categories. In particular, I'd love to know if people are dealing with data quality issues, feature engineering issues, or something else all together. In my opinion, this is representative of the problems with data science tools today. There is so much focus on the machine learning algorithms rather than getting…

Of course everyone agrees that "cleaning data" is difficult and boring, and it's always mentioned, but what I don't really understand is what kind of tools people expect for this beyond what are already available. E.g. pandas is pretty good at merging tables, re-ordering, finding doubles, filling or dropping unknowns etc. There are also tools for visualizing large amounts of data, look for outliers, etc. Beyond the b…

I am the lead contributor of a python library called Featuretools[0]. It is intended to perform automated feature engineering on time-varying and multi-table datasets. We see it as bridging a gap between pandas and libraries for machine learning like scikit-learn. It doesn't handle data cleaning necessarily, but it does help get raw datasets ready for machine learning algorithms.

We have actually used it to compete on Kaggle without having to perform manual feature engineering with success.

[0] https://www.featuretools.com

Post reply on HN