This is true, that getting and cleaning data is an important part of a workflow in data science and it's a necessary skill to have. But I don't believe anyone ever claimed Kaggle is enough to become a good data scientist. That's like studying algorithms is very useful, but who said it's enough to become good at software engineering? The skill consist of multiple parts which can be practiced separately or in combination in a real project where you do everything from scratch.
Machine learning isn't Kaggle competitions
11–20 of 24 posts
Re: Machine learning isn't Kaggle competitions
#12Re: Machine learning isn't Kaggle competitions
#13Hm, looks like this argument about Kaggle pops up all the time. This is true, that getting and cleaning data is an important part of a workflow in data science and it's a necessary skill to have. But I don't believe anyone ever claimed Kaggle is enough to become a good data scientist. That's like studying algorithms is very useful, but who said it's enough to become good at software engineering? The skill consist of…
In real life, there is a feedback loop from modelling, to data processing. In Kaggle all you can do is improve your model. You can't go back and change how data was processed, or collect different sorts of data.
Also, some flows simply don't fit into Kaggle's train/test paradigm. E.g. suppose you have an online algorithm that continuously updates parameters. Or situations where the model is used to generate the train/test data.
Finally some situations may have stringent computational requirements, e.g. <100ms to classify a single instance.
Re: Machine learning isn't Kaggle competitions
#14"Once I played against an NBA player and I couldn't do anything against him. I felt sad and demoralized. I still don't think I could win against an NBA player, but everyday I do (among other things) play basketball with my friends! And, you know, there're so many problems in basketball aside of dribbling and stuff: you have to find space to play with your friends, you have to convince the guy that doesn't want to pla…
Re: Machine learning isn't Kaggle competitions
#15Re: Machine learning isn't Kaggle competitions
#16 I used an out-of-the-box algorithm, messed around a bit,
and definitely did not make the leaderboard.
Because that is not Kaggle competitions. Nearly everyone on the leaderboard is proficient in data analysis and machine learning. They all tried that out-of-the-box algorithm for their attempt. But they did not give up so easily. Understand the business problem
If you want to predict flight arrival times, what are
you really trying to do?
This is not different from Kaggle competitions, this is a tip for performing better in Kaggle competitions. See also the GE Flight Quest: https://www.gequest.com/c/flight Those winners used industry-standard machine learning, optimization techniques, but also creative insights and hunches, like tweaking the target labels:"A next step is to ask, “What should I actually be predicting?”. This is an important step that is often missed by many – they just throw the raw dependent variable into their favorite algorithm and hope for the best. But sometimes you want to create a derived dependent variable. I’ll use the GE Flight Quest as an example: you don't want to predict the actual time the airplane will land; you want to predict the length of the flight; and maybe the best way to do that is to use the ratio of how long the flight actually was to how long it was originally estimated to be and then multiply that times the original estimate." - Steve Donoho - http://blog.kaggle.com/2014/08/01/learning-from-the-best/
Furthermore, it is entirely clear to everyone that data science in a business setting and in a competitive sport setting is different. To say they are equal, would be to say something like: paintball is equal to being in the military. But to say that Kaggle is not machine learning is to say: paintball requires no marksmanship.
There are some very messy, unwieldy datasets on Kaggle right now. For example the Seizure Detection challenge has many GBs of raw sensor data, from just a few patients. This would require a competitor to clean, understand problem domain, understand evaluation metrics, measure cross validation and put your model in production on your laptop in the evening hours.
The author of that blogpost is invited to team up, with me or others. Let's see if we can use machine learning to improve some pressing issues. I'd also love it if Stripe can host a contest on Kaggle.
Re: Machine learning isn't Kaggle competitions
#17Hm, looks like this argument about Kaggle pops up all the time. This is true, that getting and cleaning data is an important part of a workflow in data science and it's a necessary skill to have. But I don't believe anyone ever claimed Kaggle is enough to become a good data scientist. That's like studying algorithms is very useful, but who said it's enough to become good at software engineering? The skill consist of…
It's not as straightforward as Kaggle representing one stage in a pipeline. Kaggle also fixes this stage in an unnatural way. In real life, there is a feedback loop from modelling, to data processing. In Kaggle all you can do is improve your model. You can't go back and change how data was processed, or collect different sorts of data. Also, some flows simply don't fit into Kaggle's train/test paradigm. E.g. suppose…
The thing is Kaggle helps developing certain skills. It doesn't mean it helps you developing all necessary skills for your "real life". Your real life and your job may require unique combination of skills and the only way to fully prepare for it is to actually do real project that are specific to your job.
Re: Machine learning isn't Kaggle competitions
#18"Once I played against an NBA player and I couldn't do anything against him. I felt sad and demoralized. I still don't think I could win against an NBA player, but everyday I do (among other things) play basketball with my friends! And, you know, there're so many problems in basketball aside of dribbling and stuff: you have to find space to play with your friends, you have to convince the guy that doesn't want to pla…
To me it sounded more like "the problems they do at Mathematical Olympiads are not the real mathematics done at academia/the enterprise," which turns out to be quite true in many ways.
I've worked at Google for nearly 10 years -- not sure I can solve the Google code Jams :) Or at least I haven't been motivated to do so. I know all my basic data structures and algorithms, but the questions don't seem that motivating.
Re: Machine learning isn't Kaggle competitions
#19Why Machine Learning is Kaggle competitions. I used an out-of-the-box algorithm, messed around a bit, and definitely did not make the leaderboard. Because that is not Kaggle competitions. Nearly everyone on the leaderboard is proficient in data analysis and machine learning. They all tried that out-of-the-box algorithm for their attempt. But they did not give up so easily. Understand the business problem If you want…
I think the most legitimate knock against Kaggle is that in many business settings there probably isn't much value in improving that extra .00001 (but obviously there are exceptions).
Anyway, I think that that even someone very experienced in machine learning would learn something doing a Kaggle competition, especially if the competition is in an area outside their core expertise.
Re: Machine learning isn't Kaggle competitions
#20"Once I played against an NBA player and I couldn't do anything against him. I felt sad and demoralized. I still don't think I could win against an NBA player, but everyday I do (among other things) play basketball with my friends! And, you know, there're so many problems in basketball aside of dribbling and stuff: you have to find space to play with your friends, you have to convince the guy that doesn't want to pla…
To me it sounded more like "the problems they do at Mathematical Olympiads are not the real mathematics done at academia/the enterprise," which turns out to be quite true in many ways.
Yes, that is true that "pure something" is almost always significantly different from "something in the real life". That's pretty obvious, actually. I guess that there's virtually nothing in the "real life" that would consist from only one "core" activity. In fact, it's the opposite: you make some activity "pure" in order to master it, because in the real life there's so much that distracts you from mastering that one essential skill, whatever would it be. Plubmers and painters (and almost everyone else) must to be able to talk to others and to solve totally human-related problems in order to do their job. Person you would call sniper actually does much more running, crawling, waiting and hiding than shooting. Or does it really surprise anyone that policemen do tons of paperwork every day and not so much of that stuff they show in the movies? And somewhat opposite example: it's obvious that cook's work isn't only slicing vegetables, but have you ever seen how any real cook handles his knife? Please do, it's sight to behold. There's nothing "pure" in this world, no surprise here.
So if it would be Yan LeCun writing something like this it would be justifiable (although I would be rather surprised to hear that), but in this particular case it's more like saying "Kaggle isn't important, data science is me!", in which case I find that NBA example to be OK.
To summarise: "data science is more than building a model" — yes, "Machine learning isn't Kaggle competitions" — no.