Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data Science Is America’s Hottest Job

bloomberg.com

Post reply on HN
Print view View original

181–190 of 224 posts

Re: Data Science Is America’s Hottest Job

#181

Post by kayoone »

kitanata wrote: post #106
erw1 wrote: post #100

Earlier quoted context omitted.

What value does an advanced degree, in your opinion, bring to a data science position then? To hear you say it, it brings no value.

What value does a bachelor’s degree have if some kid can just learn ruby and JavaScript on his own and go make $100k/yr at some startup? Of course a degree has value but it is possible to provide value without one. This is especially true in 2018 when anyone can learn pretty much anything online, and download whatever papers they want online and read them. Your mistake is in thinking that somehow a degree makes you s…

Thats a pretty ignorant statement. I'd wager that there aren't many people who could/would put the same effort into self studying that would be required to pass a CS degree with decent grades. They also would probably not learn a lot of stuff that is not interesting to them, while CS students have no choice than to go through the materials. As an employer this also tells a story about who is taking the easy route vs working through a complete program over the span of years. Obviously there are exceptions, but when there are a lot of applicants for a position, it's just an easy filter for employers.

Re: Data Science Is America’s Hottest Job

#182

Post by lordnacho »

kitanata wrote: post #41

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone wanted a PhD or a masters degree. I have 15 years of experience in software. I’ve done everything to low level game programming and graphics programming to web development, to AI (but not as a specific position title). The gatekeeping in this field surprised…

Well to put it in a data science terms, the employers are looking for precision, at the expense of recall. They don't care that they don't find all the good data scientists so long as the people they do find are good data scientists.

I actually think that requiring an advanced degree doesn't help toward that end.

In the finance world we call these people quants. And actually in my experience having a phd does very little for someone; the critical skill required is software engineering.

Thinking about investment strategies is about testing hypotheses, and you can't test things properly if you don't know a few things about how to organize code. This is actually an insidious problem, because there's nobody telling you how to actually build an alpha generating strategy. And if you can't investigate properly, you fall into all the traps (you make excuses to do the following): too many features, choosing too small a sample that happens to do well, filtering the data in so may ways one of them is bound to "work", and so on.

Imagine that you're a chef, but you can't chop stuff effectively. You would then work around that limitation, maybe work on dishes where it's not needed, or perhaps get a junior guy to do the chopping. You might think this solves the problem, but actually it just swaps one problem for another, because now you need to communicate and coordinate with this other person. Or you don't explore that whole area of food with chopped stuff in it.

CI pipelines, version control (branching, diffs, etc), database maintenance, a bit of OS basics. All things that tended to differentiate the productive quants from those who merely thought they were useful. I've seen things done in a few weeks that others had spent years not achieving.

As for the ML skills themselves, you do need a bit of math to do it, and the math is relatively easy to learn. There's loads of materials to help you as well. What's not explained so much is certain philosophical issues around what is being examined. A course in economics has examples of these things: endogeneity, Lucas critique (which is Hume rehashed), experiment design (do the observations mean what you think?).

Re: Data Science Is America’s Hottest Job

#183

Post by stilley2 »

kitanata wrote: post #87
hellofunk wrote: post #83

Earlier quoted context omitted.

Of the skills necessary to succeed in data science, the ability to program is actually the least important. Perhaps you were over-emphasising your software skills at the expense of really demonstrating proper understanding of the science of data analysis. A data science position is not going to require your experience in low-level game programming, for example. In fact, being able to program at all is secondary, sinc…

This right here is a perfect example of why this position is gatekeeped so much. Having a PhD doesn’t automatically mean you think like a scientist of a mathematician, and having a bachelor’s degree and 15 years of experience writing code and then putting in the effort to read Elements of Statistical Learning, compeleting several online courses in machine learning and data science, studying probability, combinatorics…

Just FYI many (most?) people with STEM PhDs don't pay for it, but in fact get a stipend.

Re: Data Science Is America’s Hottest Job

#184

Post by randcraw »

mikert5671 wrote: post #58
kitanata wrote: post #41

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone wanted a PhD or a masters degree. I have 15 years of experience in software. I’ve done everything to low level game programming and graphics programming to web development, to AI (but not as a specific position title). The gatekeeping in this field surprised…

I actually think that machine learning can be much harder than you are claiming here, and that your inexperience is showing when you make such claims. Sure a toy problem from some dataset in a book or pulled off the internet is easy. So is implementing a k-means algorithm. But go to a large corporation with 100 datasets, all with 100 fields each, each having significant seasonal bias (among other biases), and build s…

So true. Missing from university textbooks is the fact that real world data is often dirty as hell, inconsistent, distorted, biased, incomplete, and/or just plain invalid (measured the wrong source or too imprecisely to be useful).

Unless your DS group is big enough to warrant hiring data engineers / cleaners, as a data scientist it'll be your job not only to eventually choose the algorithm, but foremost, to confirm that the data is sufficient to serve the intended purpose of mining it, ideally before you waste a lot of time curating it or paying for a raw data dump you can't use.

Re: Data Science Is America’s Hottest Job

#185

Post by _raoulcousins »

stilley2 wrote: post #183
kitanata wrote: post #87

Earlier quoted context omitted.

This right here is a perfect example of why this position is gatekeeped so much. Having a PhD doesn’t automatically mean you think like a scientist of a mathematician, and having a bachelor’s degree and 15 years of experience writing code and then putting in the effort to read Elements of Statistical Learning, compeleting several online courses in machine learning and data science, studying probability, combinatorics…

Just FYI many (most?) people with STEM PhDs don't pay for it, but in fact get a stipend.

To be fair, I didn't pay for my PhD, but I gave up the opportunity to earn more than a grad student's stipend for six years.

Re: Data Science Is America’s Hottest Job

#186

Post by sidlls »

reureu wrote: post #141
kitanata wrote: post #41

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone wanted a PhD or a masters degree. I have 15 years of experience in software. I’ve done everything to low level game programming and graphics programming to web development, to AI (but not as a specific position title). The gatekeeping in this field surprised…

In my experience working as a data scientist in the health tech field for the past decade (as an "epidemiologist" and "health informaticist" before "data scientist" was a title), I've found folks who just pick up data science from an engineering background often have a simplistic view of working with data generated from human subjects. My training heavily stressed bias and confounding, study design, problem specifica…

I've also seen multiple people build models that reflect the data collection instead of the biologic/medical process, and present it without even realizing how wrong their results are. The problems in these cases is often their "objective" measures of performance (e.g., precision, recall, accuracy, AUC, whatever) look pretty good, but they don't see that it's because the model AND data are both heavily influenced by this larger problem.

This is great. It's something I notice I have to mentor my junior data scientists and ML practitioners on with some regularity. Data science and machine learning aren't necessarily useful without some additional domain knowledge and discipline to recognize the need to view the data from many different perspectives and with many different relationships highlighted. Too often they see high P/R numbers and call the job done, or something similar.

Re: Data Science Is America’s Hottest Job

#187

Post by randcraw »

dominotw wrote: post #80
kitanata wrote: post #41

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone wanted a PhD or a masters degree. I have 15 years of experience in software. I’ve done everything to low level game programming and graphics programming to web development, to AI (but not as a specific position title). The gatekeeping in this field surprised…

I think it might be a easier path if you apply for "data engineering" jobs and convince internally to give you a chance at data science. Get data science on your resume and look for next job.

Agreed. A DE job is based on more traditional software skills, expecting you to understand the priorities and purposes of an given ML technique, without having to choose, architect, or defend the choice of that technique over alternatives. I can see most of the OP's objections as being more appropriate to the superciliousness of hiring software PhDs into DE jobs rather than DS. The DS role is more quantitative and theoretical (befitting a PhD), while the DE is more an engineer and experimentalist (befitting a less academic mindset).

Re: Data Science Is America’s Hottest Job

#188

Post by jriot »

kitanata wrote: post #87
hellofunk wrote: post #83

Earlier quoted context omitted.

Of the skills necessary to succeed in data science, the ability to program is actually the least important. Perhaps you were over-emphasising your software skills at the expense of really demonstrating proper understanding of the science of data analysis. A data science position is not going to require your experience in low-level game programming, for example. In fact, being able to program at all is secondary, sinc…

This right here is a perfect example of why this position is gatekeeped so much. Having a PhD doesn’t automatically mean you think like a scientist of a mathematician, and having a bachelor’s degree and 15 years of experience writing code and then putting in the effort to read Elements of Statistical Learning, compeleting several online courses in machine learning and data science, studying probability, combinatorics…

Every job is gatekeeping. You chose a path in life and it wasn't data analysis. Why would anyone interview you when there are 100's of other people that have the actual skill-set they are looking for?

Re: Data Science Is America’s Hottest Job

#189

Post by aje403 »

whataretensors wrote: post #160
mikert5671 wrote: post #67

Earlier quoted context omitted.

this is 100% wrong

That's not a rebuttal. It's absolutely non-technical gatekeepers looking for certificates. Graduates get mad because they realize they've wasted a bunch of money on something they can learn for free, so they reject the notion and make others go through the same abusive system.

Do you have any proof that non technical managers are setting the hiring bar and writing the requirements for data science positions at most companies, which is what it seriously sounds like you’re implying. The guy above you was joking
Post reply on HN