Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What's the state of the job market in data science and machine learning?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 140 posts

Re: Ask HN: What's the state of the job market in data science and machine learning?

#51

Post by cardine »

rm999 wrote: post #23

Speaking for NYC, but I imagine silicon valley is similar. The supply-demand dynamics have changed a lot in the last couple years. I'd roughly break it out into two groups: people with work experience + strong software development skills, and those without. The first group is in higher demand than ever, and tend to add a lot of value to companies that really need it. The second group has gotten extremely crowded, esp…

Even outside Silicon Valley this is very similar. When I post a programming job I get more applicants from data scientists than from actual programmers. When I post a marketing job I get more applicants from data scientists than from actual marketers. It is clear to me that supply has far outpaced demand.

Re: Ask HN: What's the state of the job market in data science and machine learning?

#52

Post by dnautics »

cardine wrote: post #51
rm999 wrote: post #23

Speaking for NYC, but I imagine silicon valley is similar. The supply-demand dynamics have changed a lot in the last couple years. I'd roughly break it out into two groups: people with work experience + strong software development skills, and those without. The first group is in higher demand than ever, and tend to add a lot of value to companies that really need it. The second group has gotten extremely crowded, esp…

Even outside Silicon Valley this is very similar. When I post a programming job I get more applicants from data scientists than from actual programmers. When I post a marketing job I get more applicants from data scientists than from actual marketers. It is clear to me that supply has far outpaced demand.

I'm getting burned because I have a STEM PhD and am a competent programmer (have been programming on the side for decades) and am looking for a not-necessarly-data-science job, but it seems like all the places I'm applying to want to bin me in the "data-science" category.

Re: Ask HN: What's the state of the job market in data science and machine learning?

#53

Post by dnautics »

caminante wrote: post #3

Currently, Gartner analysts place ML at the "peak" of its Hype Cycle for Emerging Tech [0] with a runway of 2-5 years for mainstream adoption. [0] http://www.gartner.com/newsroom/id/3412017

that must be qualified because there are a lot of ML applications that have basically been adopted in the mainstream, like voice recognition (which is pretty good, probably in the 80-90% accuracy range and better for specialized contextual tasks). I remember driving for lyft and being able to input destination addresses by voice (2 years ago; had a Moto X where this was cutting edge) was a godsend and improved my customer service ratings.

Re: Ask HN: What's the state of the job market in data science and machine learning?

#54

Post by huac »

rm999 wrote: post #23

Speaking for NYC, but I imagine silicon valley is similar. The supply-demand dynamics have changed a lot in the last couple years. I'd roughly break it out into two groups: people with work experience + strong software development skills, and those without. The first group is in higher demand than ever, and tend to add a lot of value to companies that really need it. The second group has gotten extremely crowded, esp…

How do you test for programming ability in an interview? Especially for people without a lot of work experience (e.g. graduating from college)

One data engineering role asked me to implement k-means from scratch and one data science role asked me to do some algorithms whiteboarding. But beyond this, people just asked SQL. From the rest of this thread it doesn't look like people would consider that to be 'SWE ability'.

Re: Ask HN: What's the state of the job market in data science and machine learning?

#55

Post by LeanderK »

rm999 wrote: post #23

Speaking for NYC, but I imagine silicon valley is similar. The supply-demand dynamics have changed a lot in the last couple years. I'd roughly break it out into two groups: people with work experience + strong software development skills, and those without. The first group is in higher demand than ever, and tend to add a lot of value to companies that really need it. The second group has gotten extremely crowded, esp…

I am currently studying Bachelor CS an am interested in ML/DS. How can i make sure i fall in the first group? I try to do some side-projects to expand my skill set and practice software dev. skills, but this whole thread seems pretty discouraging.

Re: Ask HN: What's the state of the job market in data science and machine learning?

#56

Post by czep »

shas3 wrote: post #36
rm999 wrote: post #23

Speaking for NYC, but I imagine silicon valley is similar. The supply-demand dynamics have changed a lot in the last couple years. I'd roughly break it out into two groups: people with work experience + strong software development skills, and those without. The first group is in higher demand than ever, and tend to add a lot of value to companies that really need it. The second group has gotten extremely crowded, esp…

Frankly, MOOC-type machine learning- being able to do plain vanilla logistic regression or black-box deep learning techniques is not enough to get a job. This knowledge and experience has to be paired with one or more strengths: excellence in programming, grad-level math/stat/numerical skills/theoretical machine learning, domain specific expertise or experience (e.g. vision, audio, natural language, networks, geophys…

> plain vanilla logistic regression

Eh? It's not as easy as it sounds. Search for "mlelr" for a rather detailed illustration of how to code LR in C with only standard libraries. Now that was some fun to put together!

Re: Ask HN: What's the state of the job market in data science and machine learning?

#57

Post by shas3 »

czep wrote: post #56
shas3 wrote: post #36

Earlier quoted context omitted.

Frankly, MOOC-type machine learning- being able to do plain vanilla logistic regression or black-box deep learning techniques is not enough to get a job. This knowledge and experience has to be paired with one or more strengths: excellence in programming, grad-level math/stat/numerical skills/theoretical machine learning, domain specific expertise or experience (e.g. vision, audio, natural language, networks, geophys…

> plain vanilla logistic regression Eh? It's not as easy as it sounds. Search for "mlelr" for a rather detailed illustration of how to code LR in C with only standard libraries. Now that was some fun to put together!

But that does mean that you have pretty good C programming skills! I think your comment reinforces my claim about pairing MOOC-depth ML with other skills.

(Edit: I'm not saying MOOC-depth derisively. Serious learners and autodidacts can go pretty deep with just MOOCs for guidance. I just mean prima facie, based on content of some of the lighter and more popular MOOCs like Andrew Ng's. Abu-Mostafa's on EdX is meatier.)

Re: Ask HN: What's the state of the job market in data science and machine learning?

#58

Post by sigstoat »

user5994461 wrote: post #31
pcsanwald wrote: post #29

Earlier quoted context omitted.

I've only been hiring DS folks since 2012, but my experience matches what you've said exactly. The biggest differentiator I've seen is to be able to participate in actually building production quality systems vs being proficient enough in R or python to hack together a prototype on a very small dataset. The former kind of data scientists were very successful at our company, the latter, not so much. Both categories I…

That sounds weird to me. Does American PhD don't have to work for a few years at real companies as part of the PhD curriculum?

no. it's even a stereotype that they've never worked at a company. where is it required that PhDs work at companies to receive a degree?

Re: Ask HN: What's the state of the job market in data science and machine learning?

#59

Post by lowglow »

We hire applied ML/AI specialists. For me it's not just an understanding of mathematical concepts, but also being able to apply new ideas to new problems.

This depends quite a bit on critical thinking, a good fundamental ability to analyze a problem and understand its parameters, then manage the logical operations required to deliver the feature and solve the problem.

As for why I think it's on HN every day: I also like to think of an innovation pipeline happening something like this:

     [---------explore------|----------exploit-----------]
  ,->developers -> engineers/scientists -> data scientists->--,
 /----------
We're now in some sort of refinement cycle of innovation, where the current medium has been saturated on some level and there is a lot of push to mine value from the discoveries.

Re: Ask HN: What's the state of the job market in data science and machine learning?

#60

Post by nicoburns »

huac wrote: post #54
rm999 wrote: post #23

Speaking for NYC, but I imagine silicon valley is similar. The supply-demand dynamics have changed a lot in the last couple years. I'd roughly break it out into two groups: people with work experience + strong software development skills, and those without. The first group is in higher demand than ever, and tend to add a lot of value to companies that really need it. The second group has gotten extremely crowded, esp…

How do you test for programming ability in an interview? Especially for people without a lot of work experience (e.g. graduating from college) One data engineering role asked me to implement k-means from scratch and one data science role asked me to do some algorithms whiteboarding. But beyond this, people just asked SQL. From the rest of this thread it doesn't look like people would consider that to be 'SWE ability'…

Set someone the interviewee a programming task. This is pretty standard for developer jobs, and is absolutely fantastic for people without work experience (as it gives them another way to probe their skill).
Post reply on HN