Live data from Hacker News

Ask HN: What's the state of the job market in data science and machine learning?

news.ycombinator.com

61–70 of 140 posts

Re: Ask HN: What's the state of the job market in data science and machine learning?

#61
post #23

Speaking for NYC, but I imagine silicon valley is similar. The supply-demand dynamics have changed a lot in the last couple years. I'd roughly break it out into two groups: people with work experience + strong software development skills, and those without. The first group is in higher demand than ever, and tend to add a lot of value to companies that really need it. The second group has gotten extremely crowded, esp…

I am currently studying Bachelor CS an am interested in ML/DS. How can i make sure i fall in the first group? I try to do some side-projects to expand my skill set and practice software dev. skills, but this whole thread seems pretty discouraging.

Given you're strong in CS, study as much statistical theory as possible. That'd be my advice. You need to be a great programmer, but statistics is what's really in short supply.

Re: Ask HN: What's the state of the job market in data science and machine learning?

#62
post #59

We hire applied ML/AI specialists. For me it's not just an understanding of mathematical concepts, but also being able to apply new ideas to new problems. This depends quite a bit on critical thinking, a good fundamental ability to analyze a problem and understand its parameters, then manage the logical operations required to deliver the feature and solve the problem. As for why I think it's on HN every day: I also l…

It seems that you made it reverse of what you wanted. As far as I understand, data scientists should start the exploration and developers should finish exploitation.

Re: Ask HN: What's the state of the job market in data science and machine learning?

#63
post #23

Speaking for NYC, but I imagine silicon valley is similar. The supply-demand dynamics have changed a lot in the last couple years. I'd roughly break it out into two groups: people with work experience + strong software development skills, and those without. The first group is in higher demand than ever, and tend to add a lot of value to companies that really need it. The second group has gotten extremely crowded, esp…

I am currently studying Bachelor CS an am interested in ML/DS. How can i make sure i fall in the first group? I try to do some side-projects to expand my skill set and practice software dev. skills, but this whole thread seems pretty discouraging.

Depending on where you are studying you can take ML courses. However the easiest way is to get a Master's degree in CS from Stanford, GA Tech, Cornell, CMU, etc. Few Universities offer a Data Science Masters degree e.g. I think NYU, Columbia & Harvard. You can also try joining a comapny and then starting the degree program a year later.

As far as this thread, don't worry too much about it.

Re: Ask HN: What's the state of the job market in data science and machine learning?

#64

Earlier quoted context omitted.

Do you have advice for someone in the second camp? I have a master's in econ and I'm comfortable in R but I don't have good development skills. I'm wondering if I need to give up on data science jobs for a bit and try and find an entry level software development job

Take whatever job you can... as far as it's related to the kind of things you want to do. Forget about the big companies that everyone knows, forget about the high pay that appear in the newspapers. It's not real, it's not for you. Get an entry or junior level position and get experience. There are many unknown places that will take people only because they're cheap. That's where everyone started.

I'm looking to go through a DS boot camp this summer, have a master's in aerospace engineering. I'd be very content with mid-five figures (40% lower than what I made as an engineer) at a no-name company if it's in the city that I want (not SF). Are such lower-end data analyst positions in decent supply?

Re: Ask HN: What's the state of the job market in data science and machine learning?

#65
I am the Chief Data Scientist of Dice.com. If you are interested in working as a junior Data Scientist, and are smart and hard working, please apply here: http://careeropportunities.dhigroupinc.com/. The position is a telecommute role. We will absolutely consider people with no data science experience, so long as they demonstrate an aptitude for data science \ machine learning and can code.

Re: Ask HN: What's the state of the job market in data science and machine learning?

#66

In my limited experience, there's a difference between a data scientist who can process data given data and a set of questions about it, and a data scientist who can figure out what data you need, and the questions that need to be answered. I think making the transition from the first role to the second role comes with experience, both with the toolsets, and thinking about the problem as a whole.

> data scientist who can figure out what data you need, and the questions that need to be answered.

Isn't that describing a statistician?

Re: Ask HN: What's the state of the job market in data science and machine learning?

#67

Currently, Gartner analysts place ML at the "peak" of its Hype Cycle for Emerging Tech [0] with a runway of 2-5 years for mainstream adoption. [0] http://www.gartner.com/newsroom/id/3412017

that must be qualified because there are a lot of ML applications that have basically been adopted in the mainstream, like voice recognition (which is pretty good, probably in the 80-90% accuracy range and better for specialized contextual tasks). I remember driving for lyft and being able to input destination addresses by voice (2 years ago; had a Moto X where this was cutting edge) was a godsend and improved my cus…

> voice recognition ... probably in the 80-90% accuracy range

State of the art systems are far better than this. Microsoft recently published a paper with a 5.9% word error rate for conversational speech. Speech directed at computers/assistants is already in the high 90s, though I don't have a figure off the top of my head.

Re: Ask HN: What's the state of the job market in data science and machine learning?

#68
post #23

Speaking for NYC, but I imagine silicon valley is similar. The supply-demand dynamics have changed a lot in the last couple years. I'd roughly break it out into two groups: people with work experience + strong software development skills, and those without. The first group is in higher demand than ever, and tend to add a lot of value to companies that really need it. The second group has gotten extremely crowded, esp…

The problem is DS is really 2-3 different disciplines under one nebulous title. What you're describing is folks who are prototyping and productionizing models. That's definitely in short supply, but random STEM PhDs are in no way competitive for those roles unless they're coming from CS programs + have work experience in production engineering.

But that's by no means all of the DS field. There are lots of DS jobs where you're collecting and interpreting and communicating about complex data sets. An engineering mindset is occasionally helpful, but a bias towards building versus towards analyzing and writing can just as often be counter-productive. Not all problems are solved by systems; lots of problems are solved by better understanding the problem and then letting other specialists build the right solution.

The bootcamps have contributed to the problem by focusing so much on building things. The idea that you can go from an econ undergrad to being a self-sufficient member of a production ML team in 6-12 weeks is nuts. What's less nuts (and what I wish programs like Insight focused on) is taking people from having data skills in one domain and with one set of tools (e.g. logitudinal medical record data, stored in CSVs and handled in Stata) to another set of tools (billions of rows of event-based product data stored in a data warehouse, processed in R or Python). But instead the bootcamps behave like the missing skillset is the ability to make a predictive random forest model on some arbitrary data set and build an AWS web app around it. THAT job market definitely doesn't exist and is completely over-saturated.

But people who are smart communicators about data, can manipulate and make sense of massive data sets, can ask incisive questions about their data, and can use data to convince people of a complex argument are always going to have job opportunities, even if they're not production grade engineers. If that sounds like you, I'm hiring - hit me up on Twitter: @drewwww.

Re: Ask HN: What's the state of the job market in data science and machine learning?

#69
post #54

Earlier quoted context omitted.

How do you test for programming ability in an interview? Especially for people without a lot of work experience (e.g. graduating from college) One data engineering role asked me to implement k-means from scratch and one data science role asked me to do some algorithms whiteboarding. But beyond this, people just asked SQL. From the rest of this thread it doesn't look like people would consider that to be 'SWE ability'…

Set someone the interviewee a programming task. This is pretty standard for developer jobs, and is absolutely fantastic for people without work experience (as it gives them another way to probe their skill).

I mean, I did a bunch of take-home tests. But those were focused on doing an analysis on some dataset, focused on answering a question.

Should data scientists be expected to do more than that? Or are you expecting a certain level of code quality from them? Within a 48 hour time window?

Re: Ask HN: What's the state of the job market in data science and machine learning?

#70
post #5

I have only anecdotal experience (I live in Warsaw, but do contracts mostly for Poland, UK and US). General data science is in need. I can get contracts easily, I know that people looking for competent people need to wait; especially as it is a skill much harder to pick than, say, front-end web dev (unless someone starts from a highly quantitive background like physics, modelling in biology, etc). My general impressi…

How do you find contracts? I'm interested in doing contract data science work, but I don't know how to start finding interested potential clients.
Post reply on HN