Ask HN: Is the industry saturated with data scientists already? • Hacker News Board

Re: Ask HN: Is the industry saturated with data scientists already?

#11

Post by thedrexster » Fri, Dec 23, 2022, 12:45 AM UTC

ProjectArcturis wrote: post #9

Anecdotally, as a Principal Data Scientist, I'm getting a lot less recruiter spam than I did a year ago. Though I think that's probably something that's affecting everything in tech as the industry as a whole takes their foot off the gas. At the entry level, I'm sure there's more competition for fewer spots. I think the new generative AI models are absolute game-changers and will only get better. If I were starting o…

>

For one thing, there's a lot less recruiters than there were a year ago!

Re: Ask HN: Is the industry saturated with data scientists already?

#12

Post by GianFabien » Fri, Dec 23, 2022, 12:49 AM UTC

ffssffss wrote: post #5

I think it's less a case of saturation and more a case of companies realizing that most data scientists don't actually deliver value commensurate to the salaries they were asking. Most companies simply don't have enough data, or don't have hygienic enough data, or don't have the engineering heft to build a reliable data pipeline, so the data scientists often find themselves set up to fail. I've seen that happen a few…

In my experience it is common to meet managers who think they have far more data than is really the case. What data they do have tends to be inconsistent due to changes in processes and program changes. This is particularly gruesome when a migration between ERP systems had been undertaken or after a company merger or even reorganization.

It is impossible for a data scientist to reliably identify the data problems of a prospective employer during the interview stage. The biggest companies often have the messiest data. But you would need to talk to the IT staff in the trenches to discover that fact.

Re: Ask HN: Is the industry saturated with data scientists already?

#13

Post by mturmon » Fri, Dec 23, 2022, 12:51 AM UTC

sage76 wrote: post #7

MontyCarloHall wrote: post #4

To be blunt, the market is saturated with people who call themselves “data scientists” but are actually just reasonably skilled software engineers with at best a college sophomore level understanding of math/stats. On the other hand, the market is nowhere near saturation for people with both advanced software engineering and math/stats skills (i.e. PhD-level).

Just out of curiosity, is there a way for someone to show off those skills if they don't have a PhD?

I'm a data science hiring manager. Other things besides just the raw academic credential (a PhD which wouldn't typically be in "Data Science" anyway, i.e., even the PhD's are just Data-Science-adjacent) are publications, conference appearances/posters, generated data products/pipelines, and contributions to relevant software. (For me, in that approximate order.)

I'd work on building some of those, because you do need to stand out against the field @MonteCarloHall described. (Software engineer + undergrad math/stats)

Those kind of achievements would satisfy screening filters. Then of course you'd have to have knowledge to back that up. I think it's reasonable to say that typically this will be domain-specific, e.g. you would end up with a different background knowledge base for NLP than for spatiotemporal problems than for network/graph problem domains. With all the growth has come specialization.

Re: Ask HN: Is the industry saturated with data scientists already?

#14

Post by mdcds » Fri, Dec 23, 2022, 1:21 AM UTC

a side, but a sincere question: what do Data Scientists do and what can I expect one to produce as productive output?

I've worked as an SDE on data engineering projects myself (Spark / Hadoop stuff) and have friends who are ML researches and develop things like better recommendation results. Never met a data scientist.

Re: Ask HN: Is the industry saturated with data scientists already?

#15

Post by boredemployee » Fri, Dec 23, 2022, 1:25 AM UTC

ffssffss wrote: post #5

I think it's less a case of saturation and more a case of companies realizing that most data scientists don't actually deliver value commensurate to the salaries they were asking. Most companies simply don't have enough data, or don't have hygienic enough data, or don't have the engineering heft to build a reliable data pipeline, so the data scientists often find themselves set up to fail. I've seen that happen a few…

Well, where I work at, I was hired as a data scientist and end up doing everything related to data lol, engineering, analytics and what not. It doesn't matter as long as they pay me to do that.

Re: Ask HN: Is the industry saturated with data scientists already?

#16

Post by leplen » Fri, Dec 23, 2022, 1:59 AM UTC

Data science as a title has bifurcated several times. A lot of the engineering heavy DS roles are now data engineer or machine learning engineer. Probably those are the roles you should be looking for. Many of the DS jobs now are essentially analyst jobs, where SQL and stakeholder management is 75% of the role and writing code is just a bonus.

Data science isn't going away. Leadership is always going to need numbers explained to them, but DS roles have never been as numerous or as well paid level for level as software engineering.

Re: Ask HN: Is the industry saturated with data scientists already?

#17

Post by morelandjs » Fri, Dec 23, 2022, 2:19 AM UTC

MontyCarloHall wrote: post #4

To be blunt, the market is saturated with people who call themselves “data scientists” but are actually just reasonably skilled software engineers with at best a college sophomore level understanding of math/stats. On the other hand, the market is nowhere near saturation for people with both advanced software engineering and math/stats skills (i.e. PhD-level).

This is the truth. Model tinkering in Jupyter notebooks is heavily saturated and won’t get you very far now days.

If you know data engineering and a little devops as well as graduate level stats and linear algebra… the jobs are bountiful.

Re: Ask HN: Is the industry saturated with data scientists already?

#18

Post by sebg » Fri, Dec 23, 2022, 2:22 AM UTC

peatfreak wrote: post #8

sebg wrote: post #2

> Is the industry saturated with data scientists already? No :) > Did all the talk of data science being the "sexiest" job cause the market to become saturated No :) > is it still a viable career path? Yes Source: a) I Co-run the Data Science Weekly newsletter. b) I was a mod of https://www.reddit.com/r/datascience/ from 15k to 30k members and people were asking that about 5 to 6 years ago. The sub now has ~829k memb…

> a) I Co-run the Data Science Weekly newsletter. Is the newsletter still running? I looked at the archive on the website and I didn't see anything more recent than April 2022.

https://us3.campaign-archive.com/home/?u=71a2b2a38789d4d25b7...

We are moving platforms (email & hosting and some personal stuff happened related to a flood, so website archives haven't been updated yet).

Thanks for checking it out!

Re: Ask HN: Is the industry saturated with data scientists already?

#19

Post by [deleted] » Fri, Dec 23, 2022, 2:23 AM UTC

dkqmduems wrote: post #3

sebg wrote: post #2

> Is the industry saturated with data scientists already? No :) > Did all the talk of data science being the "sexiest" job cause the market to become saturated No :) > is it still a viable career path? Yes Source: a) I Co-run the Data Science Weekly newsletter. b) I was a mod of https://www.reddit.com/r/datascience/ from 15k to 30k members and people were asking that about 5 to 6 years ago. The sub now has ~829k memb…

This is a non answer.

[deleted]

Re: Ask HN: Is the industry saturated with data scientists already?

#20

Post by listenallyall » Fri, Dec 23, 2022, 2:42 AM UTC

mturmon wrote: post #13

sage76 wrote: post #7

Earlier quoted context omitted.

Just out of curiosity, is there a way for someone to show off those skills if they don't have a PhD?

I'm a data science hiring manager. Other things besides just the raw academic credential (a PhD which wouldn't typically be in "Data Science" anyway, i.e., even the PhD's are just Data-Science-adjacent) are publications, conference appearances/posters, generated data products/pipelines, and contributions to relevant software. (For me, in that approximate order.) I'd work on building some of those, because you do need…

> the market is nowhere near saturation

Need to show > publications, conference appearances/posters, generated data products/pipelines, and contributions to relevant software.

These appear to be in conflict. If companies need people with such skills, they can't just hope to get the elite few who present at conferences, they need to be hiring among the audience members as well. If they are in fact just hiring the speakers, then the jobs aren't really "bountiful"