We’re in the Middle of a Data Engineering Talent Shortage
blog.stitchdata.com
We’re in the Middle of a Data Engineering Talent Shortage
1–10 of 166 posts
Re: We’re in the Middle of a Data Engineering Talent Shortage
#2I'm trying to switch careers into "Data Engineering" now, as a full stack developer who is more interested in ML, and I've found almost no traction internally at my company or externally. It looks like I may just accept a full stack position at a good company that does a lot of data science for now, but though I would ask - Where are all these jobs?
Re: We’re in the Middle of a Data Engineering Talent Shortage
#3anything and everything is marketed as "data science" and "data engineering" these days becasue this is the buzzword of the day.
I've been dealing with large data even before "big data" was a word but i dont call myself "data scientist" or "data engineer". I am still a software engineer working on what benefits my organization.
"Serial Entrepreneur" is the same these days, claimed by anyone who had a lemonade stand as a kid.
Re: We’re in the Middle of a Data Engineering Talent Shortage
#4Ahh the ol' write a post about a not well understood distinction and then proceed to not explain the distinction.
Looks like we need more English engineers too.
Re: We’re in the Middle of a Data Engineering Talent Shortage
#5I'm trying to switch careers into "Data Engineering" now, as a full stack developer who is more interested in ML, and I've found almost no traction internally at my company or externally. It looks like I may just accept a full stack position at a good company that does a lot of data science for now, but though I would ask - Where are all these jobs?
"Data Engineering" is most of the work that needs to be done, but I think companies haven't identified it as a category.
From my P.O.V., "Full Stack Engineer" is a place you don't want to be because it means putting out fires with whatever junk javascript is in the front end. It seems like everybody who's built a serious javascript application has invented their own Virtual DOM because none of the popular Virtual DOM libraries are good for much other than wasting time and CPU cycles.
"Data Scientist" is a bad title in it's own way, in the sense that "Computer Science" is bad, but worse. To a lot of people there is a Brahmin kind of attitude associated with "Scientist" -- i.e. an aversion to getting your hands dirty. Real world data is pretty dirty and you aren't going to get far in getting value out of it unless you spend 80-90% of your time dealing with the dirt.
Re: We’re in the Middle of a Data Engineering Talent Shortage
#6I'm trying to switch careers into "Data Engineering" now, as a full stack developer who is more interested in ML, and I've found almost no traction internally at my company or externally. It looks like I may just accept a full stack position at a good company that does a lot of data science for now, but though I would ask - Where are all these jobs?
We (Kaggle) run a data science jobs board (https://www.kaggle.com/jobs) that gets a few data engineer listings from time to time. Not all of these are active, but you may find a few interested companies via - https://www.google.com/#q=site:https://www.kaggle.com/jobs+%...
Re: We’re in the Middle of a Data Engineering Talent Shortage
#7I'm trying to switch careers into "Data Engineering" now, as a full stack developer who is more interested in ML, and I've found almost no traction internally at my company or externally. It looks like I may just accept a full stack position at a good company that does a lot of data science for now, but though I would ask - Where are all these jobs?
ML falls more under a Data Science role than Data Engineering, although ML is much more difficult without proper Data Engineering.
Re: We’re in the Middle of a Data Engineering Talent Shortage
#8I'm trying to switch careers into "Data Engineering" now, as a full stack developer who is more interested in ML, and I've found almost no traction internally at my company or externally. It looks like I may just accept a full stack position at a good company that does a lot of data science for now, but though I would ask - Where are all these jobs?
I see tons of them. If you're interested in ML, you're probably more looking towards data science. Data engineering (in general) is more about getting the data in a state where it can be used (extracted, cleaned, moved, transformed, etc.) at least from what i've commonly seen in the industry. A decent breakdown is here: https://blog.insightdatascience.com/data-science-vs-data-eng...
Re: We’re in the Middle of a Data Engineering Talent Shortage
#9I'm trying to switch careers into "Data Engineering" now, as a full stack developer who is more interested in ML, and I've found almost no traction internally at my company or externally. It looks like I may just accept a full stack position at a good company that does a lot of data science for now, but though I would ask - Where are all these jobs?
You might want to look at "Machine Learning Engineer" positions if you want to do ML in practice, it's starting to be a title I see somewhat often now.
As others have pointed out Data Engineering is more about building data pipelines, making architecture decisions for your ML stack, things like that. Less about model building, prototyping and training, which is what I think of when somebody says they 'do' ML.
Re: We’re in the Middle of a Data Engineering Talent Shortage
#10We surely need data mechanics.