My belief in an AI breakthrough is so strong that I would invite another AI winter to try to play catch up
Deep learning job postings have collapsed in the past six months
121–130 of 274 posts
Re: Deep learning job postings have collapsed in the past six months
#122Earlier quoted context omitted.
Everyone wants to fire up Tensorflow, Keras and PyTorch these days. Fewer people want to work in Airflow and SSIS, spend days tuning ETL, etc. This is the domain of data engineering, which bridges software engineering and data science with a dash of devops. I’ve been working in this field for a couple of years and it’s clear to me that data engineering is a necessary foundation and impact multiplier for data science.
Don't forget data cleaning. A huge issue I've seen is just getting sufficient data of a high enough quality. Also, (for supervised classification problems) labelling is a big problem. It is almost as if we need a "data janitor" title.
Phht you don't want to call it data janitor; no-one good will want that title. At least call it Data Integrity Engineer or something reasonably high-status.
Re: Deep learning job postings have collapsed in the past six months
#123Is there a LinkedIn tool that allows you to make similar trend plots as shown in the Twitter thread, or has the author been archiving the data over time?
Re: Deep learning job postings have collapsed in the past six months
#124I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…
I also witnesses this first hand at a Biotech company I worked at... we were using many variants of machine learning algorithms to develop predictive models of cell culture and separation processes. Problem is... the models have so many parameters in order to get a useful fit that the same model can also fit a carrot or an elephant. We found that dynamic parameter estimation on ODE/DAE/PDE system models, while harder…
Feature engineering is a big part of ML. If you know something about the process you should incorporate that..
Re: Deep learning job postings have collapsed in the past six months
#125Is that a worldwide trend, or is it based on US data? That's not clearly stated in the tweet.
Re: Deep learning job postings have collapsed in the past six months
#126My belief in an AI breakthrough is so strong that I would invite another AI winter to try to play catch up
What is your belief based on?
Re: Deep learning job postings have collapsed in the past six months
#127Earlier quoted context omitted.
> they paid more to get those insights than they were worth! This understates how awful ML is at many of these companies. I've seen quite a few companies that rushed to hire teams of people with a PhD in anything that barely made it through a DS/ML boot camp. To prove that they're super smart ML researchers without fail these hires rush to deploy a 3+ layer MLP to solve a problem that need at most a simple regression…
I worked in a place full of deep learning PhDs, and you'd have people trying to apply reinforcement learning to problems that had known mathematical solutions, and integer programming problems. I don't think the issue is just that companies hire people who are awful at ML, it's also that people are trying to shoehorn deep learning into everything, even when it currently has nothing to offer and we have better solutio…
How is this any different to developers who insist on using some shiny new web framework, micro service spaghetti and kubernates overkill infrastructure for their silly little CRUD app?
Re: Deep learning job postings have collapsed in the past six months
#128I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…
"Like Big Data, I think we'll see a few companies execute well and actually get some value, while most will just jump to the next shiny thing in a year or two." Here's another aspect - in many places nobody listens to the actual people doing the work. In my last job I was hired to lead a Data Science team and to help the company get value of Stats/ML/AI/DL/Buzzword. And I (and my team) were promptly overridden on eve…
I think the only places where it yields consistent results is organizations that have at least 80% of their staff doing the ML/DS work and less than 20% managing the people doing the work (up and down in the organization.)
Re: Deep learning job postings have collapsed in the past six months
#129Something I've learned: when non-engineers ask for an AI or ML implementation, they almost certainly don't understand the difference between that and an "algorithmic" solution. If you solve "trending products" by building a SQL statement that e.g. selects items with the largest increase of purchases this month in comparison to the same month a year ago, that's still "AI" to them. Knowing this can save you a lot of wa…
Any sufficiently misunderstood algorithm is indistinguishable from AI.
Re: Deep learning job postings have collapsed in the past six months
#130Earlier quoted context omitted.
> they paid more to get those insights than they were worth! This understates how awful ML is at many of these companies. I've seen quite a few companies that rushed to hire teams of people with a PhD in anything that barely made it through a DS/ML boot camp. To prove that they're super smart ML researchers without fail these hires rush to deploy a 3+ layer MLP to solve a problem that need at most a simple regression…
I worked in a place full of deep learning PhDs, and you'd have people trying to apply reinforcement learning to problems that had known mathematical solutions, and integer programming problems. I don't think the issue is just that companies hire people who are awful at ML, it's also that people are trying to shoehorn deep learning into everything, even when it currently has nothing to offer and we have better solutio…
Unfortunately, this is where the incentives of the company and that of the employee diverges. For the employee, if they choose some simpler, appropriate model or solution to the problem, they will not be able to get that next DL job. Especially early in their career. I cannot bring myself to do resume driven development, but I understand why people do it.