Something I've learned: when non-engineers ask for an AI or ML implementation, they almost certainly don't understand the difference between that and an "algorithmic" solution. If you solve "trending products" by building a SQL statement that e.g. selects items with the largest increase of purchases this month in comparison to the same month a year ago, that's still "AI" to them. Knowing this can save you a lot of wa…
Deep learning job postings have collapsed in the past six months
151–160 of 274 posts
Re: Deep learning job postings have collapsed in the past six months
#152According to data from Revealera.com, if you normalize the data, the % of job openings that mention 'deep learning' has actually remained stable YoY: https://i.imgur.com/sDoKwD0.png * Revealera.com crawls job openings from over 10,000 company websites and analyzes them for technology trends for hedge funds.
Re: Deep learning job postings have collapsed in the past six months
#153Earlier quoted context omitted.
I think if a business is set up to scale by volume they can see gains from it. For example, say a business is already doing well at 100k conversions a day. They manage to apply "big data/ML" to optimize those conversions and gain a 3% lift, they are now making over a 1,095,000 extra conversions a year they would not have otherwise made.
So they need to make $1 profit for each of those conversions just to make it worth if they hire 1 ML scientist for 95k/year. Or $10 if they hire 10 for 950k/year in total. And so on... And there‘s the point where - IMHO - 3% gain may not be profitable enough.
Re: Deep learning job postings have collapsed in the past six months
#154Earlier quoted context omitted.
> they paid more to get those insights than they were worth! This understates how awful ML is at many of these companies. I've seen quite a few companies that rushed to hire teams of people with a PhD in anything that barely made it through a DS/ML boot camp. To prove that they're super smart ML researchers without fail these hires rush to deploy a 3+ layer MLP to solve a problem that need at most a simple regression…
My sense is that the original sin here is conflating data science with machine learning. A good data scientist might choose to use machine learning to accomplish their job. Or they might find that classical statistical inference is the better tool for the task at hand. A good data scientist, having built this model, might choose to put it into production. Or they might find that a simple if-statement could do the job…
Re: Deep learning job postings have collapsed in the past six months
#155Earlier quoted context omitted.
Don't forget data cleaning. A huge issue I've seen is just getting sufficient data of a high enough quality. Also, (for supervised classification problems) labelling is a big problem. It is almost as if we need a "data janitor" title.
Phht you don't want to call it data janitor; no-one good will want that title. At least call it Data Integrity Engineer or something reasonably high-status.
Re: Deep learning job postings have collapsed in the past six months
#156Earlier quoted context omitted.
This is just my general sense, as a very non-expert with more experience of doing than theory...but the benefit is someone knowing the theory AND being able to translate that into revenue. I think most people view the hard part as doing the PHd, and so lots of people value that experience, and because they have that experience you have this endowment effect: wow, that PHd was hard, I must do very hard and complex thi…
The Man Group has had respectable returns, especially during Coronavirus. Nothing amazing, but certainly not terrible. Regardless, there's more to the picture: Sharpe ratio, vol, correlation to the market, etc
And again, the key point was: they have had this institute for how long? Decade plus? Are they a leading quant fund? No. Are they in the top 10? No. Are they doing anything particularly inventive? See returns. No.
Re: Deep learning job postings have collapsed in the past six months
#157In conjunction with this, it turns out 99% of the problems the customer is facing, despite their belief to the contrary, aren't solved best with ML, but with good old fashioned engineering.
In cases where the problem can be approached either way, the ML approach typically takes much longer, is much harder to accomplish, has more engineering challenges to get it into production, and the early ramp-up stages around data collecting, cleaning and labeling are often almost impossible to surmount.
All that being said, there are some things that are only really solvable with some ML techniques, and that's where the discipline shines.
One final challenge is that a lot of data scientists and ML people seem to think that if it's not being solved using a standard ML or DL algorithm then it isn't ML, even if it has all of the characteristics of being one. The gatekeeping in the field is horrendous and I suspect it comes from people who don't have strong CS backgrounds wrapping themselves too tightly against their hard-earned knowledge rather than having an expansive view of what can solve these problems.
Re: Deep learning job postings have collapsed in the past six months
#158Earlier quoted context omitted.
Don't forget data cleaning. A huge issue I've seen is just getting sufficient data of a high enough quality. Also, (for supervised classification problems) labelling is a big problem. It is almost as if we need a "data janitor" title.
Phht you don't want to call it data janitor; no-one good will want that title. At least call it Data Integrity Engineer or something reasonably high-status.
Re: Deep learning job postings have collapsed in the past six months
#159Re: Deep learning job postings have collapsed in the past six months
#160Earlier quoted context omitted.
Everyone wants to fire up Tensorflow, Keras and PyTorch these days. Fewer people want to work in Airflow and SSIS, spend days tuning ETL, etc. This is the domain of data engineering, which bridges software engineering and data science with a dash of devops. I’ve been working in this field for a couple of years and it’s clear to me that data engineering is a necessary foundation and impact multiplier for data science.
Any formalized paths I could take to enter this field?