Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep learning job postings have collapsed in the past six months

twitter.com

Post reply on HN
Print view View original

141–150 of 274 posts

Re: Deep learning job postings have collapsed in the past six months

#141

Post by chrisandchris »

poorman wrote: post #117
apohn wrote: post #43

Earlier quoted context omitted.

"Like Big Data, I think we'll see a few companies execute well and actually get some value, while most will just jump to the next shiny thing in a year or two." Here's another aspect - in many places nobody listens to the actual people doing the work. In my last job I was hired to lead a Data Science team and to help the company get value of Stats/ML/AI/DL/Buzzword. And I (and my team) were promptly overridden on eve…

I think if a business is set up to scale by volume they can see gains from it. For example, say a business is already doing well at 100k conversions a day. They manage to apply "big data/ML" to optimize those conversions and gain a 3% lift, they are now making over a 1,095,000 extra conversions a year they would not have otherwise made.

So they need to make $1 profit for each of those conversions just to make it worth if they hire 1 ML scientist for 95k/year. Or $10 if they hire 10 for 950k/year in total. And so on...

And there‘s the point where - IMHO - 3% gain may not be profitable enough.

Re: Deep learning job postings have collapsed in the past six months

#142

Post by pkaye »

andi999 wrote: post #120
cashsterling wrote: post #55

Earlier quoted context omitted.

I also witnesses this first hand at a Biotech company I worked at... we were using many variants of machine learning algorithms to develop predictive models of cell culture and separation processes. Problem is... the models have so many parameters in order to get a useful fit that the same model can also fit a carrot or an elephant. We found that dynamic parameter estimation on ODE/DAE/PDE system models, while harder…

Dyson asked Fermi about his take on his model fitting with four parameters. The reply was: I remember my friend Johnny von Neumann used to say, with four parameters I can fit an elephant, and with five I can make him wiggle his trunk.

Also reminds me this one:

> Everything is linear if plotted log-log with a fat magic marker

Re: Deep learning job postings have collapsed in the past six months

#143

Post by bonoboTP »

huffmsa wrote: post #139
bonoboTP wrote: post #90

Earlier quoted context omitted.

> because of silly management decisions? The whole point is, from their point of view those decisions are rational. It's much more lucrative from their (managers') personal point of view to develop a smokes-and-mirrors looks-good-on-ppt AI project. To be safe from risk, don't give the AI people too much responsibility, let them "do stuff", who cares, the point is we can now say we are an AI-driven company on the broc…

I think the problem a lot of places has been wanting "appealing" ML/AI solutions. The kind you write papers about and put on Powerpoints. The useful AI/ML isn't glamorous, it's quite boring and ugly. Things like spam detection, image labeling, event parsing, text classification. It's hard to get a big, shiny model into direct user facing systems.

What would you categorize as shiny in this case? "spam detection, image labeling, event parsing, text classification" can be implemented in lots of ways, simple and shiny as well.

Either way I don't think it matters too much because people can't really tell simple from shiny as long as the buzzword bullet points are there.

The point is rather that the job of the data science team is to deliver prestige to the manager, not to deliver data science solutions to actual practical problems. It's enough if they work on toy data and show "promising results" and can have percentages, impressive serious charts and numbers on the powerpoint slides.

I've heard from many data scientists in such situations that they don't get any input on what they should actually do, so they make up their own questions and own tasks to model, which often has nothing to do with actual business value, but they toy around with their models, produce accuracy percentages and that's enough.

Re: Deep learning job postings have collapsed in the past six months

#144

Post by AznHisoka »

According to data from Revealera.com, if you normalize the data, the % of job openings that mention 'deep learning' has actually remained stable YoY: https://i.imgur.com/sDoKwD0.png

* Revealera.com crawls job openings from over 10,000 company websites and analyzes them for technology trends for hedge funds.

Re: Deep learning job postings have collapsed in the past six months

#145

Post by AznHisoka »

eanzenberg wrote: post #72

This needs to be normalized to “job posting collapse in the past 6 months” unless you expect DL jobs to grow while everything shrinks? I’m somewhat surprised by the analysis from someone’s who’s “data driven.” I mean, he even says so as much in the twitter thread: “To be clear, I think this is an economic recession indicator, not the start of a new AI winter.” So, looks like he discovered an economic recession.

If you normalize the data, there is absolutely 0% change in the # of job openings for deep learning: https://i.imgur.com/sDoKwD0.png

Re: Deep learning job postings have collapsed in the past six months

#146

Post by [deleted] »

simonw wrote: post #97

Something I've learned: when non-engineers ask for an AI or ML implementation, they almost certainly don't understand the difference between that and an "algorithmic" solution. If you solve "trending products" by building a SQL statement that e.g. selects items with the largest increase of purchases this month in comparison to the same month a year ago, that's still "AI" to them. Knowing this can save you a lot of wa…

[deleted]

Re: Deep learning job postings have collapsed in the past six months

#147

Post by Izkata »

simonw wrote: post #97

Something I've learned: when non-engineers ask for an AI or ML implementation, they almost certainly don't understand the difference between that and an "algorithmic" solution. If you solve "trending products" by building a SQL statement that e.g. selects items with the largest increase of purchases this month in comparison to the same month a year ago, that's still "AI" to them. Knowing this can save you a lot of wa…

Some decades ago, that was AI to everyone.

In the future, I expect ML to also fall out of the "AI" umbrella - it gets used primarily for "smart code we don't know• how to write", so once that understanding comes, it gets a more-specific name and is no longer "AI".

•"know" being intentionally vague here, as obviously we can write both query planners and ML engines, but the latter isn't nearly as commonplace yet to completely fall out of the umbrella.

Re: Deep learning job postings have collapsed in the past six months

#148

Post by toomanybeersies »

eric_b wrote: post #28

I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…

That happened/is happening at my job. There's been a push to implement features that utilise AI/ML.

Not because it would be a good use case (although there are some for our product), or because it would be of any practical benefit, but because it makes for good marketing copy. Never mind the fact that nobody on the team has any experience with machine learning (I actually failed the paper at university).

Re: Deep learning job postings have collapsed in the past six months

#149

Post by mgleason_3 »

apohn wrote: post #84
mrtksn wrote: post #61

Earlier quoted context omitted.

If you think about it, that's the natural outcome. Why? Because people in corporations don't have the incentive to benefit the business but to progress their careers and that's done through meeting the goals for their position and make their upper ups progress with their careers too. So essentially, you have a system where people spend other people's resources for living and their success is judged by making the chai…

>If you think about it, that's the natural outcome. Why? Because people in corporations don't have the incentive to benefit the business but to progress their careers and that's done through meeting the goals for their position and make their upper ups progress with their careers too. This is one of the reasons I roll my eyes whenever I read something like "McKinsey says 75% of Big Data/AI/Buzzword projects do not de…

OK, so, we're scientists...and we're in the middle of a pandemic...amplifying/arguing over a graph showing a steep decline in job listing...that doesn't control for the pandemic...or even include a line for "overall job loss"...

https://www.burning-glass.com/u-s-job-postings-increase-four...

Looks like all job postings "collapsed during the pandemic"

Re: Deep learning job postings have collapsed in the past six months

#150

Post by google234123 »

jnwatson wrote: post #115
pbourke wrote: post #91

Earlier quoted context omitted.

Everyone wants to fire up Tensorflow, Keras and PyTorch these days. Fewer people want to work in Airflow and SSIS, spend days tuning ETL, etc. This is the domain of data engineering, which bridges software engineering and data science with a dash of devops. I’ve been working in this field for a couple of years and it’s clear to me that data engineering is a necessary foundation and impact multiplier for data science.

Don't forget data cleaning. A huge issue I've seen is just getting sufficient data of a high enough quality. Also, (for supervised classification problems) labelling is a big problem. It is almost as if we need a "data janitor" title.

Data cleaning always sounded suspicious to me.
Post reply on HN