Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep learning job postings have collapsed in the past six months

twitter.com

Post reply on HN
Print view View original

181–190 of 274 posts

Re: Deep learning job postings have collapsed in the past six months

#181

Post by stjohnswarts »

barkingcat wrote: post #168
apohn wrote: post #43

Earlier quoted context omitted.

"Like Big Data, I think we'll see a few companies execute well and actually get some value, while most will just jump to the next shiny thing in a year or two." Here's another aspect - in many places nobody listens to the actual people doing the work. In my last job I was hired to lead a Data Science team and to help the company get value of Stats/ML/AI/DL/Buzzword. And I (and my team) were promptly overridden on eve…

This can be applied as "nobody listens to the people who actually do the work" as in company hires ML/AI experts to analyze purchase records and service records, and spits back out trends that the service front line workers (tier 1) already knew dead solid. Then the company doesn't listen to either group of people (neither tier 1 sales/support people, nor the ML people) and then fires / shuts down the entire division…

Some of the better historic manufacturers that "made it" were known to have good managers go and visit the filthy masses on the factory floor and get a feel for what's going on. It was very valuable for me when I used to help with manufacturing testing. I always spent some time with the techs and the people on the floor assembling stuff. A lot of it was useless but a lot of it was worthwhile and we learned to trust each other better instead of the "eggheads upstairs" and the "jarheads downstairs" that seemed to be most prevalent there.

Re: Deep learning job postings have collapsed in the past six months

#182

Post by berzerk »

pbourke wrote: post #91
supergeek133 wrote: post #81

I feel like it was also a classic case of running before we could crawl. Jumping from A to Z before we could go from 0 to 1. I work at an Residential IoT company, there are quite a few really valid use cases for Big Data and even ML. (Think about predictive failure). We hired more than one expensive data scientist in the past few years, and had big strategies more than once. But at the end of the day it's still "hard…

Everyone wants to fire up Tensorflow, Keras and PyTorch these days. Fewer people want to work in Airflow and SSIS, spend days tuning ETL, etc. This is the domain of data engineering, which bridges software engineering and data science with a dash of devops. I’ve been working in this field for a couple of years and it’s clear to me that data engineering is a necessary foundation and impact multiplier for data science.

I'm finding myself really enjoying this type of work and I think I would like to specialize in it. Any good learning resources you used on your path to where you are now?

Re: Deep learning job postings have collapsed in the past six months

#183

Post by pm90 »

mywittyname wrote: post #87
supergeek133 wrote: post #81

I feel like it was also a classic case of running before we could crawl. Jumping from A to Z before we could go from 0 to 1. I work at an Residential IoT company, there are quite a few really valid use cases for Big Data and even ML. (Think about predictive failure). We hired more than one expensive data scientist in the past few years, and had big strategies more than once. But at the end of the day it's still "hard…

This is absolutely right. And when you think about it, the reason behind has been staring us in the face: people who want to do machine learning approach everything as a machine learning problem. It's really common to see people handwave away the "easy stuff" because they want to get credit for doing the "hard stuff." It's not just the data scientists fault. I once heard our chief data scientist point out that they d…

How good are data scientists in building reliable, scalable systems? My anecdotal experience has been that many don’t bother or care to learn good software development practices, so the systems they build almost always work well for specific use cases but are hard to productionize.

Re: Deep learning job postings have collapsed in the past six months

#184

Post by ska »

jon_richards wrote: post #129
simonw wrote: post #97

Something I've learned: when non-engineers ask for an AI or ML implementation, they almost certainly don't understand the difference between that and an "algorithmic" solution. If you solve "trending products" by building a SQL statement that e.g. selects items with the largest increase of purchases this month in comparison to the same month a year ago, that's still "AI" to them. Knowing this can save you a lot of wa…

Any sufficiently misunderstood algorithm is indistinguishable from AI.

AI is what we call algorithms before we really understand them.

Re: Deep learning job postings have collapsed in the past six months

#185

Post by ska »

ajb wrote: post #122
jnwatson wrote: post #115

Earlier quoted context omitted.

Don't forget data cleaning. A huge issue I've seen is just getting sufficient data of a high enough quality. Also, (for supervised classification problems) labelling is a big problem. It is almost as if we need a "data janitor" title.

Phht you don't want to call it data janitor; no-one good will want that title. At least call it Data Integrity Engineer or something reasonably high-status.

Data Sanitation Engineer :)

Re: Deep learning job postings have collapsed in the past six months

#187

Post by alexslobodnik »

barkingcat wrote: post #168
apohn wrote: post #43

Earlier quoted context omitted.

"Like Big Data, I think we'll see a few companies execute well and actually get some value, while most will just jump to the next shiny thing in a year or two." Here's another aspect - in many places nobody listens to the actual people doing the work. In my last job I was hired to lead a Data Science team and to help the company get value of Stats/ML/AI/DL/Buzzword. And I (and my team) were promptly overridden on eve…

This can be applied as "nobody listens to the people who actually do the work" as in company hires ML/AI experts to analyze purchase records and service records, and spits back out trends that the service front line workers (tier 1) already knew dead solid. Then the company doesn't listen to either group of people (neither tier 1 sales/support people, nor the ML people) and then fires / shuts down the entire division…

Or it could be that a lot of data is wrong. It may be "technically" correct, ie the table in a database produces X. It is no surprise that executives would ignore what the "data" says because they don't trust it.

A lot of time they are right to ignore it. I've seen tables say X, but there was some flaw up the capture stack. Very few data analyst have the broad based knowledge and dedication needed to trace the data stack to establish the needed trust with the executive team.

Re: Deep learning job postings have collapsed in the past six months

#188

Post by proverbialbunny »

apohn wrote: post #43
eric_b wrote: post #28

I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…

"Like Big Data, I think we'll see a few companies execute well and actually get some value, while most will just jump to the next shiny thing in a year or two." Here's another aspect - in many places nobody listens to the actual people doing the work. In my last job I was hired to lead a Data Science team and to help the company get value of Stats/ML/AI/DL/Buzzword. And I (and my team) were promptly overridden on eve…

Ditto. The same thing happened to me a few companies back. I lead a data science team of two solving difficult problems that would determine the company's success. However, management was the type to be uncomfortable with ignorance so they had to pretend to know data science and demand tasks be solved a certain way, which for anyone who has any familiar experience has already guessed it: what they were pushing made no sense.

So, I switched from predictive analytics and put on my prescriptive analytics hat. Over the time I was there I created several presentations containing multiple paths forward letting management feel like they were deciding the path forward.

This continued until I was fired. The board didn't like that I wasn't using a neural nets to solve the companies problems. Startups often do not have enough labeled data, so DNNs were not considered. Oddly, I didn't get a warning or a request about this before being let go. I suspect management got tired of me managing upward. In response my coworker quit right then and there and took me out to lunch. ^_^

Re: Deep learning job postings have collapsed in the past six months

#190

Post by wwweston »

mrtksn wrote: post #61
apohn wrote: post #43

Earlier quoted context omitted.

"Like Big Data, I think we'll see a few companies execute well and actually get some value, while most will just jump to the next shiny thing in a year or two." Here's another aspect - in many places nobody listens to the actual people doing the work. In my last job I was hired to lead a Data Science team and to help the company get value of Stats/ML/AI/DL/Buzzword. And I (and my team) were promptly overridden on eve…

If you think about it, that's the natural outcome. Why? Because people in corporations don't have the incentive to benefit the business but to progress their careers and that's done through meeting the goals for their position and make their upper ups progress with their careers too. So essentially, you have a system where people spend other people's resources for living and their success is judged by making the chai…

> Because people in corporations don't have the incentive to benefit the business but to progress their careers

AKA the principal-agent problem:

https://en.wikipedia.org/wiki/Principal%E2%80%93agent_proble...

Post reply on HN