Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep learning job postings have collapsed in the past six months

twitter.com

Post reply on HN
Print view View original

91–100 of 274 posts

Re: Deep learning job postings have collapsed in the past six months

#91

Post by pbourke »

supergeek133 wrote: post #81

I feel like it was also a classic case of running before we could crawl. Jumping from A to Z before we could go from 0 to 1. I work at an Residential IoT company, there are quite a few really valid use cases for Big Data and even ML. (Think about predictive failure). We hired more than one expensive data scientist in the past few years, and had big strategies more than once. But at the end of the day it's still "hard…

Everyone wants to fire up Tensorflow, Keras and PyTorch these days. Fewer people want to work in Airflow and SSIS, spend days tuning ETL, etc. This is the domain of data engineering, which bridges software engineering and data science with a dash of devops. I’ve been working in this field for a couple of years and it’s clear to me that data engineering is a necessary foundation and impact multiplier for data science.

Re: Deep learning job postings have collapsed in the past six months

#92

Post by lallysingh »

mumblemumble wrote: post #83
austinl wrote: post #74

Earlier quoted context omitted.

I've heard this happen in a lot of places — companies want to be "data-driven", but then leadership simply ignores the data. I think being data-driven is something that is built into company culture, or otherwise it's too easy to just ignore the results and ship. The place I currently work is data-driven (perhaps to a fault). Every change is wrapped behind an experiment and analyzed. Engineers play a major role in th…

Imagine what it must be like for the senior leadership of an established company to actually become data-driven. All of a sudden the leadership is going to consent to having all of their strategic and tactical decision-making be questioned by a bunch of relatively new hires from way down the org chart, whose entire basis for questioning all that expertise and business acumen is that they know how to fiddle around wit…

I expect data driven leaders to be good at analyzing data. The rest are bullshitters.

Re: Deep learning job postings have collapsed in the past six months

#93

Post by bishalb »

eric_b wrote: post #28

I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…

Or like conversion rate optimization tools.

Re: Deep learning job postings have collapsed in the past six months

#94

Post by kfk »

Data science and ML In big companies are pulling resources away from the real value add activities like proper data integrity, blending sources, improving speed performance. Yes Business Intelligence is not cool anymore. Yes I also call my team “data analytics”. But let’s not forget the simple fact that “data driven” means we give people insights when and where they need them. Insights could be coming from an sql group by, ML, AI, watching the flying of birds, but they are still simply a data point for some human to make a decision. That means we need to produce the insight, being able to communicate it to people, have the the credibility for said people to actually listen to what we are saying. Focusing on how we put that data point together is irrelevant, focusing on hiring PHDs to do ML is most likely going to end in a failure because PHDs are not predictive of great analytical skills, experience and things like sql are much better predictors.

Re: Deep learning job postings have collapsed in the past six months

#95

Post by ideals »

mrtksn wrote: post #61
apohn wrote: post #43

Earlier quoted context omitted.

"Like Big Data, I think we'll see a few companies execute well and actually get some value, while most will just jump to the next shiny thing in a year or two." Here's another aspect - in many places nobody listens to the actual people doing the work. In my last job I was hired to lead a Data Science team and to help the company get value of Stats/ML/AI/DL/Buzzword. And I (and my team) were promptly overridden on eve…

If you think about it, that's the natural outcome. Why? Because people in corporations don't have the incentive to benefit the business but to progress their careers and that's done through meeting the goals for their position and make their upper ups progress with their careers too. So essentially, you have a system where people spend other people's resources for living and their success is judged by making the chai…

Others have also pointed out that too many ML engineers and researchers rush into problems and end up with useless results also hinges on this. These people have to deliver something because their job depends on it. Everything is move fast even when that doesn't make sense.

Re: Deep learning job postings have collapsed in the past six months

#96

Post by synthc »

monksy wrote: post #86
eric_b wrote: post #28

I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…

> big data That's because it didn't get a chance to mature and to show how it could be powerful. People kept trying to force hadoop into it and call themselves "big data experts" We've gotten a bit more clarity in this world with streaming technologies. However, there hasn't been a good and clear voice to say "hey .. this is how it fits in with your web app and this is what you expect of it". (I'm thinking about deve…

These days it's people trying to force Kafka into it and call themselves "streaming experts"

Re: Deep learning job postings have collapsed in the past six months

#97

Post by simonw »

Something I've learned: when non-engineers ask for an AI or ML implementation, they almost certainly don't understand the difference between that and an "algorithmic" solution.

If you solve "trending products" by building a SQL statement that e.g. selects items with the largest increase of purchases this month in comparison to the same month a year ago, that's still "AI" to them.

Knowing this can save you a lot of wasted time.

Re: Deep learning job postings have collapsed in the past six months

#98

Post by flavor8 »

x87678r wrote: post #68

In general does anyone know if its a good time to look for a new dev job? I was really going to move this year, but it seems sensible to wait. Just sucks to see friends with RSUs going up in value so quickly.

No harm in having a recruiter or two feed you opportunities on a regular basis to interview at (just be up front with them that you're holding out for a solid fit for your criteria). Better to have a job while interviewing than be under pressure to accept the first half decent thing that comes along.

Re: Deep learning job postings have collapsed in the past six months

#99

Post by mumblemumble »

baron_harkonnen wrote: post #51
eric_b wrote: post #28

I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…

> they paid more to get those insights than they were worth! This understates how awful ML is at many of these companies. I've seen quite a few companies that rushed to hire teams of people with a PhD in anything that barely made it through a DS/ML boot camp. To prove that they're super smart ML researchers without fail these hires rush to deploy a 3+ layer MLP to solve a problem that need at most a simple regression…

My sense is that the original sin here is conflating data science with machine learning.

A good data scientist might choose to use machine learning to accomplish their job. Or they might find that classical statistical inference is the better tool for the task at hand. A good data scientist, having built this model, might choose to put it into production. Or they might find that a simple if-statement could do the job almost as effectively but not nearly as expensively. A good data scientist, having decided to productionize a model, will also provide some information about how it might break down - for example, describing shifts in customer behavior, or changes in how some input signal is generated, or feedback effects that might invalidate the model.

OTOH, if your job has been framed in terms of cutting-edge machine learning, then you may well know - at a gut level, if not consciously - that your job is basically just a pissing match to see who can deploy the most bleeding-edge or expensive technology the fastest. It's like the modern hospital childbirth scene in Monty Python's The Meaning of Life, where the doctor is more interested in showing off the machine that goes, "ping!" in order to impress the other doctors than he is in paying attention to the mother.

Post reply on HN