Deep learning job postings have collapsed in the past six months • Hacker News Board

Re: Deep learning job postings have collapsed in the past six months

#271

Post by mistasteve » Tue, Sep 08, 2020, 1:48 AM UTC

disgruntledphd2 wrote: post #257

ghaff wrote: post #211

Earlier quoted context omitted.

Probably the biggest part of a lot of projects is cleaning up and normalizing the data (just like it was with data warehousing last century). A lot of high power data scientists don't want to hear that--especially if they're research-y.

If you don't at least tolerate data cleaning, you should find a different job than data scientist. I would have expected the researchy people to be better at it, as often you'll need to collect and analyse your own data during grad programs, and thus have some experience.

I kind of like data cleaning, I'm a weirdo like that

Re: Deep learning job postings have collapsed in the past six months

#272

Post by wsgreen » Fri, Sep 11, 2020, 4:57 PM UTC

apohn wrote: post #43

eric_b wrote: post #28

I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…

"Like Big Data, I think we'll see a few companies execute well and actually get some value, while most will just jump to the next shiny thing in a year or two." Here's another aspect - in many places nobody listens to the actual people doing the work. In my last job I was hired to lead a Data Science team and to help the company get value of Stats/ML/AI/DL/Buzzword. And I (and my team) were promptly overridden on eve…

Currently experiencing the exact same thing. Roughly 15 machine learning engineers through hiring and acquisitions and all of them have no product or management power. Everyone with any management or product power has zero ML/Data Science experience. I spend half my day explaining to managers what ML is.

Re: Deep learning job postings have collapsed in the past six months

#273

Post by Breza » Sat, Sep 12, 2020, 2:08 PM UTC

simonw wrote: post #97

Something I've learned: when non-engineers ask for an AI or ML implementation, they almost certainly don't understand the difference between that and an "algorithmic" solution. If you solve "trending products" by building a SQL statement that e.g. selects items with the largest increase of purchases this month in comparison to the same month a year ago, that's still "AI" to them. Knowing this can save you a lot of wa…

Exactly! I run a data science department at a corporation. I've had exactly one production level project that was sufficiently complicated to require deep learning. I am currently working on the second. I always start with the simplest approach. That's a message that I push hard for every recent grad and intern who comes here.

Re: Deep learning job postings have collapsed in the past six months

#274

Post by Breza » Sat, Sep 12, 2020, 2:16 PM UTC

jnwatson wrote: post #115

pbourke wrote: post #91

Earlier quoted context omitted.

Everyone wants to fire up Tensorflow, Keras and PyTorch these days. Fewer people want to work in Airflow and SSIS, spend days tuning ETL, etc. This is the domain of data engineering, which bridges software engineering and data science with a dash of devops. I’ve been working in this field for a couple of years and it’s clear to me that data engineering is a necessary foundation and impact multiplier for data science.

Don't forget data cleaning. A huge issue I've seen is just getting sufficient data of a high enough quality. Also, (for supervised classification problems) labelling is a big problem. It is almost as if we need a "data janitor" title.

This is a hugely important point. Having the right data is most of the battle in the typical project. I'm fortunate to run the data science department at a company with an amazing and supportive team of devs. We work together to make sure that all data is recorded in the most useful way and we talk regularly. Without their support, my models wouldn't be very useful.