Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep learning job postings have collapsed in the past six months

twitter.com

Post reply on HN
Print view View original

61–70 of 274 posts

Re: Deep learning job postings have collapsed in the past six months

#61

Post by mrtksn »

apohn wrote: post #43
eric_b wrote: post #28

I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…

"Like Big Data, I think we'll see a few companies execute well and actually get some value, while most will just jump to the next shiny thing in a year or two." Here's another aspect - in many places nobody listens to the actual people doing the work. In my last job I was hired to lead a Data Science team and to help the company get value of Stats/ML/AI/DL/Buzzword. And I (and my team) were promptly overridden on eve…

If you think about it, that's the natural outcome. Why? Because people in corporations don't have the incentive to benefit the business but to progress their careers and that's done through meeting the goals for their position and make their upper ups progress with their careers too.

So essentially, you have a system where people spend other people's resources for living and their success is judged by making the chain link above happy. In especially large companies it's easy to have a disconnect from the product because people in the top specialise in topics that have nothing to do with the product. If the people at the top want to have this shiny new thing that the press and everyone else is saying that it's the next big thing, you better give them the new shiny thing if you want to have a smooth career. In publicly traded companies, this is even more prevalent because people who buy and sell the stocks would be even more disconnected from the product and tied to the buzzwords.

The more technical minded people who have the hunch on tech miss the point of the organisation that they are in and get very frustrated. It's probably the reason why startups can be much more fulfilling for deeply technical people.

Re: Deep learning job postings have collapsed in the past six months

#62

Post by x86_64Ubuntu »

cashsterling wrote: post #55
eric_b wrote: post #28

I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…

I also witnesses this first hand at a Biotech company I worked at... we were using many variants of machine learning algorithms to develop predictive models of cell culture and separation processes. Problem is... the models have so many parameters in order to get a useful fit that the same model can also fit a carrot or an elephant. We found that dynamic parameter estimation on ODE/DAE/PDE system models, while harder…

Kind of weird that they would use ML/AI for a separations process. Separations and chemical engineering in general absolutely LOVES parameters and systems of equations. And don't go anywhere near colloids, those have so many empirically sourced parameters it will make your head spin.

Re: Deep learning job postings have collapsed in the past six months

#63

Post by HlessClaudesman »

eric_b wrote: post #28

I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…

Yeah the big data comparison is apt, and a few years ago was The Block-Chain that got middle managers frothing like Pavlov's dog.

It is clear that for most of the companies who are investing in deep learning are tangible results are always around the corner, and maybe 1 in 100 will build something worthwhile. But here is the carrot driving them all on, it's like the lottery: you have to be in to win. The stick is the fear that their competitors will do so.

This field is more art than science, give talented people incentive to play and don't expect too much for the next decade.

Re: Deep learning job postings have collapsed in the past six months

#64

Post by apohn »

blaird wrote: post #54
eric_b wrote: post #28

I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…

Curious if there is a correlation with companies that failed to capitalize with the ones who relied on consultants versus really reshaping their own people. I worked for a financial services co that saw massive gains from big data/ML/AWS. Given, we were already using statistical models for everything, we just now could build more powerful features, more complex models, and move many things to more-real time, with mor…

>Curious if there is a correlation with companies that failed to capitalize with the ones who relied on consultants versus really reshaping their own people.

I've worked in Data Science customers facing roles for 2 companies, and one anecdotal correlation between success with Stats/ML/AI I've seen is how "Data Driven" people really are for their daily decision making. The more data driven you are, the more likely you are to identify a problem that can actually be improved by an Stat/ML/AI algorithm. This is because you really understand your data and the value you can get from it.

Everybody has metrics, KPIs, OKS, etc, but the reality is that there's a spectrum from 100% gut to 100% data driven. And a lot of people are on the gut side of things while thinking (or claiming they are) they are on the data side.

I'll provide an example. I currently work for a company that sells to (among others) companies working with industrial machinery. If your industrial machine runs in a remote area (e.g. an Oil Field), then any question about that machine starts with pulling up data. Being data driven is the only way to figure out what's going on. These folks have a good sense for identifying the value they can get from their data and they usually understand when you say dealing with their data is a engineering task in itself.

The other side of this is a factory filled with people. Since somebody is always operating and watching the machine, the "data driven" part is mainly alarms (e.g. is my temp over 100C) and some external KPI (e.g. a quality measurement). They are much less data driven than they think they are, and a lot of them don't understand what value they could get out of their data beyond some simple stuff you don't really need ML/AI for.

I mention industrial equipment because I think a lot of people (even me) are really surprised when they hear about people working in factories not being super data driven. You think of factories, engineering, and data as being very lumped together. It's amazing how many areas (sales, marketing, HR, are other great examples) exist where people aren't as data driven as they think they are.

Re: Deep learning job postings have collapsed in the past six months

#65

Post by Barrin92 »

samfisher83 wrote: post #31

A lot of thee c folks aren't tech folks or even math folks. They want to try to use deep learning to do prediction or get some insight when something as simple as regression would have worked.

what's particularly surprised me is how effective gradient boosting is in practise. I've seen so many cases of real world applications where just using catboost or whatever worked ~95% as well or even just as well as some super complicated deep learning approach and it saves you ten times the cost

Re: Deep learning job postings have collapsed in the past six months

#66

Post by freeone3000 »

abrichr wrote: post #18
hprotagonist wrote: post #3

Earlier quoted context omitted.

at least some is pure 2020. we want to hire, we can’t right now.

Why not? I would have thought it was a buyer’s market now with all the layoffs.

There's tons of layoffs because businesses are doing really badly. Current cashflow may not support another developer. Future cashflow doesn't look that great in any B2C market, either, and the B2B markets will start to look slim pickings not too far after that.

Re: Deep learning job postings have collapsed in the past six months

#69

Post by ellisv »

eric_b wrote: post #28

I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…

> they paid more to get those insights than they were worth!

> They were forcing ML in to places it didn't (and may never) belong.

I find that I spend a lot of time as a senior MLE telling someone why they don’t need ML

Post reply on HN