Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep learning job postings have collapsed in the past six months

twitter.com

Post reply on HN
Print view View original

131–140 of 274 posts

Re: Deep learning job postings have collapsed in the past six months

#131

Post by grumple »

bratao wrote: post #59
maxlamb wrote: post #53

Earlier quoted context omitted.

Interesting. What's the main goal(s) of your NLP models?

We work on multiple models, all related to legal proceedings and lawsuits, such as: - Structure Judicial Federal Register texts - Identify entities in Legal texts (citation to laws, other lawsuits) - Predict time to completion, risk and amount due of a lawsuit - Classifying judicial proceedings to non lawyers

How accurate is your prediction of time/risk/amount? How useful is identifying entities or classifying proceedings?

Re: Deep learning job postings have collapsed in the past six months

#132

Post by mrosett »

jon_richards wrote: post #129
simonw wrote: post #97

Something I've learned: when non-engineers ask for an AI or ML implementation, they almost certainly don't understand the difference between that and an "algorithmic" solution. If you solve "trending products" by building a SQL statement that e.g. selects items with the largest increase of purchases this month in comparison to the same month a year ago, that's still "AI" to them. Knowing this can save you a lot of wa…

Any sufficiently misunderstood algorithm is indistinguishable from AI.

Ha! I'm going to have to borrow this phrase.

Re: Deep learning job postings have collapsed in the past six months

#133

Post by visarga »

baron_harkonnen wrote: post #51
eric_b wrote: post #28

I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…

> they paid more to get those insights than they were worth! This understates how awful ML is at many of these companies. I've seen quite a few companies that rushed to hire teams of people with a PhD in anything that barely made it through a DS/ML boot camp. To prove that they're super smart ML researchers without fail these hires rush to deploy a 3+ layer MLP to solve a problem that need at most a simple regression…

It's just a process of exploration, people trying out ideas to see what works. Over time, with sharing of results, we will gradually discover more nuanced approaches, but the exploration phase is necessary in order to map a path forward and train a generation of ML engineers and PM's who don't have seniors to learn from.

Of course it sucks on the short term, but there is zero chance the field will be abandoned. It has enough uses already.

Re: Deep learning job postings have collapsed in the past six months

#134

Post by tomhallett »

I know very little about the DL/ML space, but as a full-stack engineer it feels like most companies have tried to replicate what FAANG companies do (heavy investment in data/ml) when the cost/benefit simply isn't there.

Small companies need to frame the problem as:

1) Do we have a problem where the solution is discrete and already solved by an existing ML/DL model/architecture?

2) Can we have one of our existing engineers (or a short-term contractor) do transfer learning to slightly tweak that model to our specific problem/data?

Once that "problem" actually turns into multiple "machine learning problems" or "oh, we just need todo this one novel thing", they will probably need to bail because it'll be too hard/expensive and the most likely outcome will be no meaningful progress.

Said in another way: can we expect an engineer to get a fastai model up and running very quickly for our problem? If so, great - if not, then bail.

ie: the solution for most companies will be having 1 part-time "citizen data scientist" [1] on your engineering team.

[1]: https://www.datarobot.com/wiki/citizen-data-scientist/

Re: Deep learning job postings have collapsed in the past six months

#136

Post by smabie »

hogFeast wrote: post #106
tachyonbeam wrote: post #88

Earlier quoted context omitted.

I worked in a place full of deep learning PhDs, and you'd have people trying to apply reinforcement learning to problems that had known mathematical solutions, and integer programming problems. I don't think the issue is just that companies hire people who are awful at ML, it's also that people are trying to shoehorn deep learning into everything, even when it currently has nothing to offer and we have better solutio…

This is just my general sense, as a very non-expert with more experience of doing than theory...but the benefit is someone knowing the theory AND being able to translate that into revenue. I think most people view the hard part as doing the PHd, and so lots of people value that experience, and because they have that experience you have this endowment effect: wow, that PHd was hard, I must do very hard and complex thi…

The Man Group has had respectable returns, especially during Coronavirus. Nothing amazing, but certainly not terrible. Regardless, there's more to the picture: Sharpe ratio, vol, correlation to the market, etc

Re: Deep learning job postings have collapsed in the past six months

#138

Post by siliconvalley1 »

mritchie712 wrote: post #78
occamrazor wrote: post #75

Missing in the original chart/data: have ML/DL job postings decrease more or less than other comparable job categories (programming, business analyst, etc.)

Great point. Not as good point: is looking for pytorch and tf the right measure?

In his tweet I thought he made it clear he wasn't predicting an AI specific slowdown but a universal recession due to Covid?

Re: Deep learning job postings have collapsed in the past six months

#139

Post by huffmsa »

bonoboTP wrote: post #90
apohn wrote: post #84

Earlier quoted context omitted.

>If you think about it, that's the natural outcome. Why? Because people in corporations don't have the incentive to benefit the business but to progress their careers and that's done through meeting the goals for their position and make their upper ups progress with their careers too. This is one of the reasons I roll my eyes whenever I read something like "McKinsey says 75% of Big Data/AI/Buzzword projects do not de…

> because of silly management decisions? The whole point is, from their point of view those decisions are rational. It's much more lucrative from their (managers') personal point of view to develop a smokes-and-mirrors looks-good-on-ppt AI project. To be safe from risk, don't give the AI people too much responsibility, let them "do stuff", who cares, the point is we can now say we are an AI-driven company on the broc…

I think the problem a lot of places has been wanting "appealing" ML/AI solutions. The kind you write papers about and put on Powerpoints.

The useful AI/ML isn't glamorous, it's quite boring and ugly. Things like spam detection, image labeling, event parsing, text classification.

It's hard to get a big, shiny model into direct user facing systems.

Re: Deep learning job postings have collapsed in the past six months

#140

Post by whoisjuan »

Companies trying to add machine learning to everything they do like if that's going to solve all their problems or unlock new revenue streams.

80 or 90% of what companies are doing with machine learning results in systems with a high computing cost that are clearly unprofitable if seen as revenue impacting units. Many similar things can be achieved with low-level heuristics that result in way smaller computing costs.

But nobody wants to do that anymore. There's nothing "sexy" or "cool" about breaking down your problems and trying to create rule-based systems that addresses the problem. Semantic software is not cool anymore, and what became cool is this super expensive blackbox that requires more computer power than regular software. Companies have developed this bias for ML solutions because they seem to have this unlimited potential for solving problems, so it seems like a good long term investment. Everyone wants to take that bus.

Don't get me wrong. I love ML, but people use it for the stupidest things.

Post reply on HN