A lot of thee c folks aren't tech folks or even math folks. They want to try to use deep learning to do prediction or get some insight when something as simple as regression would have worked.
Deep learning job postings have collapsed in the past six months
31–40 of 274 posts
Re: Deep learning job postings have collapsed in the past six months
#32I observe the state of the art on most Nlp tasks since many years: In 2018,2019 there was huge progress made each year on most tasks. 2020,except for a few tasks have mostly stagnated... NLP accuracy is generally not production ready but the pace of progress was quick enough to have huge hopes. The root cause of the evil is: Nobody has build upon the state of the art pre trained language: XLnet while there are hundre…
Just to clarify one of your points regarding Google's involvement: XLnet, and the underlying TransformerXL technology, did have Google researchers involved:
* https://ai.googleblog.com/2019/01/transformer-xl-unleashing-...
* https://arxiv.org/pdf/1901.02860.pdf
* https://arxiv.org/pdf/1906.08237.pdf
My understanding is that a CMU student interned at Google and developed most of the pieces of TransformerXL, which formed the basis of XLNet. The student and the Google researcher further collaborated with CMU researchers to finalize the work.
(For the record, I think the remainder of your points do not match my understanding of NLP, which I do research in, but I just really wanted to clarify the XLNet story a bit).
Re: Deep learning job postings have collapsed in the past six months
#33Isn't it the same pattern every 10 years or so for "AI" related tech ? Some people hype tech X as being a game changer - tech X is way less amazing than advertised - investors bail out - tech X dies - rinse and repeat. https://en.wikipedia.org/wiki/AI_winter
This is more akin to the Internet bubble than the previous AI winter. The technology is valuable for business, but the hype is huge and companies aren't ready for it yet.
Re: Deep learning job postings have collapsed in the past six months
#34I observe the state of the art on most Nlp tasks since many years: In 2018,2019 there was huge progress made each year on most tasks. 2020,except for a few tasks have mostly stagnated... NLP accuracy is generally not production ready but the pace of progress was quick enough to have huge hopes. The root cause of the evil is: Nobody has build upon the state of the art pre trained language: XLnet while there are hundre…
Could you give an example of a major task that you think the state of the art could be trivially improved on with the xlnet approach?
Long (>2048 tokens) sequences.
But GP is too focused on hyping XLNet for some reason. There are much more elegant attempts at improving the transformer architecture in just the past 8 months: Reformer, Performer, Macaron Net, and my current pet paper, Normalized Attention Pooling (https://arxiv.org/abs/2005.09561).
Re: Deep learning job postings have collapsed in the past six months
#35I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth! I think this same thing is ha…
This is sadly so consistent with what I'm seeing at a big corporation. We are working so hard to make a centralized ML platform, get our data up to par, etc. but so many ML projects either have no chance of succeeding or have so little business value that they're not worth pursuing. Everyone on the development team for the project I'm working on is silently in agreement that our model would be better off being replaced by a well-managed rules engine, but every time we bring up these concerns, they're effectively disregarded.
There are obviously places in my company where ML is making an enormous impact, it's just not something that's fit for every single place where decisions need to be made. Sometimes doing some analysis to inform blunt rules works just as well - without the overhead of ML model management.
Re: Deep learning job postings have collapsed in the past six months
#36AI has a business problem.
Very few businesses I know actually have a deep learning problem. But they want a deep learning solution. Lest they get left out of the hype train.
Re: Deep learning job postings have collapsed in the past six months
#37I observe the state of the art on most Nlp tasks since many years: In 2018,2019 there was huge progress made each year on most tasks. 2020,except for a few tasks have mostly stagnated... NLP accuracy is generally not production ready but the pace of progress was quick enough to have huge hopes. The root cause of the evil is: Nobody has build upon the state of the art pre trained language: XLnet while there are hundre…
I do not agree with this. I work heavily with NLP models for production in the Legal domain (where my baseline is where a 8GB 1080 must predict more than 1000 words/sec). This year was when our team glued enough pieces of Deep Learning to outperform our previous statistic/old ML pipeline that was been optimized for years.
Little things compound such as optimizers ( Ranger/Adahessian), better RNN ( IndRNN, Linear Transformers, Hopfield networks ) and techniques (cache everywhere, Torch script,gradient accumulation training)
Re: Deep learning job postings have collapsed in the past six months
#38In my industry (research), we still have a strong line of business. Some commercial clients have killed their contracts with us to save money during the COVID era, but government contracts are still going strong. In areas where there's a clear use case I think there is still work to go around.
Re: Deep learning job postings have collapsed in the past six months
#39Finally! Big companies need to realize they must understand what what they are doing with technology to get any value of out it.
They've long resisted that, of course, but I'm pretty sure half the popular of deep learning was it leveled the playing field, making engineers as ignorant of the inner-workings of their creations as the middle managers.
May the middle-manager-fication of work, and acceptance of ignorance that goes with, fail.
-----
Then again, I do prefer it when many of those old moronic companies flounder, so maybe this is a bad thing that they're wising up.
Re: Deep learning job postings have collapsed in the past six months
#40Earlier quoted context omitted.
Why not? I would have thought it was a buyer’s market now with all the layoffs.
If it weren't urgent (i.e., lost a job) I'd be a little reluctant to join a company/team that I'd never met in person. I can imagine that others would be equally reluctant to hire someone they've only seen through Zoom.
Also if you didn't lose a job, you might not want to change right now if you're in a stable position, even if it's not your dream job.