Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anyone else witnessing a panic inside NLP orgs of big tech companies?

old.reddit.com

Post reply on HN
Print view View original

121–130 of 527 posts

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#121

Post by MonkeyMalarky »

not-chatgpt wrote: post #95
MonkeyMalarky wrote: post #84

I'm not at a big tech company, and we don't sell algorithms, but my team does use a lot of NLP stuff in internal algorithms. The only panic I have is trying to keep up and take the time to learn the new stuff. If anything, things like GPT-4 are going to make my team 10x more successful without having to hire an army of PhDs.

What does your team do? It feels like GPT4 can handle any task out there. Only drawback is latency and cost.

The price isn't even that bad, even the most expensive at 6cents per 1k tokens, it won't cost me much. It's the context size that's amazing. Gone are the days of only being able to pass ~500 tokens into something like BERT.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#122

Post by antegamisou »

goethes_kind wrote: post #63
Yoric wrote: post #59

Earlier quoted context omitted.

Hired in academia? Sure. Hired in industry. That's the opposite. I've had a friend who had to hide that they had a PhD to be hired...

I guess we are living in two different universes. Any job ad for an ML role or ML adjacent role says Ph.d required or Ph.d preferable. Maybe it is also a matter of location. I am in Germany. For a plain SWE role a Ph.d might be a disadvantage here too, but for anything ML related it is mandatory from what I can see.

Yeah, you don't want to be anywhere near a place claiming to hire HS graduates/4chan posters in disciplines requiring advanced knowledge for successful product development, unless, idk, they have demonstrated mathematical talent through well-established media e.g. math olympiads, thesis on some relevant discipline.

Almost all the time, they're shitty startups, where bankruptcy is a matter of time, run by overpromising-underdelivering grifter CTOs pursuing a get-rich-quick scheme using whatever is trendy right now -crypto, AI, whatever has the most density on the frontpage-.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#123

Post by mostlysimilar »

hn_throwaway_99 wrote: post #119

Wow - this is just wild. I've seen lots of arguments around "AI won't take everyone's job, it will just open up new areas for new jobs." Even if you take that with the benefit of the doubt (which I don't really think is warranted): 1. You don't need to take everyone's job. You just need to take a shitload of people's jobs. I think a lot of our current sociological problems, problems associated with wealth inequality,…

This is possibly a death spiral. GPT is only possible because it's been trained on the work humans have learned to do and then put out in the world. Now GPT is as good as them and will put them all out of work. How can it improve if the people who fed it are now jobless?

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#124

Post by akavi »

matthewdgreen wrote: post #40
echelon wrote: post #8

Earlier quoted context omitted.

Get out of academia and into industry. Why the hell stay in in academia? This is clearly the next technological wave, and you shouldn't sleep on it. Especially when you're so well positioned to take advantage of your experience. You can make $500,000/yr (maybe more with all the new startups and options) and be on the bleeding edge. If you want to go back to academia later, you can comfortably do so. Most don't, but t…

If you go into industry you’ll be given a chance to deploy these models and rush them into products. You’ll also make good money. If you go into academia (or research, whether it’s in academia or industry) you’ll be given the chance to try to understand what they’re doing. I can see the appeal of making money and rushing products out. But it wouldn’t even begin to compete with my curiosity. Makes me wish I was younge…

The danger is that the opportunity academia is giving you is something more like "you’ll be given the chance to try to understand what they were doing 5 years ago".

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#125

Post by hn_throwaway_99 »

sgt101 wrote: post #37
bippingchip wrote: post #4

As one of the comments on reddit posts - it's not just big tech companies, but also entire university teams which feel the goalposts moving miles ahead all of a sudden. Imagine working on your PhD on chat bots since start of 2022. Your entire PhD topic might be irrelevant already...

Perhaps - but normally you'll have a narrowly defined and very specific technical topic/hypothesis that you're working on, and many/most of these aren't going to be closed off by ChatGPT4 Will this effect the job market (both academic and commercial) for these folks? It's very hard to say. Clearly lots of value will be generated by the new generation of models. There will be a lot of catchup and utilisation work wher…

> but normally you'll have a narrowly defined and very specific technical topic/hypothesis that you're working on, and many/most of these aren't going to be closed off by ChatGPT4

Pretty hard disagree. Even if your NLP PhD topic is looking at hypotheses on underlying processes about how languages work (and LLMs can't give you this insight), 9 times out of 10 it's with an eye for some sort of "applicability" of this for the future. GPT-4 just cut off the applicability parts of this for huge swaths of NLP research.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#126

Post by org3 »

mashygpig wrote: post #109
dserban wrote: post #103

The PR folks at my current company are in full panic mode on Linkedin, judging from the passive-aggressive tone of their posts (sometimes very nearly begging customers not to use ChatGPT and friends). They fully understand that LLMs are stealing lunch money from established information retrieval industry players selling overpriced search algorithms. For a long time, my company was deluded about being protected by ins…

> Here's a new trend happening these days. Upon releasing new non-fiction books to the general public, authors are simultaneously offering an LLM-based chatbot box where you can ask the book any question. Can you link to an example?

https://portal.konjer.xyz/

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#127

Post by MonkeyMalarky »

mostlysimilar wrote: post #123
hn_throwaway_99 wrote: post #119

Wow - this is just wild. I've seen lots of arguments around "AI won't take everyone's job, it will just open up new areas for new jobs." Even if you take that with the benefit of the doubt (which I don't really think is warranted): 1. You don't need to take everyone's job. You just need to take a shitload of people's jobs. I think a lot of our current sociological problems, problems associated with wealth inequality,…

This is possibly a death spiral. GPT is only possible because it's been trained on the work humans have learned to do and then put out in the world. Now GPT is as good as them and will put them all out of work. How can it improve if the people who fed it are now jobless?

Also what happens to the intuition and unwritten skills that humans learned and passed on over time? Sure, the model has probably internalized them implicitly from the training data. But what happens in a case where you need to have a human perform the task again (say after a devastating war)? The ones with the arcane knowledge are gone, and now humans are starting from scratch.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#128

Post by jgust »

mostlysimilar wrote: post #123
hn_throwaway_99 wrote: post #119

Wow - this is just wild. I've seen lots of arguments around "AI won't take everyone's job, it will just open up new areas for new jobs." Even if you take that with the benefit of the doubt (which I don't really think is warranted): 1. You don't need to take everyone's job. You just need to take a shitload of people's jobs. I think a lot of our current sociological problems, problems associated with wealth inequality,…

This is possibly a death spiral. GPT is only possible because it's been trained on the work humans have learned to do and then put out in the world. Now GPT is as good as them and will put them all out of work. How can it improve if the people who fed it are now jobless?

Presumably this problem is solved with technology improvements or the need is recognized to hire experts capable of generating high quality training material. In either situation, there's going to be extreme discomfort.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#129

Post by mostlysimilar »

MonkeyMalarky wrote: post #127
mostlysimilar wrote: post #123

Earlier quoted context omitted.

This is possibly a death spiral. GPT is only possible because it's been trained on the work humans have learned to do and then put out in the world. Now GPT is as good as them and will put them all out of work. How can it improve if the people who fed it are now jobless?

Also what happens to the intuition and unwritten skills that humans learned and passed on over time? Sure, the model has probably internalized them implicitly from the training data. But what happens in a case where you need to have a human perform the task again (say after a devastating war)? The ones with the arcane knowledge are gone, and now humans are starting from scratch.

Incredible that we've been writing speculative fiction about this for decades and still we sleepwalk right into it. I'd love to be wrong, but I think we're all still too divided and self-interested for this kind of technology to be successfully integrated. A lot of people are going to suffer.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#130

Post by MonkeyMalarky »

credit_guy wrote: post #24

They may panic, but they shouldn't. They can quickly pivot. GPT programs can be used off the shelf, but they can also use custom training. Every large org has a huge internal set of documents, plus a large external set of documents relevant to its work (research articles, media articles, domain relevant rules and regulations). They can train a GPT bot to their particular codebase. And that is now. Soon (I'd give it a…

Yeah this thread has been the motivation for me to sign up on the wait list and cost out what it would take to try fine-tuning their older models on our data. There's still plenty of work out there when it comes to building a solution to a problem.
Post reply on HN