Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anyone else witnessing a panic inside NLP orgs of big tech companies?

old.reddit.com

Post reply on HN
Print view View original

41–50 of 527 posts

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#41

Post by goethes_kind »

echelon wrote: post #12
bippingchip wrote: post #4

As one of the comments on reddit posts - it's not just big tech companies, but also entire university teams which feel the goalposts moving miles ahead all of a sudden. Imagine working on your PhD on chat bots since start of 2022. Your entire PhD topic might be irrelevant already...

All these people don't understand how hireable and desirable they are now. They need to get out of academia and plugged into AI positions at tech companies and startups. Their value just went up tremendously, even if their PhD thesis got cancelled. Easily millionaires waiting to happen. --- edit: Can't respond to child comment due to rate limit, so editing instead. > That is not how it works at all. Speak for yoursel…

That is not how it works at all. You won't get hired if you don't have the academic pedigree in the first place. That means a completed Ph.D and good publications in good journals.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#42

Post by twawaaay »

I think education goal for people shifted. I teach my kids to be flexible and embrace the change. Invest in abilities that transfer well to various things you could be doing during your life. Be a problem solver.

In the future -- forget about cosy job you can be doing for the rest of your life. You no longer have any guarantees even if you own the business and even if you are farmer.

What you absolutely don't want is spend X years at uni learning something, and then 5-10 years into your "career" finding out it was obsoleted overnight and you now don't have plan B.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#43

Post by Beaver117 »

echelon wrote: post #8
oars wrote: post #5

If you were an NLP researcher at a university whose past years of experience is facing existential threat due to this rapid innovation causing your area to become obsolete, what would be some good areas to pivot to or refocus on?

Get out of academia and into industry. Why the hell stay in in academia? This is clearly the next technological wave, and you shouldn't sleep on it. Especially when you're so well positioned to take advantage of your experience. You can make $500,000/yr (maybe more with all the new startups and options) and be on the bleeding edge. If you want to go back to academia later, you can comfortably do so. Most don't, but t…

$500,000 is not a lot after all the inflation we had.

$100,000 in 1970 is worth almost $800,000 today.

Yes, downvote me all you want. But if you're an NLP expert thinking of working for a company that will make billions off your work, you can and should demand millions at least.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#44

Post by rr888 »

twawaaay wrote: post #42

I think education goal for people shifted. I teach my kids to be flexible and embrace the change. Invest in abilities that transfer well to various things you could be doing during your life. Be a problem solver. In the future -- forget about cosy job you can be doing for the rest of your life. You no longer have any guarantees even if you own the business and even if you are farmer. What you absolutely don't want is…

Liberal arts education will one day be back in fashion.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#45

Post by twawaaay »

rr888 wrote: post #44
twawaaay wrote: post #42

I think education goal for people shifted. I teach my kids to be flexible and embrace the change. Invest in abilities that transfer well to various things you could be doing during your life. Be a problem solver. In the future -- forget about cosy job you can be doing for the rest of your life. You no longer have any guarantees even if you own the business and even if you are farmer. What you absolutely don't want is…

Liberal arts education will one day be back in fashion.

Oh I do believe it. There will always be a market for snobs who will want to pay extra for handmade things vs AI-generated. The issue here is that it is all driven by fads and unstable. If you want to make money you will have to be flexible.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#46

Post by throwaway4aday »

gattilorenz wrote: post #23
mach1ne wrote: post #16

Earlier quoted context omitted.

Depending on definition, it is solved.

You're using the wrong definition, then. /s Where is some evidence that NLP is 'solved'? What does it even mean? OpenAI itself acknowledges the fundamental limitations of ChatGPT and the method of training it, but apparently everybody is happily sweeping them under the rug: "ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training,…

Is the goal of NLP for the model to actually understand the language it is processing? By understand I mean having the ability to relate the language to the real world and reason about it the same way a human would. To me, that goes far beyond NLP into true AI territory where the "model" is at the least conscious of its environment and possesses a true memory of past experiences. Maybe it would not be consciously aware of its self but it would be damn close.

I think LLMs have essentially solved the natural language processing problem but they have not solved reasoning or logical abilities including mathematics.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#47

Post by twa34532 »

oh no?!

so finally the tech sector is experiencing themselves what they have done to other lines of professions for the past decades, namely eradicting them (rightfully) with innovation?

well same advice applies then:

* embrace, move on and retrain for another profession * learn empathy from the panic and hurt

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#48

Post by CuriouslyC »

matthewdgreen wrote: post #40
echelon wrote: post #8

Earlier quoted context omitted.

Get out of academia and into industry. Why the hell stay in in academia? This is clearly the next technological wave, and you shouldn't sleep on it. Especially when you're so well positioned to take advantage of your experience. You can make $500,000/yr (maybe more with all the new startups and options) and be on the bleeding edge. If you want to go back to academia later, you can comfortably do so. Most don't, but t…

If you go into industry you’ll be given a chance to deploy these models and rush them into products. You’ll also make good money. If you go into academia (or research, whether it’s in academia or industry) you’ll be given the chance to try to understand what they’re doing. I can see the appeal of making money and rushing products out. But it wouldn’t even begin to compete with my curiosity. Makes me wish I was younge…

Plot twist: as these models increase in function, complexity and size, behaviors given activations will be as inscrutable to us as our behaviors are given gene and neuron activations.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#49

Post by CuriouslyC »

api wrote: post #2

It does seem like the (misnamed because it’s not open) OpenAI is very far ahead of most other efforts, especially at the edges in areas like instruction training and output filtering. Playing with Llama 65G gave me a sense for what the median raw effort is probably like. It seems to take a lot of work to fine tune and harness these systems and get them reliably producing useful output.

I don't think it's possible to build a moat around models at all. The model architectures are public, and there are already distributed group training projects so the compute isn't a barrier. The only moat is data.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#50

Post by snowwrestler »

IanCal wrote: post #31
gonzo41 wrote: post #30

Earlier quoted context omitted.

It'd be great if GPT could provide it's sources for the text it generated. I've been asking it about lyrics from songs that I know of, but where I can't find the original artist listed. I was hoping chat gpt had consumed a stack of lyrics and I could just ask it, "What song has this chorus or one similar to X..." It didn't work. Instead it firmly stated the wrong answer. And when I gave it time ranges it just noped o…

Have you tried bing chat? That search & sourcing is exactly what it does.

Sure, but the sources list is generated by the same system that generated the text, so it’s equally subject to hallucinations. Some examples in here:

https://dkb.blog/p/bing-ai-cant-be-trusted

To answer the question above, these systems cannot provide sources because they don’t work that way. Their source for everything is, basically, everything. They are trained on a huge corpus of text data and every output depends on that entire training.

They have no way to distinguish or differentiate which piece of the training data was the “actual” or “true” source of what they generated. It’s like the old questions “which drop caused the flood” or “which pebble caused the landslide”.

Post reply on HN