Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anyone else witnessing a panic inside NLP orgs of big tech companies?

old.reddit.com

Post reply on HN
Print view View original

191–200 of 527 posts

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#191

Post by moffkalast »

WalterBright wrote: post #161
mostlysimilar wrote: post #123

Earlier quoted context omitted.

This is possibly a death spiral. GPT is only possible because it's been trained on the work humans have learned to do and then put out in the world. Now GPT is as good as them and will put them all out of work. How can it improve if the people who fed it are now jobless?

Imagine the devastation wrought by automatic looms, that put all the weavers out of a job! 97% of jobs used to be working on the farm. Now it's something like 2%.

Can't wait for the economy that is 97% twitch streamers because that's all what humans are left qualified for. /s

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#192

Post by jMyles »

hn_throwaway_99 wrote: post #119

Wow - this is just wild. I've seen lots of arguments around "AI won't take everyone's job, it will just open up new areas for new jobs." Even if you take that with the benefit of the doubt (which I don't really think is warranted): 1. You don't need to take everyone's job. You just need to take a shitload of people's jobs. I think a lot of our current sociological problems, problems associated with wealth inequality,…

> given our current economic systems

What can possibly be the benefit of requiring this constraint?

Remove the idea that this is necessary and watch how much relaxation comes to the deliberation on this topic.

"Current economic systems" will simply have to yield. Along with states. This has been obvious for decades now. Deep breaths, everybody. :-)

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#193

Post by moffkalast »

psychphysic wrote: post #167
dogcomplex wrote: post #153

Earlier quoted context omitted.

It is very obvious there is a mass unemployment wave coming - or at least a mass "retraining" wave, though the new jobs "teaching AIs" or whatever remain to be seen. I hope everyone currently just questioning whether this will happen now is prepared to state it with conviction in the coming months and fight for some sort of social protection program for all these displaced people, because the profits from this new wo…

If not unemployment and retraining then a lot of people are going to need to miraculously become better at their jobs. I somehow imagine it'll be the worst of both worlds but I'm a glass half empty kind of guy.

Well it won't be miraculously, it'll be by using the AI tools to augment their work if anything. But probably unemployment.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#194

Post by yunyu »

jll29 wrote: post #154
hn_throwaway_99 wrote: post #119

Wow - this is just wild. I've seen lots of arguments around "AI won't take everyone's job, it will just open up new areas for new jobs." Even if you take that with the benefit of the doubt (which I don't really think is warranted): 1. You don't need to take everyone's job. You just need to take a shitload of people's jobs. I think a lot of our current sociological problems, problems associated with wealth inequality,…

> Imagine if you spent your entire career working on NLP, and now find GPT-4 will run rings around whatever you've done. What do you do now? I have been doing NLP since 1993. Before ca. 1996, there were mostly rule-based systems that were just toys. They lacked robustness. Then statistical systems came up and things like spell-checking (considering context when doing it), part of speech tagging and eventually even pa…

BERT can process 512 tokens. LLAMA and FLAN-UL2 can process 2048 tokens. GPT-4 can process 32768 tokens, and is much better at ignoring irrelevant context.

These general models can be fine tuned with domain specific data with a very small number of samples, and have surprisingly good transfer performance (beating classical models). New research like LORA/PEFT are making things like continuous finetuning possible. Statistical models also do a much better job at translating sentences to formal structure than the old ways ever did – so I wouldn't necessarily view those fields are disconnected.

I agree with the general sentiment, there are still major issues with the newer generation of models and things aren't fully cracked yet. But the scaling laws are saying there's still a lot of upside, even without new paradigms or architectural improvements.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#195

Post by leroy-is-here »

hnbad wrote: post #26

When I was studying Computational Linguistics I kept running into the unspoken question: given that Google Translate already exists, what is even the point of all of this? We were learning all these ideas about how to model natural language and tag parts of speech using linguistic theory so we could eventually discover that utopian solution that would let us feed two language models into a machine to make it perfectl…

I personally think that humans easily apply structure to language that doesn’t really exist. In fact, we restructure our languages daily, as individuals, when communicating verbally and through text. We make up words and shorthands and abbreviations and portmanteaus. But I think the brain simply makes connections between words and things and the structure of speaking those words is interpreted like audio or visuals in our brains — just patterns to be placed.

Really, words, utterances by themselves, carry meaning. Language is just a structure for _us_, so to speak, that we agree on for ease of communication. I think this is why probabilistic models do so well: the ideas we all have are mostly similar, it really is about just mapping from one kind of word to another, or kind of phrase to another.

Feel free to respond, I’m most certainly out of my depth here.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#196

Post by DangerousPie »

davidktr wrote: post #10
davidkuennen wrote: post #3

I tried translating something from English to German (my native language) yesterday with ChatGPT4 and compared it to Microsoft Translate, Google Translate and DeepL. My ranking: 1. ChatGPT4 - flawless translation. I was blown away 2. DeepL - very close, but one mistake 3. Google Translate - good translation, some mistakes 4. Microsoft Translate - bad translation, many mistakes I can understand the panic.

Fellow German here. Funny thing about DeepL: It translates "pathetisch" as "pathetic". For example: "Das war eine pathetische Rede." -> "That was a pathetic speech." I guess we have to get used to software redefining the meaning of words. It was kind of funny when that happened regarding Google Maps / neighborhood names, but with LLMs it's a different ballgame.

Another German here, and I have to admit I would have actually translated "pathetisch" as "pathetic" as well. I guess my German vocabulary has suffered quite a bit over the years of living abroad.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#197

Post by eternalban »

hn_throwaway_99 wrote: post #119

Wow - this is just wild. I've seen lots of arguments around "AI won't take everyone's job, it will just open up new areas for new jobs." Even if you take that with the benefit of the doubt (which I don't really think is warranted): 1. You don't need to take everyone's job. You just need to take a shitload of people's jobs. I think a lot of our current sociological problems, problems associated with wealth inequality,…

> how are these people supposed to eat?

My gut feeling is that AI is the 'social historic change' that will make UBI politically viable and a reality.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#198

Post by amrocha »

prottog wrote: post #182
xp84 wrote: post #149

Earlier quoted context omitted.

I think you and the one you're replying to are both very right. Yes, more of this money is going, instead of middle-class workers, straight to the capital class who own the "machines" that do the work people used to do. Except instead of it being a factory that makes industrial machines owned by some wealthy industrialist, the machines are things like Google and AWS and the owners are the small number of people with…

> but you could take lots of retail or clerical jobs, advance through hard work and support a family on those wages. Those are the days that are super gone Only in certain places, and only mostly due to crazy policies that made housing ridiculously unaffordable. I'm in an area where my barber lives on 10 acres of land he didn't inherit and together with his wife raises two children. This type of relaxed life is possi…

That kind of life is not achievable on minimum wage, even if you choose to live in a small city

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#199

Post by kelipso »

antegamisou wrote: post #122
goethes_kind wrote: post #63

Earlier quoted context omitted.

I guess we are living in two different universes. Any job ad for an ML role or ML adjacent role says Ph.d required or Ph.d preferable. Maybe it is also a matter of location. I am in Germany. For a plain SWE role a Ph.d might be a disadvantage here too, but for anything ML related it is mandatory from what I can see.

Yeah, you don't want to be anywhere near a place claiming to hire HS graduates/4chan posters in disciplines requiring advanced knowledge for successful product development, unless, idk, they have demonstrated mathematical talent through well-established media e.g. math olympiads, thesis on some relevant discipline. Almost all the time, they're shitty startups, where bankruptcy is a matter of time, run by overpromisin…

Yeah true, I've had to work with too many fresh college grads to not relate to this. People try to take some rare case and generalize when that's really not applicable.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#200

Post by woah »

jurassic wrote: post #186
qwerty3344 wrote: post #185

Earlier quoted context omitted.

There are entire sectors of the economy that LLMs can't touch - hospitality, manufacturing, caregivers, religious sectors, live-action entertainment, etc. Sure some of these will be replaced by robots but there will always be new jobs too.

No, there are not. Everything in the economy is connected and you can't have a vibrant industry without customers. The customers of hospitality/entertainment/healthcare/etc businesses are largely the middle class who will be put out of work by LLMs. So the person who today makes $200/night in tips waiting tables at a nice restaurant.... who will be buying those meals?

Someone who uses an LLM as a tool to perform a useful service
Post reply on HN