Live data from Hacker News

Anyone else witnessing a panic inside NLP orgs of big tech companies?

old.reddit.com

71–80 of 527 posts

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#71
post #14

Earlier quoted context omitted.

Fellow German here. Funny thing about DeepL: It translates "pathetisch" as "pathetic". For example: "Das war eine pathetische Rede." -> "That was a pathetic speech." I guess we have to get used to software redefining the meaning of words. It was kind of funny when that happened regarding Google Maps / neighborhood names, but with LLMs it's a different ballgame.

Pathetic can mean emotional in English as well. Though I only discovered that by reading the dictionary. For anyone who doesn't speak German, pathetisch means with pathos, impassioned.

This strikes me as a good example of how nuanced language can be.

A native English speaker probably would only use "pathetic" to mean "emotional" if the emotions were specifically negative. They also would use pathetic to describe someone experiencing non-emotional suffering such as injury or poverty.

Therefore, a native English speaker probably would not use "pathetic" to mean "emotional" in everyday writing. However, I could definitely see someone using it to mean emotional when they were being more poetic. For example, I could see someone calling an essay on the emotional toll of counseling "The Pathetic Class" in order to imply that social workers are a class that society has tasked with confronting negative emotions.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#72
post #68

Earlier quoted context omitted.

I think my post is more representative of the truth than yours. I am sure you are telling the truth, but these unique talents you are talking about are not representative of the bulk of people working in research.

(My posting rate limit went away) The demand for AI/ML will fast outstrip available talent. We'll be pulling students right out of undergrad if they can pass an interview. I'm hiring folks off Reddit and 4chan that show an ability to futz with PyTorch and read papers. Also, from your sibling comment: > Maybe it is also a matter of location. I am in Germany. Huge factor. US cares about getting work done and little els…

Thanks for the insight. I hope you are right of course. Unfortunately, Germany is a bit hopeless in this respect.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#73
post #67
post #53

Earlier quoted context omitted.

I learnt some very basics of computational linguistics since it was related to a side project. I kept wondering why people were spending huge amounts of resources into tagging and labelling corpora of thousands of words, while to me it seems that in theory it should be possible to feed wikipedia (of a certain language) into a program and have it spit out some statistically correct rules about words and grammar. I gue…

I think a huge part is that computational linguistics still chases the idea of a universal language model, which may simply not be possible. I haven't followed the science in general linguistics but something feels off when most of the information ends up being tagged onto nil particles (i.e. parts of speech present neither in utterances nor written language and not affecting intonation or otherwise being detectable…

In a sense the model is universal. It's just a 100GB (give or take) neural network.

And apparently (or so I heard, I think) feeding transformer models training data of Language A could improve its ability to understand Language B. So maybe there's something truly universal in some sense.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#74
post #18

During my master's degree in data science, we had several companies visit our faculty to recruit students. Not a single one was a specialized NLP company, but many of them had NLP projects going on. Most of those projects were the usual "solution looking for a problem to solve". Even those projects that might have had _some_ utility, would have been way more effective to buy/license a product than to develop an in-ho…

>> "solution looking for a problem to solve" I wonder if this is a bad as everyone thinks. When a new technology arrives which is not completely understood, isn't the right approach to try to find some applications for it? Sure, most will fail, but some valid use cases will likely emerge. I'm pretty sure almost all technologies at some point were solutions looking for a problem to solve. Examples include the internet…

The computer was always designed to be a computational machine. It didn't just appear and then someone thought "what could I actually use this for?"

Also the Internet came out of DARPA which was a method of sharing data between geographically remote military facilities. It wasn't like they wired up devices and thought "what could we use this for?".

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#75

Earlier quoted context omitted.

>> "solution looking for a problem to solve" I wonder if this is a bad as everyone thinks. When a new technology arrives which is not completely understood, isn't the right approach to try to find some applications for it? Sure, most will fail, but some valid use cases will likely emerge. I'm pretty sure almost all technologies at some point were solutions looking for a problem to solve. Examples include the internet…

> I wonder if this is a bad as everyone thinks. I think it is. If they actually do end up finding a problem to solve, that would be serendipitous but I imagine the vast majority of the time they find themselves in the business of trying to convince the rest of us to buy a thing that we don’t need. And while the latter may drive the economy to some degree as I get older I detest it more and more.

This appears to be the computing model of the past 20 years, from what I can tell?

There have been no real advancements since the desktop model of the late 1990s. We might have more animations and applications running in virtual machines for security purposes, but literally nothing new has come out.

Even all the web apps are reimplementation of basic desktop capabilities from the decades before, but slower and with more RAM usage. They might be easier to write (I personally don't think so - RAD apps from the 90s were quicker to write and use) but the actual utility hasn't changed; if anything it's just shoving all of your data from your microcomputer to someone else's microcomputer, and being tracked and losing control of said data whilst you're at it!

And we have easier access to videos on the Internet, I guess??

It all seems to be missing the point of actually having a computational device locally. There is no computation going on. It's all digital paper pushing.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#76
post #21

Earlier quoted context omitted.

Does it translate hate speech too?

Of course it can

ButI thought ChatGPT has guardrails that prevent it from outputting hate speech, praising certain politicians and so on.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#77

Earlier quoted context omitted.

You're using the wrong definition, then. /s Where is some evidence that NLP is 'solved'? What does it even mean? OpenAI itself acknowledges the fundamental limitations of ChatGPT and the method of training it, but apparently everybody is happily sweeping them under the rug: "ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training,…

Is the goal of NLP for the model to actually understand the language it is processing? By understand I mean having the ability to relate the language to the real world and reason about it the same way a human would. To me, that goes far beyond NLP into true AI territory where the "model" is at the least conscious of its environment and possesses a true memory of past experiences. Maybe it would not be consciously awa…

LLMs have (maybe/probably) solved the language modeling problem, sure. That’s hardly NLP, right? NLG is more than “producing text with no semantics” and both NLG and NLU are only part of NLP.

ChatGPT cannot even reason reliably on what it knows and doesn’t know… it’s the library of Babel, but every book is written in excellent English.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#79
post #18

During my master's degree in data science, we had several companies visit our faculty to recruit students. Not a single one was a specialized NLP company, but many of them had NLP projects going on. Most of those projects were the usual "solution looking for a problem to solve". Even those projects that might have had _some_ utility, would have been way more effective to buy/license a product than to develop an in-ho…

Counterpoint, if one doesn't have their own baseline model how does one know the vendor is providing value.

Yeah having a whole big team create the internal baseline is not cost effective, but having at least one or two people work on something to actually know the vendor is worth their cost is important.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#80

Earlier quoted context omitted.

>> "solution looking for a problem to solve" I wonder if this is a bad as everyone thinks. When a new technology arrives which is not completely understood, isn't the right approach to try to find some applications for it? Sure, most will fail, but some valid use cases will likely emerge. I'm pretty sure almost all technologies at some point were solutions looking for a problem to solve. Examples include the internet…

The computer was always designed to be a computational machine. It didn't just appear and then someone thought "what could I actually use this for?" Also the Internet came out of DARPA which was a method of sharing data between geographically remote military facilities. It wasn't like they wired up devices and thought "what could we use this for?".

Do you assert that we had a good understanding of all of the problems that a computer could solve before making it? This seems absurd to me.
Post reply on HN