Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When ChatGPT broke the field of NLP: An oral history

quantamagazine.org

Post reply on HN
Print view View original

31–40 of 181 posts

Re: When ChatGPT broke the field of NLP: An oral history

#31

Post by teruakohatu »

I am in academia and worked in NLP although I would describe myself as NLP adjacent.

I can confirm LLMs have essentially confined a good chunk of historical research into the bin. I suspect there are probably still a few PhD students working on traditional methods knowing full well a layman can do better using the mobile ChatGPT app.

That said traditional NLP has its uses.

Using the VADER model for sentiment analysis while flawed is vastly cheaper than LLMs to get a general idea. Traditional NLP is suitable for many tasks people are now spending a lot of money asking GPT to do just because they know GPT.

I recently did an analysis on a large corpus and VADER was essentially free while the cloud costs to run a Llama based sentiment model was about $1000. I ran both because VADER costs nothing but minimal CPU time.

NLP can be wrong but it can’t be jailbroken and it won’t make stuff up.

Re: When ChatGPT broke the field of NLP: An oral history

#32

Post by bsder »

dekhn wrote: post #8
aidenn0 wrote: post #4

Earlier quoted context omitted.

Some people will never be convinced that a machine demonstrates intelligence. This is because for a lot of people, intelligence exists a subjective experience that they have and the belief that others have it too is only inasmuch as others appear to be like the self.

This is why I want the field to go straight to building indistinguishable agents- specifically, you should be able to video chat with an avatar that is impossible to tell from a human. Then we can ask "if this is indistinguishable from a human, how can you be sure that anybody is intelligent?" Personally I suspect we can make zombies that appear indistinguishable from humans (limited to video chat; making a robot tha…

"There is considerable overlap between the intelligence of the smartest bears and the dumbest tourists."

LLMs are not artificial intelligence but artificial stupidity.

LLMs will happily hallucinate. LLMs will happily tell you total lies with complete confidence. LLMs will give you grammatically perfect completely vapid content. etc.

And yet that is still better than what most humans could do in the same situation.

We haven't proved that machines can have intelligence, but instead we are happily proving that most people, most of the time just aren't very intelligent at all.

Re: When ChatGPT broke the field of NLP: An oral history

#33

Post by dekhn »

levocardia wrote: post #25
dekhn wrote: post #10

Earlier quoted context omitted.

The way I see this is that for a long time there was an academic field that was working on parsing natural human language and it was influenced by some very smart people who had strong opinions. They focused mainly on symbolic approaches to parsing, rather than probabilistic. And there were some fairly strong assumptions about structure and meaning. Norvig wrote about this: https://norvig.com/chomsky.html and I think…

Sounds like the bitter lesson is bitter indeed!

On the contrary, to some of us (who have focused on probability, big data, algorithms, and HPC, while eschewing complex theories that require geniuses to understand) the bitter lesson is incredibly sweet.

Very much like when I moved from tightly coupled to "embarassing" parallelism. A friend said "don't call it embarassing... it's pleasant not to have to think about hard distributed computing problems".

Re: When ChatGPT broke the field of NLP: An oral history

#34

Post by JumpCrisscross »

languagehacker wrote: post #5

Great seeing Ray Mooney (who I took a graduate class with) and Emily Bender (a colleague of many at the UT Linguistics Dept., and a regular visitor) sharing their honest reservations with AI and LLMs. I try to stay as far away from this stuff as possible because when the bottom falls out, it's going to have devastating effects for everyone involved. As a former computational linguist and someone who built similar too…

> learned the hard way not to trust the efficacy of these models or their ability to get the sort of reliability that a naive user would expect from them in practical application

But…they work. Linguistics as a science is still solid. But as a practical exercise, it seems to be moot other than for finding niches where LLMs are too pricey.

Re: When ChatGPT broke the field of NLP: An oral history

#35

Post by simonw »

aidenn0 wrote: post #4
AndrewKemendo wrote: post #2

If Chomsky was writing papers in 2020 his paper would’ve been “language is all you need.” That is clearly not true and as the article points out wide scale very large forecasting models beat that hypothesis that you need an actual foundational structure for language in order to demonstrate intelligence when in fact is exactly the opposite. I’ve never been convinced by that hypothesis if for no other reason that we ca…

Some people will never be convinced that a machine demonstrates intelligence. This is because for a lot of people, intelligence exists a subjective experience that they have and the belief that others have it too is only inasmuch as others appear to be like the self.

It's called the AI effect: https://en.wikipedia.org/wiki/AI_effect

> The author Pamela McCorduck writes: "It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, 'that's not thinking'."

Re: When ChatGPT broke the field of NLP: An oral history

#36

Post by JumpCrisscross »

bsder wrote: post #32
dekhn wrote: post #8

Earlier quoted context omitted.

This is why I want the field to go straight to building indistinguishable agents- specifically, you should be able to video chat with an avatar that is impossible to tell from a human. Then we can ask "if this is indistinguishable from a human, how can you be sure that anybody is intelligent?" Personally I suspect we can make zombies that appear indistinguishable from humans (limited to video chat; making a robot tha…

"There is considerable overlap between the intelligence of the smartest bears and the dumbest tourists." LLMs are not artificial intelligence but artificial stupidity . LLMs will happily hallucinate. LLMs will happily tell you total lies with complete confidence. LLMs will give you grammatically perfect completely vapid content. etc. And yet that is still better than what most humans could do in the same situation. W…

> yet that is still better than what most humans could do in the same situation

Yup. A depressing takeaway from LLMs is most humans don’t demonstrate a drive to be curious and understand, but instead, to sort of muddle through most (economic) tasks.

Re: When ChatGPT broke the field of NLP: An oral history

#37

Post by Agingcoder »

sp1nningaway wrote: post #7

For me as a lay-person, the article is disjointed and kinda hard to follow. It's fascinating that all the quotes are emotional responses or about academic politics. Even now, they are suspicious of transformers and are bitter that they were wrong. No one seems happy that their field of research has been on an astonishing rocketship of progress in the last decade.

Well, if you’ve built a career on something, you will usually actively resist anything that threatens to destroy it.

In other words, what is progress for the field might not be progress for you !

This reminds me of Thomas Kuhn’s excellent book ´the structure of scientific revolutions’ https://en.wikipedia.org/wiki/The_Structure_of_Scientific_Re...

Re: When ChatGPT broke the field of NLP: An oral history

#38

Post by Cheer2171 »

teruakohatu wrote: post #31

I am in academia and worked in NLP although I would describe myself as NLP adjacent. I can confirm LLMs have essentially confined a good chunk of historical research into the bin. I suspect there are probably still a few PhD students working on traditional methods knowing full well a layman can do better using the mobile ChatGPT app. That said traditional NLP has its uses. Using the VADER model for sentiment analysis…

That's because VADER is just a dictionary mapping each word to a single sentiment weight and adding it up with some basic logic for negations and such. There's an ocean of smaller NLP ML between that naive approach and LLMs. LLMs are trained to do everything. If all you need is a model trained to do sentiment analysis, using VADER over something like DistilBERT is NLP malpractice in 2025.

Re: When ChatGPT broke the field of NLP: An oral history

#40

Post by coffeeaddict1 »

aidenn0 wrote: post #15
6stringmerc wrote: post #9

Earlier quoted context omitted.

It is until proven otherwise because modern science still doesn’t have a consensus or standards or biological tests which can account for it. As in, highly “intelligent” people often lack “common sense” or fall prey to con artists. It’s pompous as shit to assert a black box mimicry constitutes intelligence. Wake me up when it can learn to play a guitar and write something as good as Bob Dylan and Tom Petty. Hint: we’…

I can't write something as good as Bob Dylan and Tom Petty. Ergo I'm not intelligent.

This to me is a weak argument. You have the ability to appreciate and judge something as good as Bob Dylan and Tom Petty. That's what makes you intelligent.
Post reply on HN