Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When ChatGPT broke the field of NLP: An oral history

quantamagazine.org

Post reply on HN
Print view View original

11–20 of 181 posts

Re: When ChatGPT broke the field of NLP: An oral history

#11

Post by meroes »

aidenn0 wrote: post #4
AndrewKemendo wrote: post #2

If Chomsky was writing papers in 2020 his paper would’ve been “language is all you need.” That is clearly not true and as the article points out wide scale very large forecasting models beat that hypothesis that you need an actual foundational structure for language in order to demonstrate intelligence when in fact is exactly the opposite. I’ve never been convinced by that hypothesis if for no other reason that we ca…

Some people will never be convinced that a machine demonstrates intelligence. This is because for a lot of people, intelligence exists a subjective experience that they have and the belief that others have it too is only inasmuch as others appear to be like the self.

It doesn’t mean they tie intelligence to subjective experience. Take digestion. Can a computer simulate digestion, yes. But no computer can “digest” if it’s just silicon in the corner of an office. There are two hurdles. The leap from simulating intelligence to intelligence, and the leap from intelligence to subjective experience. If the computer gets attached to a mechanism that physically breaks down organic material, that’s the first leap. If the computer gains a first person experience of that process, that’s the second.

You can’t just short-circuit from simulates to does to has subjective experience.

And the claim other humans don’t have subjective experience is such non-starter.

Re: When ChatGPT broke the field of NLP: An oral history

#12

Post by mistrial9 »

dekhn wrote: post #10
sp1nningaway wrote: post #7

For me as a lay-person, the article is disjointed and kinda hard to follow. It's fascinating that all the quotes are emotional responses or about academic politics. Even now, they are suspicious of transformers and are bitter that they were wrong. No one seems happy that their field of research has been on an astonishing rocketship of progress in the last decade.

The way I see this is that for a long time there was an academic field that was working on parsing natural human language and it was influenced by some very smart people who had strong opinions. They focused mainly on symbolic approaches to parsing, rather than probabilistic. And there were some fairly strong assumptions about structure and meaning. Norvig wrote about this: https://norvig.com/chomsky.html and I think…

> most effective way to make forward progress

powerful response but.. "fit for what purposes" .. All of human writings are not functionally equivalent. This has been discussed at length. e.g. poetry versus factual reporting or summation..

Re: When ChatGPT broke the field of NLP: An oral history

#13

Post by dekhn »

mistrial9 wrote: post #12
dekhn wrote: post #10

Earlier quoted context omitted.

The way I see this is that for a long time there was an academic field that was working on parsing natural human language and it was influenced by some very smart people who had strong opinions. They focused mainly on symbolic approaches to parsing, rather than probabilistic. And there were some fairly strong assumptions about structure and meaning. Norvig wrote about this: https://norvig.com/chomsky.html and I think…

> most effective way to make forward progress powerful response but.. "fit for what purposes" .. All of human writings are not functionally equivalent. This has been discussed at length. e.g. poetry versus factual reporting or summation..

https://www.amazon.com/dp/B0DYDGZTMV makes the case that DeepSeek is a poet.

Re: When ChatGPT broke the field of NLP: An oral history

#14

Post by Legend2440 »

languagehacker wrote: post #5

Great seeing Ray Mooney (who I took a graduate class with) and Emily Bender (a colleague of many at the UT Linguistics Dept., and a regular visitor) sharing their honest reservations with AI and LLMs. I try to stay as far away from this stuff as possible because when the bottom falls out, it's going to have devastating effects for everyone involved. As a former computational linguist and someone who built similar too…

They are far far more capable than anything your fellow computational linguists have come up with.

As the saying goes, 'every time I fire a linguist, the performance of the speech recognizer goes up'

Re: When ChatGPT broke the field of NLP: An oral history

#15

Post by aidenn0 »

6stringmerc wrote: post #9
aidenn0 wrote: post #4

Earlier quoted context omitted.

Some people will never be convinced that a machine demonstrates intelligence. This is because for a lot of people, intelligence exists a subjective experience that they have and the belief that others have it too is only inasmuch as others appear to be like the self.

It is until proven otherwise because modern science still doesn’t have a consensus or standards or biological tests which can account for it. As in, highly “intelligent” people often lack “common sense” or fall prey to con artists. It’s pompous as shit to assert a black box mimicry constitutes intelligence. Wake me up when it can learn to play a guitar and write something as good as Bob Dylan and Tom Petty. Hint: we’…

I can't write something as good as Bob Dylan and Tom Petty. Ergo I'm not intelligent.

Re: When ChatGPT broke the field of NLP: An oral history

#16

Post by aidenn0 »

meroes wrote: post #11
aidenn0 wrote: post #4

Earlier quoted context omitted.

Some people will never be convinced that a machine demonstrates intelligence. This is because for a lot of people, intelligence exists a subjective experience that they have and the belief that others have it too is only inasmuch as others appear to be like the self.

It doesn’t mean they tie intelligence to subjective experience. Take digestion. Can a computer simulate digestion, yes. But no computer can “digest” if it’s just silicon in the corner of an office. There are two hurdles. The leap from simulating intelligence to intelligence, and the leap from intelligence to subjective experience. If the computer gets attached to a mechanism that physically breaks down organic materi…

I think you're talking about consciousness rather than intelligence. While I do see people regularly distinguishing between simulation and reality for consciousness, I don't often see people make that distinction for intelligence.

> And the claim other humans don’t have subjective experience is such non-starter.

What about other primates? Other mammals? The smarter species of cephalopods?

Certain many psychopaths seem to act as if they have this belief.

Re: When ChatGPT broke the field of NLP: An oral history

#17

Post by shmel »

aidenn0 wrote: post #4
AndrewKemendo wrote: post #2

If Chomsky was writing papers in 2020 his paper would’ve been “language is all you need.” That is clearly not true and as the article points out wide scale very large forecasting models beat that hypothesis that you need an actual foundational structure for language in order to demonstrate intelligence when in fact is exactly the opposite. I’ve never been convinced by that hypothesis if for no other reason that we ca…

Some people will never be convinced that a machine demonstrates intelligence. This is because for a lot of people, intelligence exists a subjective experience that they have and the belief that others have it too is only inasmuch as others appear to be like the self.

How do they convince themselves that other people have intelligence too?

Re: When ChatGPT broke the field of NLP: An oral history

#18

Post by peterldowns »

dekhn wrote: post #10
sp1nningaway wrote: post #7

For me as a lay-person, the article is disjointed and kinda hard to follow. It's fascinating that all the quotes are emotional responses or about academic politics. Even now, they are suspicious of transformers and are bitter that they were wrong. No one seems happy that their field of research has been on an astonishing rocketship of progress in the last decade.

The way I see this is that for a long time there was an academic field that was working on parsing natural human language and it was influenced by some very smart people who had strong opinions. They focused mainly on symbolic approaches to parsing, rather than probabilistic. And there were some fairly strong assumptions about structure and meaning. Norvig wrote about this: https://norvig.com/chomsky.html and I think…

All of this matches my understanding. It was interesting taking an NLP class in 2017, the professors said basically listen, this curriculum is all historical and now irrelevant given LLMs, we’ll tell you a little about them but basically it’s all cutting edge sorry.

Re: When ChatGPT broke the field of NLP: An oral history

#20

Post by rdedev »

peterldowns wrote: post #18
dekhn wrote: post #10

Earlier quoted context omitted.

The way I see this is that for a long time there was an academic field that was working on parsing natural human language and it was influenced by some very smart people who had strong opinions. They focused mainly on symbolic approaches to parsing, rather than probabilistic. And there were some fairly strong assumptions about structure and meaning. Norvig wrote about this: https://norvig.com/chomsky.html and I think…

All of this matches my understanding. It was interesting taking an NLP class in 2017, the professors said basically listen, this curriculum is all historical and now irrelevant given LLMs, we’ll tell you a little about them but basically it’s all cutting edge sorry.

Same for my nlp class of 2021. Just directly went onto talking about transformers after a brief intro of the old stuff
Post reply on HN