Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When ChatGPT broke the field of NLP: An oral history

quantamagazine.org

Post reply on HN
Print view View original

101–110 of 181 posts

Re: When ChatGPT broke the field of NLP: An oral history

#101

Post by Legend2440 »

throwaway314155 wrote: post #88
parpfish wrote: post #61

Earlier quoted context omitted.

Over my years in academia, I noticed that the linguistics departments were always the most fiercely ideological. Almost every comment of a talk would be get contested by somebody from the audience. It was annoying, but as a psych guy I was also jealous of them for having such clearly articulated theoretical frameworks. It really helped them develop cohesive lines of research to delineate the workings of each theory

Are there really all that many parallels between linguistics (the study of langauge) and computational-linguistics/NLP (subject of discussion)?

Computational linguistics, yes - it is the application of linguistics to computers.

Modern NLP, not really - it's all based around statistical modeling with very little linguistics.

Re: When ChatGPT broke the field of NLP: An oral history

#102

Post by wickedsight »

teruakohatu wrote: post #31

I am in academia and worked in NLP although I would describe myself as NLP adjacent. I can confirm LLMs have essentially confined a good chunk of historical research into the bin. I suspect there are probably still a few PhD students working on traditional methods knowing full well a layman can do better using the mobile ChatGPT app. That said traditional NLP has its uses. Using the VADER model for sentiment analysis…

So... What were the results? How did the Llama based model compare to VADER?

Re: When ChatGPT broke the field of NLP: An oral history

#103

Post by teruakohatu »

bravura wrote: post #55
teruakohatu wrote: post #51

Earlier quoted context omitted.

> using VADER over something like DistilBERT is NLP malpractice in 2025. Ouch. Was that necessary? I used $1000 worth of GPU credits and threw in VADER because it’s basically free both in time and credits. I usually do this on large dataset out of pure interest in how it correlates with expensive methods on English language text. I am well aware of how VADER works and its limitations, I am also aware of the limitatio…

Sorry, I side with GP. Just because you don't want to use Llama/GPT because of cost, the middle-ground of DistilBERT etc (which can run on a single CPU) is a much more sensible cost/benefit tradeoff than VADER's decade old lexicon-based approach. I can't really think of many NLP things that are one-decade old and don't have a better / faster / cheaper alternative.

I must have explained myself extremely poorly. I spent a fair bit of money ~$1,000 USD running a near SOTA fine-tuned llama model on cloud GPUs for this very particular task.

Re: When ChatGPT broke the field of NLP: An oral history

#104

Post by teruakohatu »

keyserj wrote: post #70
Karrot_Kream wrote: post #57

Earlier quoted context omitted.

Curious how big your dataset was if you used $1000 of GPU credits on DistilBERT. I've run BERT on CPU on moderate cloud instances no problem for datasets I've worked with, but which admittedly are not huge.

If I'm reading correctly, they used $1000 running a Llama model, not DistilBERT.

You read it correctly. I obviously didn't explain myself well.

Re: When ChatGPT broke the field of NLP: An oral history

#105

Post by JohnKemeny »

simonw wrote: post #95
sph wrote: post #80

Earlier quoted context omitted.

> every time somebody figured out how to make a computer do something Well, there’s the clue it is not really thinking if somebody told the machine how to do things. My Roomba isn’t intelligent because it’s been programmed to clean the floor, now is it? Wake me up when machines learn to do something on their own. I know everybody is on the AI hype train, but please show your extraordinary evidence to your extraordina…

Who's making extraordinary claims here?

I think they referred to the claim that AIs playing checkers should be considered thinking.

Re: When ChatGPT broke the field of NLP: An oral history

#106

Post by teruakohatu »

mootothemax wrote: post #86
teruakohatu wrote: post #31

I am in academia and worked in NLP although I would describe myself as NLP adjacent. I can confirm LLMs have essentially confined a good chunk of historical research into the bin. I suspect there are probably still a few PhD students working on traditional methods knowing full well a layman can do better using the mobile ChatGPT app. That said traditional NLP has its uses. Using the VADER model for sentiment analysis…

I’d love to hear your thoughts on BERTs - I’ve dabbled a fair bit, fairly amateurishly, and have been astonished by their performance. I’ve also found them surprisingly difficult and non-intuitive to train, eg deliberately including bad data and potentially a few false positives has resulted in notable success rate improvements. Do you consider BERTs to be the upper end of traditional - or, dunno, transformer archite…

That is a really good question, I am not sure where to draw the line.

I think it would be safe to say BERT is/was firmly in the non-traditional side of NLP.

A variety of task specific RNN models preceded BERT, and RNN as a concept has been around for quite a long time, with the LSTM being more modern.

Maybe word2vec ushered in the end of traditional NLP and was simultaneously also the beginning of non-traditional NLP? Much like Newton has been said to be both the first scientist and also the last magician.

I find discussing these kind of questions with NLP academics to be awkward.

Re: When ChatGPT broke the field of NLP: An oral history

#107

Post by JohnKemeny »

simianwords wrote: post #97
Animats wrote: post #45

"It helps to have tenure when something like this happens."

I wonder whether tenures are causing inefficiencies in the market? You might be encouraging someone to work on an outdated field without the correct incentives.

Just like having employees with experience, I guess.

But tenured researchers are supposed to have some more protection specifically because they do research (and reach conclusions) on topics that people in leadership positions in society might not like.

Re: When ChatGPT broke the field of NLP: An oral history

#108

Post by otabdeveloper4 »

qzw wrote: post #68
simonw wrote: post #35

Earlier quoted context omitted.

It's called the AI effect: https://en.wikipedia.org/wiki/AI_effect > The author Pamela McCorduck writes: "It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, 'that's not thinking'."

> somebody figured out how to make a computer do something Well, I would argue that in most deterministic AI systems the thinking was all done by the AI researchers and then encoded for the computer. That’s why historically it’s been easy to say, “No, the machine isn’t doing any thinking, but only applying thinking that’s embedded within.” I think that line of argument becomes less obvious when you have learning syst…

In many ways LLMs are a regression compared to what was before. They solve a huge class of problems quickly and cheaply, but they also have severe limitations that older methods didn't have.

So no, it's not a linear progress story like in a sci-fi story.

Re: When ChatGPT broke the field of NLP: An oral history

#109

Post by skissane »

qzw wrote: post #68
simonw wrote: post #35

Earlier quoted context omitted.

It's called the AI effect: https://en.wikipedia.org/wiki/AI_effect > The author Pamela McCorduck writes: "It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, 'that's not thinking'."

> somebody figured out how to make a computer do something Well, I would argue that in most deterministic AI systems the thinking was all done by the AI researchers and then encoded for the computer. That’s why historically it’s been easy to say, “No, the machine isn’t doing any thinking, but only applying thinking that’s embedded within.” I think that line of argument becomes less obvious when you have learning syst…

> It’s still fairly safe to argue that the best LLMs today are not yet thinking, at least not in a way a human does. But in another generation or two? It will become much harder to deny.

Current LLMs have a hard division between training and inference time; human brains don’t-we train as we infer (although we probably do a mix of online/offline training: you build new connections while awake, but then pruning and consolidation happens while you sleep). I think softening the training-vs-inference division is a necessary (but possibly not sufficient) condition for closing the artificial-vs-human intelligence gap. But that softening is going to require completely different architectures from current LLMs, and I don’t think anyone has much of an idea what those new architectures will look like, or how long it will take for them to arrive

Re: When ChatGPT broke the field of NLP: An oral history

#110

Post by skissane »

AndrewKemendo wrote: post #74
aidenn0 wrote: post #4

Earlier quoted context omitted.

Some people will never be convinced that a machine demonstrates intelligence. This is because for a lot of people, intelligence exists a subjective experience that they have and the belief that others have it too is only inasmuch as others appear to be like the self.

Humans are basically incapable of recognizing that there’s something that’s more powerful than them They’re never going to actively collectively admit that that’s the case, because humans collectively are so so systematically arrogant and self possessed that they’re not even open to the possibility of being lower on the intelligence totem pole The only possible way forward for AI is to create the thing that everybody…

> Humans are basically incapable of recognizing that there’s something that’s more powerful than them

> They’re never going to actively collectively admit that that’s the case, because humans collectively are so so systematically arrogant and self possessed that they’re not even open to the possibility of being lower on the intelligence totem pole

For most of human history, the clear majority of humans have believed in God(s), spirits, angels, bodhisattvas, etc - beings which by definition are above us on the “totem pole” - and although atheism is much more widespread today, I think it almost certainly remains a minority viewpoint at the global level.

So I’m sceptical of your idea humans have some inherent unwillingness to believe in superhuman entities. From an atheist perspective, one might say that the (globally/historically) average human is so eager to believe in such entities, that if they don’t actually exist, they’ll imagine them and then convince themselves that their imaginings are entirely real. (Whereas, a theist might argue that the human eagerness to believe in such entities is better explained by their existence.)

Post reply on HN