Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When ChatGPT broke the field of NLP: An oral history

quantamagazine.org

Post reply on HN
Print view View original

111–120 of 181 posts

Re: When ChatGPT broke the field of NLP: An oral history

#111

Post by dvsfish »

teruakohatu wrote: post #103
bravura wrote: post #55

Earlier quoted context omitted.

Sorry, I side with GP. Just because you don't want to use Llama/GPT because of cost, the middle-ground of DistilBERT etc (which can run on a single CPU) is a much more sensible cost/benefit tradeoff than VADER's decade old lexicon-based approach. I can't really think of many NLP things that are one-decade old and don't have a better / faster / cheaper alternative.

I must have explained myself extremely poorly. I spent a fair bit of money ~$1,000 USD running a near SOTA fine-tuned llama model on cloud GPUs for this very particular task.

This was clear both other times you explained it, the other commenters seem to want to nitpick despite it.

Re: When ChatGPT broke the field of NLP: An oral history

#112

Post by baq »

sph wrote: post #80
simonw wrote: post #35

Earlier quoted context omitted.

It's called the AI effect: https://en.wikipedia.org/wiki/AI_effect > The author Pamela McCorduck writes: "It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, 'that's not thinking'."

> every time somebody figured out how to make a computer do something Well, there’s the clue it is not really thinking if somebody told the machine how to do things. My Roomba isn’t intelligent because it’s been programmed to clean the floor, now is it? Wake me up when machines learn to do something on their own. I know everybody is on the AI hype train, but please show your extraordinary evidence to your extraordina…

> Wake me up when machines learn to do something on their own.

Be careful what you wish for if only just this once

Re: When ChatGPT broke the field of NLP: An oral history

#113

Post by baq »

throwaway314155 wrote: post #89
ahnick wrote: post #62

Earlier quoted context omitted.

How exactly is the bottom going to fall out? And are you really trying to present that you have practical experience building comparable tools to an LLM prior to the Transformer paper being written? Now, there does appear to be some shenanigans going on with circular financing involving MSFT, NVIDIA, and SMCI ( https://x.com/DarioCpx/status/1917757093811216627 ), but the usefulness of all the modern LLMs is undeniabl…

> And are you really trying to present that you have practical experience building comparable tools to an LLM prior to the Transformer paper being written? I believe (could be wrong) they were talking about their prior GOFAI/NLP experience when referencing scaling systems. In any case, is it really necessary to be so harsh about over-confidence and then go on to predict the future of solving hallucinations with your…

When you see the code it’s going to be too late.

Re: When ChatGPT broke the field of NLP: An oral history

#114

Post by [deleted] »

teruakohatu wrote: post #51
Cheer2171 wrote: post #38

Earlier quoted context omitted.

That's because VADER is just a dictionary mapping each word to a single sentiment weight and adding it up with some basic logic for negations and such. There's an ocean of smaller NLP ML between that naive approach and LLMs. LLMs are trained to do everything. If all you need is a model trained to do sentiment analysis, using VADER over something like DistilBERT is NLP malpractice in 2025.

> using VADER over something like DistilBERT is NLP malpractice in 2025. Ouch. Was that necessary? I used $1000 worth of GPU credits and threw in VADER because it’s basically free both in time and credits. I usually do this on large dataset out of pure interest in how it correlates with expensive methods on English language text. I am well aware of how VADER works and its limitations, I am also aware of the limitatio…

[deleted]

Re: When ChatGPT broke the field of NLP: An oral history

#115

Post by petesergeant »

wahern wrote: post #93
simonw wrote: post #35

Earlier quoted context omitted.

It's called the AI effect: https://en.wikipedia.org/wiki/AI_effect > The author Pamela McCorduck writes: "It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, 'that's not thinking'."

The flip side of that is that every time a new AI approach becomes popular even more people proclaim "this is what thinking is", believing that the new technology reflects the underlying process of human intelligence. This phenomenon goes back further than AI as a formal discipline, to early computers, and even during the age of mechanical computers. There are parallels with robotics, where for centuries anything tha…

> It's clearly not human intelligence

I don't think argument by assertion is appropriate where there's a lot of people who "clearly" believe that it's a good approximation of human intelligence. Given we don't understand how human intelligence works, asserting that one plausible model (a continuous journey through an embedding space held in neurons) that works in machines isn't how humans do it seems too strong.

Re: When ChatGPT broke the field of NLP: An oral history

#116

Post by suddenlybananas »

motorest wrote: post #91
suddenlybananas wrote: post #28

Earlier quoted context omitted.

Don't try and say anything pro-linguistics here, people are weirdly hostile if you think it's anything but probabilities.

> Don't try and say anything pro-linguistics here, (...) Shit-talking LLMs without providing any basis or substance is not what I would call "pro-linguistics". It just sounds like petty spiteful behavior, lashing out out of frustration for rendering old models obsolete.

From a scientific explanatory perspective, the old models are not obsolete because they are explanatory whereas LLMs do not explain anything about human linguistic behaviour.

Re: When ChatGPT broke the field of NLP: An oral history

#117

Post by intended »

teruakohatu wrote: post #31

I am in academia and worked in NLP although I would describe myself as NLP adjacent. I can confirm LLMs have essentially confined a good chunk of historical research into the bin. I suspect there are probably still a few PhD students working on traditional methods knowing full well a layman can do better using the mobile ChatGPT app. That said traditional NLP has its uses. Using the VADER model for sentiment analysis…

VADER made me sad when it couldn’t do code mixed analyses in 2020. I’m thinking of dusting off that project, but then I dread the thought of using LLMs to do the same sentiment analysis.

Re: When ChatGPT broke the field of NLP: An oral history

#118

Post by nolist_policy »

throwaway422432 wrote: post #90
qzw wrote: post #72

Earlier quoted context omitted.

And the more general version, “Humanity progresses one funeral at a time.” Which is why the hyper-longevity people are basically trying to freeze all human progress.

Or Effective Altruism's long-termism that effectively makes everyone universally poor now. Interestingly, Guillaume Verdon (e/acc) is friends with Bryan Johnson and seems to be pro-longevity.

Can you elaborate?

Re: When ChatGPT broke the field of NLP: An oral history

#119

Post by more-nitor »

mgraczyk wrote: post #87
teruakohatu wrote: post #31

I am in academia and worked in NLP although I would describe myself as NLP adjacent. I can confirm LLMs have essentially confined a good chunk of historical research into the bin. I suspect there are probably still a few PhD students working on traditional methods knowing full well a layman can do better using the mobile ChatGPT app. That said traditional NLP has its uses. Using the VADER model for sentiment analysis…

It currently costs around $2200 to run Gemini flash lite on all of Wikipedia English. It would probably cost around 10x that much to run sentiment analysis on every Yelp review ever posted. It's true that LLMs still cost a lot for some use cases, but for essentially any business case it's not worth using traditional NLP any more

idk why are you changing targets for comparison?

it's like:

"does apple cure cancer in monkeys?" vs "does blueberry cure diabetes in pigs?"

Re: When ChatGPT broke the field of NLP: An oral history

#120

Post by motorest »

suddenlybananas wrote: post #116
motorest wrote: post #91

Earlier quoted context omitted.

> Don't try and say anything pro-linguistics here, (...) Shit-talking LLMs without providing any basis or substance is not what I would call "pro-linguistics". It just sounds like petty spiteful behavior, lashing out out of frustration for rendering old models obsolete.

From a scientific explanatory perspective, the old models are not obsolete because they are explanatory whereas LLMs do not explain anything about human linguistic behaviour.

On one hand you have models which you argue are explanatory, but arguably do not work.

On the other hand, you have models that not only work but took the world by storm, and may or may not be made explanatory.

You either invest more work getting one explanatory model to work, or invest more work getting a working model to become explanatory.

What do you think is the fruitful research path?

Post reply on HN