Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When ChatGPT broke the field of NLP: An oral history

quantamagazine.org

Post reply on HN
Print view View original

171–180 of 181 posts

Re: When ChatGPT broke the field of NLP: An oral history

#171

Post by rdedev »

motorest wrote: post #96
rdedev wrote: post #30

Earlier quoted context omitted.

It's a truly bitter pill to swallow when your whole area of research goes redundant. I have a bit of background in this field so it's nice to see even people who were at the top of the field raise concerns that I had. That comment about LHC was exactly what I told my professor. That the whole field seems to be moving in a direction where you need a lot of resources to do anything. You can have 10 different ideas on h…

> That the whole field seems to be moving in a direction where you need a lot of resources to do anything. You can have 10 different ideas on how to improve LLMs but unless you have the resources there is barely anything you can do. I think you're confusing problems, or you're not realizing that improving the efficiency of a class of models is a research area on it's own. Look at any field that involves expensive com…

I felt that way maybe an year or two ago. It seemed like the most research were only concerned about building bigger models to beat benchmarks. There was also this prevalent idea that models need to be big and have massive compute. Especially from companies like openai. I was glad that models like deepseek were made. Bought back some hope

Re: When ChatGPT broke the field of NLP: An oral history

#172

Post by larodi »

horsh1 wrote: post #141
simianwords wrote: post #139

Earlier quoted context omitted.

Not totally related but I have wondered how someone who thinks they are an expert in a field may deal with contradictions presented by GPT. For example, you may consider yourself an expert on some niche philosophy like say Orientalism. A student can now contradict any theories you can come up with using GPT and the scary thing is that they will be sensible contradictions. I feel like the bar to consider yourself an e…

Why in this story students use GPT and professors don't? If you are an expert -- sit down and start working with GPT on your own. See what it can and what it can not do. See where it helps. See where it hands down lose.

I sure does use it to prepare exams etc. Problem is students don’t seem to see the reason for not using GPT to solve them ;) for me it perfectly makes sense to prepare exams and labs with some AI help… but this questions both my role and the role of in-person education.

Really I consider stopping doing it at all…

Re: When ChatGPT broke the field of NLP: An oral history

#173

Post by throwaway422432 »

nolist_policy wrote: post #118
throwaway422432 wrote: post #90

Earlier quoted context omitted.

Or Effective Altruism's long-termism that effectively makes everyone universally poor now. Interestingly, Guillaume Verdon (e/acc) is friends with Bryan Johnson and seems to be pro-longevity.

Can you elaborate?

Long-termism is essentially a utilitarian doctrine, but uses the "total utilitarianism" method:

https://en.wikipedia.org/wiki/Average_and_total_utilitariani...

Take that moral position and then extend the calculation to cover all potential generations of humans to come, and the value of maximising your utility now is infinitely small.

Re: When ChatGPT broke the field of NLP: An oral history

#174

Post by dunefox »

moffkalast wrote: post #148
Cheer2171 wrote: post #38

Earlier quoted context omitted.

That's because VADER is just a dictionary mapping each word to a single sentiment weight and adding it up with some basic logic for negations and such. There's an ocean of smaller NLP ML between that naive approach and LLMs. LLMs are trained to do everything. If all you need is a model trained to do sentiment analysis, using VADER over something like DistilBERT is NLP malpractice in 2025.

> dictionary mapping each word to a single sentiment weight That seems to me like it would flat out fail on sarcasm. How is that still considered a usable method today?

It's not.

Re: When ChatGPT broke the field of NLP: An oral history

#175

Post by dunefox »

Legend2440 wrote: post #14
languagehacker wrote: post #5

Great seeing Ray Mooney (who I took a graduate class with) and Emily Bender (a colleague of many at the UT Linguistics Dept., and a regular visitor) sharing their honest reservations with AI and LLMs. I try to stay as far away from this stuff as possible because when the bottom falls out, it's going to have devastating effects for everyone involved. As a former computational linguist and someone who built similar too…

They are far far more capable than anything your fellow computational linguists have come up with. As the saying goes, 'every time I fire a linguist, the performance of the speech recognizer goes up'

1. Sadly, they are for most tasks, yes.

2. Linguist, not computational linguist. ;)

Re: When ChatGPT broke the field of NLP: An oral history

#176

Post by dunefox »

Legend2440 wrote: post #101
throwaway314155 wrote: post #88

Earlier quoted context omitted.

Are there really all that many parallels between linguistics (the study of langauge) and computational-linguistics/NLP (subject of discussion)?

Computational linguistics, yes - it is the application of linguistics to computers. Modern NLP, not really - it's all based around statistical modeling with very little linguistics.

IME, computational linguistics is just NLP and has been for 10 years or so.

Re: When ChatGPT broke the field of NLP: An oral history

#177

Post by qnleigh »

teruakohatu wrote: post #31

I am in academia and worked in NLP although I would describe myself as NLP adjacent. I can confirm LLMs have essentially confined a good chunk of historical research into the bin. I suspect there are probably still a few PhD students working on traditional methods knowing full well a layman can do better using the mobile ChatGPT app. That said traditional NLP has its uses. Using the VADER model for sentiment analysis…

How well did VADER correlate with Llama? Did you try any other methods intermediate between them?

Re: When ChatGPT broke the field of NLP: An oral history

#178

Post by famouswaffles »

suddenlybananas wrote: post #169
canjobear wrote: post #162

Earlier quoted context omitted.

I don't see how it's absurd. The point isn't that languages don't have word order constraints, it's that they're organized in such a way that word order is usually redundant with other things when it comes to expressing meaning. This redundancy is a real and nontrivial property of language which becomes obvious when you analyze it in a statistical and usage-based way, but which isn't so clear if you're focused only o…

The redundancy is not a property of language it's a property of communication. This just speaks to the pseudoscientific nature of usage based linguistics; it's as if physicists claimed that heavier objects fell faster than lighter ones because a piece of paper will fall slower than a brick. You can't just look at the most frequent cases, you need to look at the actual edge cases to understand a phenomenon.

>The redundancy is not a property of language it's a property of communication.

Meaningless semantics. Language is communication. It only exists and evolved to allow for more complex communication. Thinking of them as 2 separate things is just nonsensical.

Re: When ChatGPT broke the field of NLP: An oral history

#179

Post by ActorNightly »

tomrod wrote: post #154
ActorNightly wrote: post #124

Earlier quoted context omitted.

There was no leap in research. Everything had to do with availability of compute. Neural nets are quite old, and everyone knew that they were universal function approximators. The reason why models never took off was because it was very expensive to train a model even of a limited size. There was no real available hardware to do this on short of supercomputer clusters, which were just all cpus, and thus wildly ineffi…

> ever get like ASIC for ml Is this what you're mentioning? [0] https://linearmicrosystems.com/using-asic-chips-for-artifici...

Sort of. Those are just inference chips. You wouldn't be able to iterate architecture.

In terms of math, every single transformer can be expressed as a sequence of deep layers, so you could have an ASIC laid out in such a way where the architecture of the model depends on where you put the zeros.

Re: When ChatGPT broke the field of NLP: An oral history

#180

Post by [deleted] »

mgraczyk wrote: post #87
teruakohatu wrote: post #31

I am in academia and worked in NLP although I would describe myself as NLP adjacent. I can confirm LLMs have essentially confined a good chunk of historical research into the bin. I suspect there are probably still a few PhD students working on traditional methods knowing full well a layman can do better using the mobile ChatGPT app. That said traditional NLP has its uses. Using the VADER model for sentiment analysis…

It currently costs around $2200 to run Gemini flash lite on all of Wikipedia English. It would probably cost around 10x that much to run sentiment analysis on every Yelp review ever posted. It's true that LLMs still cost a lot for some use cases, but for essentially any business case it's not worth using traditional NLP any more

[deleted]
Post reply on HN