Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Has LLM killed traditional NLP?

medium.com

Post reply on HN
Print view View original

121–130 of 159 posts

Re: Has LLM killed traditional NLP?

#121

Post by magic_hamster »

segmondy wrote: post #65
axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

You are not pushing it at 100. I can classify "Is 20 bottles of ferric chloride' a service or product in probably 2 seconds with a 4090. Something that most people don't realize is you can run multiple inference. So with something like a 4090, some solid few shots, and instead of having it classify one example at a time, you can do 5. We can probably run 100 parallel inference at 5 at a time. For about a rate of 250…

Yes and no. Having used these tools extensively I think it will be some time before LLMs are truly performant. Even smaller models can't be compared to running optimized code with efficient data structures. And smaller models (in general) do reduce the quality of your results in most cases. Maybe LLMs will kill off NLP and other pursuits pretty soon, but at the moment, each have their tradeoffs.

Re: Has LLM killed traditional NLP?

#122

Post by ein0p »

You have to be more specific with that question. The "traditional" NLP has by now been killed twice: first by classical machine learning (which significantly reduced the need for linguists), and now by deep learning (which has all but eliminated it). So "traditional" NLP was killed back in late 00s. You can't kill that which is not alive, so it follows that LLMs have not, in fact, killed traditional NLP.

Re: Has LLM killed traditional NLP?

#123

Post by antonvs »

One datapoint: we were using NLP to translate natural language instructions into an executable form that could drive our product. It was part of a product we sold to enterprises.

We've completely replaced that with LLMs. We still use our own DNNs for certain tasks, but not for NLP.

Re: Has LLM killed traditional NLP?

#124

Post by arcknighttech »

They also have their own uses depending in the case scenario so it depends on what someone is working with. NLP in some cases like advanced marketing can still have great benefit without an LLM and AIs in general are streamlining certain "speed of information processing" tasks but still struggle with complex "systems thinking".

FYI - If anyone doesn't know the difference between the two or has no idea what NLP or an LLM is, this has a good breakdown: https://medium.com/@melindaboone80722/nlp-vs-llm-b339abdc651...

Re: Has LLM killed traditional NLP?

#125

Post by gbnwl »

axegon_ wrote: post #110
gbnwl wrote: post #96

Earlier quoted context omitted.

Why are you using 2 seconds? The commenter you are responding to hypothesized being able to do 250/s based on "100 parallel inference at 5 at a time". Not speaking to the validity of that, but find it strange that you ran with the 2 seconds number after seemingly having stopped reading after that line, while yourself lamenting people don't read and telling them to "read again".

Ok, let me dumb it down for you: you have a cockroach in your bathroom and you want to kill it. You have an RPG and you have a slipper. Are you gonna use the RPG or are you going to use the slipper? Even if your bathroom is fine after getting shot with an RPG somehow, isn't this an overkill? If you can code and binary classifier train a classifier in 2 hours that uses nearly 0 resources and gives you good enough resu…

Sure, but this doesn't answer my question nor tie into your last comment at all. It's Saturday evening in much of the world, are you sober?

Re: Has LLM killed traditional NLP?

#126

Post by thaumasiotes »

derbaum wrote: post #109
thaumasiotes wrote: post #90

Earlier quoted context omitted.

Take two documents. Feed one through an LLM, one word at a time, and keep track of words that experience greatly inflated probabilities of occurrence, compared to baseline English. "For" is probably going to maintain a level of likelihood close to baseline. "Engine" is not. Do the same thing for the other one. See how much overlap you get.

Wouldn't a simple comparison of the word frequency in my text against a list of usual word frequencies do the trick here without an LLM? Sort of a BM25?

It might; it's not going to do the same thing. The LLM will tell you words that would likely appear in a similar text. Word frequency will tell you words that have actually appeared in your text. I'm postulating that the first kind of list is much more likely to show strong overlap between two similar documents than the second kind of list.

Vocabulary style matters a lot to what words are actually used, but much less to what words are likely to be used. If I'm following a style guide that says to use "automobile" instead of "car", appearance probabilities for "automobile" will be greatly inflated. And appearance probabilities for "car" will also be greatly inflated, just to a lesser extent than for "automobile". Whereas actual usage of "car" will be pegged at zero.

Determining how similar two texts are is something that an LLM should be good at. It should be better than a simple comparison of word frequency. Whether it's better enough to justify the extra compute is a different question.

Re: Has LLM killed traditional NLP?

#127

Post by pona-a »

simonw wrote: post #71
axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

What NLP approaches are you using to solve the "is '20 bottles of ferric chloride' a service or a product?" problem?

How about a naive Bayesian Bag of Words? Just find/scrape/generate with an LLM a large enough corpus of products/services, build the term frequency matrix, calculate class priors and P(term|class) and inference with straightforward application of Bayes theorem.

This particular problem, at least to me, seems trivial, and to use an LLM for anything like this for more than a hundred cases seems incredibly wasteful.

Re: Has LLM killed traditional NLP?

#128

Post by griomnib »

segmondy wrote: post #65
axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

You are not pushing it at 100. I can classify "Is 20 bottles of ferric chloride' a service or product in probably 2 seconds with a 4090. Something that most people don't realize is you can run multiple inference. So with something like a 4090, some solid few shots, and instead of having it classify one example at a time, you can do 5. We can probably run 100 parallel inference at 5 at a time. For about a rate of 250…

Or you can build a DistilBERT model and get your egregiously inefficient 2 seconds down to tens of milliseconds.

Re: Has LLM killed traditional NLP?

#129

Post by griomnib »

alexwebb2 wrote: post #23
axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

I think your intuition on this might be lagging a fair bit behind the current state of LLMs. System message: answer with just "service" or "product" User message (variable): 20 bottles of ferric chloride Response: product Model: OpenAI GPT-4o-mini $0.075/1Mt batch input * 27 input tokens * 10M jobs = $20.25 $0.300/1Mt batch output * 1 output token * 10M jobs = $3.00 It's a sub-$25 job. You'd need to be doing 20 times…

This assumes you don’t care about our rapidly depleting carbon budget.

No matter how much energy you save personally, running your jobs on Sam A’s earth killer ten thousand cluster of GPUs is literally against your own self interest of delaying climate disasters.

LLM have huge negative externalities, there is a moral argument to only use them when other tools won’t work.

Re: Has LLM killed traditional NLP?

#130

Post by griomnib »

mikeocool wrote: post #75
segmondy wrote: post #65

Earlier quoted context omitted.

You are not pushing it at 100. I can classify "Is 20 bottles of ferric chloride' a service or product in probably 2 seconds with a 4090. Something that most people don't realize is you can run multiple inference. So with something like a 4090, some solid few shots, and instead of having it classify one example at a time, you can do 5. We can probably run 100 parallel inference at 5 at a time. For about a rate of 250…

If you have to classify user input as they’re inputting it to provide a response — so it can’t be batched - 2 seconds could potentially be really slow. Though LLMs sure have made creating training data to train old school models for those cases a lot easier.

Yeah, that’s what I do: use LLM to help make training data for small models. It’s ao much more efficient, fast, and ergo, scalable.
Post reply on HN