Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Has LLM killed traditional NLP?

medium.com

Post reply on HN
Print view View original

141–150 of 159 posts

Re: Has LLM killed traditional NLP?

#141

Post by specproc »

axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

I see LLMs best used as part of a more traditional NLP pipeline.

For example, an approach that does me well is clustering then using LLMs on representative docs. Tools like bertopic are great for this.

I also don't see a clear cut difference between the two in certain areas. Embeddings are critical in LLM pipelines, but for me anyway, also "old school" tools.

I think NLP as described in the article is certainly under threat, but the tools and approaches compliment LLM use well, are far more efficient, and distinguish the pros from the neophytes.

If you're using LLMs for NLP-type tasks, but don't know the NLP tools, you're missing out.

Re: Has LLM killed traditional NLP?

#142

Post by amanaplanacanal »

griomnib wrote: post #129
alexwebb2 wrote: post #23

Earlier quoted context omitted.

I think your intuition on this might be lagging a fair bit behind the current state of LLMs. System message: answer with just "service" or "product" User message (variable): 20 bottles of ferric chloride Response: product Model: OpenAI GPT-4o-mini $0.075/1Mt batch input * 27 input tokens * 10M jobs = $20.25 $0.300/1Mt batch output * 1 output token * 10M jobs = $3.00 It's a sub-$25 job. You'd need to be doing 20 times…

This assumes you don’t care about our rapidly depleting carbon budget. No matter how much energy you save personally, running your jobs on Sam A’s earth killer ten thousand cluster of GPUs is literally against your own self interest of delaying climate disasters. LLM have huge negative externalities, there is a moral argument to only use them when other tools won’t work.

It's digging fossil carbon out of the ground that's the problem, not using electricity. Switch to electricity not from fossil carbon and you're golden.

Re: Has LLM killed traditional NLP?

#143

Post by fbilhaut »

Olfurm wrote: post #30
selimthegrim wrote: post #27

Earlier quoted context omitted.

Like?

Like named entity recognition or relations recognition. Check https://github.com/urchade/GLiNER

Indeed. What's is interesting (and not so common) with models like GLiNER is that it is way lighter than LLMs while preserving good (if not better) quality and some zero-shot ability (in this case wrt to the entity classes). This feature is very significant, as most "traditional" (including Transformer-based) approaches are more or less supervised during fine-tuning. Icing on the cake, you don't have to deal with all the problems that arise when you want to get structured output from an LLM (in this case structured NER output).

Re: Has LLM killed traditional NLP?

#144

Post by Vampiero »

axegon_ wrote: post #110
gbnwl wrote: post #96

Earlier quoted context omitted.

Why are you using 2 seconds? The commenter you are responding to hypothesized being able to do 250/s based on "100 parallel inference at 5 at a time". Not speaking to the validity of that, but find it strange that you ran with the 2 seconds number after seemingly having stopped reading after that line, while yourself lamenting people don't read and telling them to "read again".

Ok, let me dumb it down for you: you have a cockroach in your bathroom and you want to kill it. You have an RPG and you have a slipper. Are you gonna use the RPG or are you going to use the slipper? Even if your bathroom is fine after getting shot with an RPG somehow, isn't this an overkill? If you can code and binary classifier train a classifier in 2 hours that uses nearly 0 resources and gives you good enough resu…

This thread is chock full of people who have no clue about what traditional AI even is. I'm sorry you have to deal with literal children

Re: Has LLM killed traditional NLP?

#145

Post by Vampiero »

gbnwl wrote: post #100
jazzyjackson wrote: post #97

Earlier quoted context omitted.

OP said 2 seconds as if that wasn't an eternity...

But then they said 250/second when running multiple inference? Again I don't know if their assertions about running multiple inference are correct but why focus on the wrong number instead of addressing the actual claim?

250/s is still nothing when compared to an actual NLP pipeline that takes a few ms per it, because you can parallelize that too.

I know it's hard to understand, but you can achieve a throughput that is a few orders of magnitude higher.

Re: Has LLM killed traditional NLP?

#146

Post by Vampiero »

vletal wrote: post #106

The idea that we can solve "language" by breaking down and understanding sentences is naive and funny with the benefit of hindsight, is it not? An equivalently funny attitude seems to be the "natural language will replace programming languages". Let's see how that one will work out when the hype is over.

It's not naive, it's how languages work.

That grammar doesn't necessarily convey 100% of semantics is a problem of natural language. Or rather, of people being poor at communicating unambiguously.

Programming languages can also be ambiguous sometimes, but that ambiguity is resolved before execution by essentially following a priority list or throwing an error if no combination of rules fits.

Re: Has LLM killed traditional NLP?

#147

Post by leobg »

tossandthrow wrote: post #85
leobg wrote: post #49

There are AI bros that will call an LLM to do what you could do with a regex. I’ve seen people do the chunking for RAG using an LLM…

If you think about chunking as "take x characters" then using LLMs is a poor idea. But syntactic chunking also works really poorly for any serious application as you loose basically all context. Semantic chunking, however, is a task you absolutely would use LLMs for.

If by LLM you mean embeddings I agree. Though you can often get away with using much smaller models for that.

I was talking about people who actually make a call to a completion endpoint and then have the LLM repeat the input text token for token just to get the split.

Re: Has LLM killed traditional NLP?

#148

Post by Pompidou »

vletal wrote: post #106

The idea that we can solve "language" by breaking down and understanding sentences is naive and funny with the benefit of hindsight, is it not? An equivalently funny attitude seems to be the "natural language will replace programming languages". Let's see how that one will work out when the hype is over.

They all fall into the pitfall that Martin Heidegger suggested avoiding, particularly in the introduction of his text What Is a Thing?.

Re: Has LLM killed traditional NLP?

#149

Post by anon373839 »

crystal_revenge wrote: post #89
axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

> LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. What? That's simply not true. Current embedding models are incredibly fast and cheap and will, in the vast majority of NLP tasks, get you far better results than any local set of features you can develop yourself. I've also done this at work numerous times, and have been working on v…

Embedding models are not LLMs in the sense that the term is being used in the title of this post. They are “traditional NLP.”

Re: Has LLM killed traditional NLP?

#150

Post by HarHarVeryFunny »

axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

10M items @ 10 tokens each ("20 bottles of ferric chloride" etc) plus 10M tokens out (category) is 100M tokens in 10M tokens out.

Claude Haiku is $0.25 per 1M tokens in, $1.05 per 1M out, so cost would be ~$35.

GPT-4o mini is even cheaper at $0.15 per 1M in.

Of course if your volume justifies the hardware cost you could always run Llama locally, for the cost of the electricity used.

Post reply on HN