Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Has LLM killed traditional NLP?

medium.com

Post reply on HN
Print view View original

151–159 of 159 posts

Re: Has LLM killed traditional NLP?

#151

Post by mystified5016 »

segmondy wrote: post #65
axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

You are not pushing it at 100. I can classify "Is 20 bottles of ferric chloride' a service or product in probably 2 seconds with a 4090. Something that most people don't realize is you can run multiple inference. So with something like a 4090, some solid few shots, and instead of having it classify one example at a time, you can do 5. We can probably run 100 parallel inference at 5 at a time. For about a rate of 250…

At 2s per query for 10m entries, that's 251 days to run through the database.

Re: Has LLM killed traditional NLP?

#152

Post by evolve7942 »

chaos_emergent wrote: post #59
dartos wrote: post #26

Earlier quoted context omitted.

> LLM costs Inference costs, not training costs. > The fact that you can replace programmers You can’t… not for any real project. For quick mockups they’re serviceable > That’s sort of like asking a horse and buggy driver whether automobiles Kind of an insult to OP, no? Horse and buggy drivers were not highly educated experts in their field. Maybe take the word of domain experts rather than AI company marketing teams…

> Inference costs, not training costs. Why does training cost matter if you have a general intelligence that can do the task for you, that’s getting cheaper to run the task on? > for quick mockups they’re serviceable I know multiple startups that use LLMs as their core bread-and-butter intelligence platform instead of tuned but traditional NLP models > take the word of domain experts I guess? I wouldn’t call myself a…

> I know multiple startups that use LLMs as their core bread-and-butter intelligence platform instead of tuned but traditional NLP models

It seems like LLMs would be perfect for start-ups that are iterating quickly. As the business, problem, and data mature though I would expect those LLMs to be consolidated into simpler models. This makes sense from a cost and reliability perspective. I wonder also about the impact of making your core IP a set of prompts beholden to the behavior of someone else’s model.

Re: Has LLM killed traditional NLP?

#153

Post by why_only_15 »

sangnoir wrote: post #131
why_only_15 wrote: post #104

Earlier quoted context omitted.

Assuming the 10M records is ~2000M input tokens + 200M output tokens, this would cost $300 to classify using llama-3.3-70b[1]. If using llama lets you do this in say one day instead of two days for a traditional NLP pipeline, it's worthwhile. [1]: https://openrouter.ai/meta-llama/llama-3.3-70b-instruct

> ...two days for a traditional NLP pipeline Why 2 days? Machine Learning took over the NLP space 10-15 years ago, so the comparison is between small, performant task-specific models versus LLMs. There is no reason to believe the "traditional" NLP pipelines are inherently slower than Large Language Models, and they aren't.

my claim is not that it would take two days for such a pipeline to run but that it would take two days to make an NLP pipeline whereas an LLM pipeline would be faster to make.

Re: Has LLM killed traditional NLP?

#154

Post by vrighter »

segmondy wrote: post #65
axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

You are not pushing it at 100. I can classify "Is 20 bottles of ferric chloride' a service or product in probably 2 seconds with a 4090. Something that most people don't realize is you can run multiple inference. So with something like a 4090, some solid few shots, and instead of having it classify one example at a time, you can do 5. We can probably run 100 parallel inference at 5 at a time. For about a rate of 250…

two seconds is a VERY VERY VERY long time. That is mind-bogglingly, insanely slow.

Re: Has LLM killed traditional NLP?

#155

Post by tossandthrow »

leobg wrote: post #147
tossandthrow wrote: post #85

Earlier quoted context omitted.

If you think about chunking as "take x characters" then using LLMs is a poor idea. But syntactic chunking also works really poorly for any serious application as you loose basically all context. Semantic chunking, however, is a task you absolutely would use LLMs for.

If by LLM you mean embeddings I agree. Though you can often get away with using much smaller models for that. I was talking about people who actually make a call to a completion endpoint and then have the LLM repeat the input text token for token just to get the split.

How do you do semantic chunking using embeddings?

And yes, I perfectly now what you are talking about. And yes, that is a perfect strategy to chunk large texts so you can index it.

It does not sound like you are familiar with chunking and it's current issues?

Re: Has LLM killed traditional NLP?

#156

Post by griomnib »

amanaplanacanal wrote: post #142
griomnib wrote: post #129

Earlier quoted context omitted.

This assumes you don’t care about our rapidly depleting carbon budget. No matter how much energy you save personally, running your jobs on Sam A’s earth killer ten thousand cluster of GPUs is literally against your own self interest of delaying climate disasters. LLM have huge negative externalities, there is a moral argument to only use them when other tools won’t work.

It's digging fossil carbon out of the ground that's the problem, not using electricity. Switch to electricity not from fossil carbon and you're golden.

Drowning isn’t the problem; just the water.

Re: Has LLM killed traditional NLP?

#157

Post by rldjbpin »

segmondy wrote: post #65
axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

You are not pushing it at 100. I can classify "Is 20 bottles of ferric chloride' a service or product in probably 2 seconds with a 4090. Something that most people don't realize is you can run multiple inference. So with something like a 4090, some solid few shots, and instead of having it classify one example at a time, you can do 5. We can probably run 100 parallel inference at 5 at a time. For about a rate of 250…

even more naive way - just club several requests into batch of classification requests into one prompt. in practice, this is not production-ready as the llm output does not always contain results for the same number of input (sometimes more than inputted even!)

Re: Has LLM killed traditional NLP?

#158

Post by EvgeniyZh »

Vampiero wrote: post #145
gbnwl wrote: post #100

Earlier quoted context omitted.

But then they said 250/second when running multiple inference? Again I don't know if their assertions about running multiple inference are correct but why focus on the wrong number instead of addressing the actual claim?

250/s is still nothing when compared to an actual NLP pipeline that takes a few ms per it, because you can parallelize that too. I know it's hard to understand, but you can achieve a throughput that is a few orders of magnitude higher.

250/s is few (4) ms per it

Re: Has LLM killed traditional NLP?

#159

Post by hulitu »

axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

That was also my impresion. LLM can "describe" but not classify. Hallucinate but nothing precise.
Post reply on HN