Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Has LLM killed traditional NLP?

medium.com

Post reply on HN
Print view View original

61–70 of 159 posts

Re: Has LLM killed traditional NLP?

#61

Post by axegon_ »

alexwebb2 wrote: post #23
axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

I think your intuition on this might be lagging a fair bit behind the current state of LLMs. System message: answer with just "service" or "product" User message (variable): 20 bottles of ferric chloride Response: product Model: OpenAI GPT-4o-mini $0.075/1Mt batch input * 27 input tokens * 10M jobs = $20.25 $0.300/1Mt batch output * 1 output token * 10M jobs = $3.00 It's a sub-$25 job. You'd need to be doing 20 times…

Yeah... Let's talk time needed for 10M prompts and how that fits into a daily pipeline. Enlighten us, please.

Re: Has LLM killed traditional NLP?

#64

Post by LeafItAlone »

IanCal wrote: post #33
LeafItAlone wrote: post #24

Earlier quoted context omitted.

>You'd need to be doing 20 times that volume every single day to even start to justify hiring an NLP engineer instead. How much for the “prompt engineer”? Who is going to be doing the work and validating the output?

Prompt engineering is less and less of an issue the simpler the job is and the more powerful the model is. You also don't need someone with deep nlp knowledge to measure and understand the output.

>less and less of an issue the simpler the job

Correct, everything is easy and simple if you make it simple and easy…

Re: Has LLM killed traditional NLP?

#65

Post by segmondy »

axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

You are not pushing it at 100. I can classify "Is 20 bottles of ferric chloride' a service or product in probably 2 seconds with a 4090. Something that most people don't realize is you can run multiple inference. So with something like a 4090, some solid few shots, and instead of having it classify one example at a time, you can do 5. We can probably run 100 parallel inference at 5 at a time. For about a rate of 250 a second on a 4090. So in 11 hours I'll be done. I'm going with a 7-8B model too. Some of the 1.5-3B models are great and will even run faster. Take a competent developer who knows python and how to use an OpenAI compatible API, they can put this together in 10-15 minutes, with no data science/scikit learn or other NLP toolchain experience.

So for personal, medium or even large workloads, I think it has killed it. It needs to be extremely large. If you are classifying or segmenting comments on a social media platform were you need to deal with billions a day, then LLM would be a very inefficient approach, but for 90+% of use cases. I think it wins.

I'm assuming you are going to run it locally because everyone is paranoid about their data. It's even cheaper if you use a cloud API.

Re: Has LLM killed traditional NLP?

#66

Post by segmondy »

elicksaur wrote: post #44
alexwebb2 wrote: post #23

Earlier quoted context omitted.

I think your intuition on this might be lagging a fair bit behind the current state of LLMs. System message: answer with just "service" or "product" User message (variable): 20 bottles of ferric chloride Response: product Model: OpenAI GPT-4o-mini $0.075/1Mt batch input * 27 input tokens * 10M jobs = $20.25 $0.300/1Mt batch output * 1 output token * 10M jobs = $3.00 It's a sub-$25 job. You'd need to be doing 20 times…

How do you validate these classifications?

The same way you validate it if you didn't use an LLM.

Re: Has LLM killed traditional NLP?

#67

Post by arandomhuman »

blindriver wrote: post #21
axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

That’s sort of like asking a horse and buggy driver whether automobiles are going to put them out of business. I think for the most part, casual nlp is dead because of LLMs. And LLM costs are going to plummet soon, so large scale nlp that you’re talking about is probably dead within 5 years or less. The fact that you can replace programmers with prompts is huge in my opinion so no one needs to learn an nlm API anymor…

>The fact that you can replace programmers with prompts

this is how you end up with 1000s of lines of slop that you have no idea how it functions.

Re: Has LLM killed traditional NLP?

#68

Post by FloorEgg »

axegon_ wrote: post #61
alexwebb2 wrote: post #23

Earlier quoted context omitted.

I think your intuition on this might be lagging a fair bit behind the current state of LLMs. System message: answer with just "service" or "product" User message (variable): 20 bottles of ferric chloride Response: product Model: OpenAI GPT-4o-mini $0.075/1Mt batch input * 27 input tokens * 10M jobs = $20.25 $0.300/1Mt batch output * 1 output token * 10M jobs = $3.00 It's a sub-$25 job. You'd need to be doing 20 times…

Yeah... Let's talk time needed for 10M prompts and how that fits into a daily pipeline. Enlighten us, please.

Run them all in parallel with a cloud function in less than a minute?

Re: Has LLM killed traditional NLP?

#69

Post by bugglebeetle »

elicksaur wrote: post #44
alexwebb2 wrote: post #23

Earlier quoted context omitted.

I think your intuition on this might be lagging a fair bit behind the current state of LLMs. System message: answer with just "service" or "product" User message (variable): 20 bottles of ferric chloride Response: product Model: OpenAI GPT-4o-mini $0.075/1Mt batch input * 27 input tokens * 10M jobs = $20.25 $0.300/1Mt batch output * 1 output token * 10M jobs = $3.00 It's a sub-$25 job. You'd need to be doing 20 times…

How do you validate these classifications?

The same way you check performance for any problem like this: by creating one or more manually-labeled test datasets, randomly sampled from the target data and looking at the resulting precision, recall, f-scores etc. LLMs change pretty much nothing about evaluation for most NLP tasks.

Re: Has LLM killed traditional NLP?

#70

Post by hnfong »

FloorEgg wrote: post #68
axegon_ wrote: post #61

Earlier quoted context omitted.

Yeah... Let's talk time needed for 10M prompts and how that fits into a daily pipeline. Enlighten us, please.

Run them all in parallel with a cloud function in less than a minute?

Obviously all the LLM API providers have a rate limit. Not a fan of GP's sarcastic tone, but I suppose many of us would like to know roughly what that limit would be for a small business using such APIs.
Post reply on HN