Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Has LLM killed traditional NLP?

medium.com

Post reply on HN
Print view View original

131–140 of 159 posts

Re: Has LLM killed traditional NLP?

#131

Post by sangnoir »

why_only_15 wrote: post #104
axegon_ wrote: post #87

Earlier quoted context omitted.

FFS... "Lots of writers, few readers". Read again and do the math: 2 seconds, multiply that by 10 million records which contain this, as well as "alarm installation in two locations" and a whole bunch of other crap with little to no repetition (<2%) and where does that get you? 2 * 10,000,000 = 20,000,000 SECONDS!!!! A day has 86,400 seconds (24 * 3600 = 86,400). The data pipeline needs to finish in <24 hours. Everyo…

Assuming the 10M records is ~2000M input tokens + 200M output tokens, this would cost $300 to classify using llama-3.3-70b[1]. If using llama lets you do this in say one day instead of two days for a traditional NLP pipeline, it's worthwhile. [1]: https://openrouter.ai/meta-llama/llama-3.3-70b-instruct

> ...two days for a traditional NLP pipeline

Why 2 days? Machine Learning took over the NLP space 10-15 years ago, so the comparison is between small, performant task-specific models versus LLMs. There is no reason to believe the "traditional" NLP pipelines are inherently slower than Large Language Models, and they aren't.

Re: Has LLM killed traditional NLP?

#132

Post by FloorEgg »

rlt wrote: post #82
FloorEgg wrote: post #68

Earlier quoted context omitted.

Run them all in parallel with a cloud function in less than a minute?

Also can’t you just combine multiple classification requests into a single prompt?

Yes, for such a simple labelling task request rate limits are more likely the bottleneck than token rate limits.

Re: Has LLM killed traditional NLP?

#133

Post by FloorEgg »

axegon_ wrote: post #117
FloorEgg wrote: post #68

Earlier quoted context omitted.

Run them all in parallel with a cloud function in less than a minute?

Yes, how did I not think of throwing more money at cloud providers on top of feeding open ai, when I could have just code a simple binary classifier and run everything on something as insignificant as an 8-th geh, quad core i5....

Did I mention openai?

Re: Has LLM killed traditional NLP?

#134

Post by llmsolutions »

axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

You can use embeddings to build classification models using various methods. Not sure what qualifies as "get help" level of cost/throughput, but certainly most providers offer large embedding APIs at much lower cost/higher throughput than their completion APIs.

Re: Has LLM killed traditional NLP?

#135

Post by fud101 »

axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

Can you recommend a way to classify a small number of objects? Local only and Python preferably.

Re: Has LLM killed traditional NLP?

#136

Post by FloorEgg »

FloorEgg wrote: post #133
axegon_ wrote: post #117

Earlier quoted context omitted.

Yes, how did I not think of throwing more money at cloud providers on top of feeding open ai, when I could have just code a simple binary classifier and run everything on something as insignificant as an 8-th geh, quad core i5....

Did I mention openai?

Ah my bad someone further up thread did.

Really it boils down to balance of time and cost, and the skill set of the person getting the job done.

But you seem really anti establishment (hung up over $25 cloud spend), so you do you.

Just don't expect everyone else to agree with you.

Re: Has LLM killed traditional NLP?

#139

Post by renewiltord »

griomnib wrote: post #129
alexwebb2 wrote: post #23

Earlier quoted context omitted.

I think your intuition on this might be lagging a fair bit behind the current state of LLMs. System message: answer with just "service" or "product" User message (variable): 20 bottles of ferric chloride Response: product Model: OpenAI GPT-4o-mini $0.075/1Mt batch input * 27 input tokens * 10M jobs = $20.25 $0.300/1Mt batch output * 1 output token * 10M jobs = $3.00 It's a sub-$25 job. You'd need to be doing 20 times…

This assumes you don’t care about our rapidly depleting carbon budget. No matter how much energy you save personally, running your jobs on Sam A’s earth killer ten thousand cluster of GPUs is literally against your own self interest of delaying climate disasters. LLM have huge negative externalities, there is a moral argument to only use them when other tools won’t work.

Haha, this is pretty good. I’m going to take a plane to SF while I laugh at this.

Re: Has LLM killed traditional NLP?

#140

Post by lyu07282 »

DebtDeflation wrote: post #18

The question seems malformed to me. Text classification, clustering, named entity recognition, etc. are NLP tasks. LLMs can perform these tasks. ML models that are not LLMs (or even not deep learning models) can also perform these tasks. Is the author perhaps asking if the concept of a "completion" has replaced all of these tasks? When I hear "traditional NLP" I think not of the above types of tasks but rather the me…

I understand the term the same way, but I don't think "traditional NLP" died with LLMs, traditional NLP already died with deep learning 7ish years ago, with LSTMs and CNNs, the gains from such models trained end-to-end are so huge it just doesn't make sense to chop up the tasks like that.
Post reply on HN