Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Has LLM killed traditional NLP?

medium.com

Post reply on HN
Print view View original

111–120 of 159 posts

Re: Has LLM killed traditional NLP?

#111

Post by gf000 »

axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

Why not just run a local LLM for practically free? You can even trivially parallelize it with multiple instances.

I would believe that many NLP problems can be easily solved even by smaller LLM models.

Re: Has LLM killed traditional NLP?

#112

Post by vpribish »

scarface_74 wrote: post #14

For my use case, definitely. I have worked on AWS Connect (online call center) and Amazon Lex (the backing NLP engine) projects. Before LLMs, it was a tedious process of trying to figure out all of the different “utterances” that people could say and the various languages you had to support. With LLMs, it’s just prompting https://chatgpt.com/share/678bab08-f3a0-8010-82e0-32cff9c0b4... I used something like this using…

link is a 404, sadly. what did it say before?

Re: Has LLM killed traditional NLP?

#113

Post by gf000 »

w10-1 wrote: post #78
alexwebb2 wrote: post #23

Earlier quoted context omitted.

I think your intuition on this might be lagging a fair bit behind the current state of LLMs. System message: answer with just "service" or "product" User message (variable): 20 bottles of ferric chloride Response: product Model: OpenAI GPT-4o-mini $0.075/1Mt batch input * 27 input tokens * 10M jobs = $20.25 $0.300/1Mt batch output * 1 output token * 10M jobs = $3.00 It's a sub-$25 job. You'd need to be doing 20 times…

The question is not average cost but marginal cost of quality - same as voice recognition, which had relatively low uptake even at ~2-4% error rates due to context switching costs for error correction. So you'd have to account for the work of catching the residue of 2-8%+ error from LLMs. I believe the premise is for NLP, that's just incremental work, but for LLM's that could be impossible to correct (i.e., cost per…

I am absolutely not an expert in NLP, but I wouldn't be surprised if for many kinds of problems LLMs would have far less error rate, than any NLP software.

Like, lemmation is pretty damn dumb in NLP, while a better LLM model will be orders of magnitude more correct.

Re: Has LLM killed traditional NLP?

#114

Post by sireat »

axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

So what would you use to classify whether a document is a critique or something else in 1M documents in a non-English language?

This is a real problem I am dealing with at a library project.

Each document is between 100 to 10k tokens.

Most top (read most expensive) LLMs available in OpenRouter work great, it is the cost (and speed) that is the issue.

If I could come up with something locally runnable that would be fantastic.

Presumably BERT based classifiers would work if I had one properly trained for the language.

Re: Has LLM killed traditional NLP?

#115

Post by scarface_74 »

vpribish wrote: post #112
scarface_74 wrote: post #14

For my use case, definitely. I have worked on AWS Connect (online call center) and Amazon Lex (the backing NLP engine) projects. Before LLMs, it was a tedious process of trying to figure out all of the different “utterances” that people could say and the various languages you had to support. With LLMs, it’s just prompting https://chatgpt.com/share/678bab08-f3a0-8010-82e0-32cff9c0b4... I used something like this using…

link is a 404, sadly. what did it say before?

The link works for me even in cognito mode.

The prompt:

you are a chatbot that helps users book flights. Please extract the origin city, destination city, travel date, and any additional preferences (e.g., time of day, class of service). If any of the details are missing, make the value “null”. If the date is relative (e.g., "tomorrow", "next week"), convert it to a specific date.

User Input: ""

Output (JSON format): { "origin": list of airport codes "destination": list of airport codes, "date": "", "departure_time": "", "preferences": "" }

The users request will be surrounded by >

Always return JSON with any missing properties having a value of null. Always return English. Return a list of airport codes for the city. For instance New York has two airports give both

Always return responses in English

Re: Has LLM killed traditional NLP?

#116

Post by gf000 »

vunderba wrote: post #77
blindriver wrote: post #29

Earlier quoted context omitted.

> Maybe take the word of domain experts rather than AI company marketing teams. Appeal to authority is a well known logical fallacy. I know how dead NLP is personally because I’ve never been able to get NLP working but once ChatGPT came around, I was able to classify texts extremely easily. It’s transformational. I was able to get ChatGPT to classify posts based on how political it was from a scale of 1 to 10 and whi…

> NLPs are dead in the water right now, except in terms of cost. False. With all due respect, the fact that you're referring to natural language parsing as "NLPs" makes me question whether you have any experience or modest knowledge around this topic, so it's rather bold of you to make such sweeping generalizations. It works for your use case because you're just one person running it on your home computer with consum…

My understanding is that inference models can absolutely scale down, we are only at the beginning of these getting minimized, and they are trivial to parallelize. That's not a good combo to be against them, their price/performance/efficiency will quickly drop/grow/grow.

Re: Has LLM killed traditional NLP?

#117

Post by axegon_ »

FloorEgg wrote: post #68
axegon_ wrote: post #61

Earlier quoted context omitted.

Yeah... Let's talk time needed for 10M prompts and how that fits into a daily pipeline. Enlighten us, please.

Run them all in parallel with a cloud function in less than a minute?

Yes, how did I not think of throwing more money at cloud providers on top of feeding open ai, when I could have just code a simple binary classifier and run everything on something as insignificant as an 8-th geh, quad core i5....

Re: Has LLM killed traditional NLP?

#118

Post by rahimnathwani »

sireat wrote: post #114
axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

So what would you use to classify whether a document is a critique or something else in 1M documents in a non-English language? This is a real problem I am dealing with at a library project. Each document is between 100 to 10k tokens. Most top (read most expensive) LLMs available in OpenRouter work great, it is the cost (and speed) that is the issue. If I could come up with something locally runnable that would be fa…

I guess you've already seen https://huggingface.co/collections/answerdotai/modernbert-67... ?

Re: Has LLM killed traditional NLP?

#119

Post by gf000 »

thangalin wrote: post #86

I created an NLP library to help curl straight quotes into curly quotes. Last I checked, LLMs struggled to curl the following straight quotation marks: ''E's got a 'ittle box 'n a big 'un,' she said, 'wit' th' 'ittle 'un 'bout 2'×6". An' no, y'ain't cryin' on th' "soap box" to me no mo, y'hear. 'Cause it 'tweren't ever a spec o' fun!' I says to my frien'. The library is integrated into my Markdown editor, KeenWrite (…

I would be interested how well would even a smaller LLM model work after fine tuning. Besides the overhead of an LLM, I would assume they would do a much better job at it in the edge cases (where contextual understanding is required).

Re: Has LLM killed traditional NLP?

#120

Post by IanCal »

LeafItAlone wrote: post #64
IanCal wrote: post #33

Earlier quoted context omitted.

Prompt engineering is less and less of an issue the simpler the job is and the more powerful the model is. You also don't need someone with deep nlp knowledge to measure and understand the output.

>less and less of an issue the simpler the job Correct, everything is easy and simple if you make it simple and easy…

Plenty of simple jobs required people with deeper knowledge of AI in the past, now for many tasks in businesses you can skip over a lot of that and use a llm.

Simple things were not always easy. Many of them are, now.

Post reply on HN