Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Has LLM killed traditional NLP?

medium.com

Post reply on HN
Print view View original

51–60 of 159 posts

Re: Has LLM killed traditional NLP?

#51

Post by elicksaur »

scarface_74 wrote: post #14

For my use case, definitely. I have worked on AWS Connect (online call center) and Amazon Lex (the backing NLP engine) projects. Before LLMs, it was a tedious process of trying to figure out all of the different “utterances” that people could say and the various languages you had to support. With LLMs, it’s just prompting https://chatgpt.com/share/678bab08-f3a0-8010-82e0-32cff9c0b4... I used something like this using…

Thank you for sharing an actual prompt thread. So much of the LLM debate is washed in biases, and it is very helpful to share concrete examples of outputs.

Re: Has LLM killed traditional NLP?

#52

Post by jeswin »

elicksaur wrote: post #44
alexwebb2 wrote: post #23

Earlier quoted context omitted.

I think your intuition on this might be lagging a fair bit behind the current state of LLMs. System message: answer with just "service" or "product" User message (variable): 20 bottles of ferric chloride Response: product Model: OpenAI GPT-4o-mini $0.075/1Mt batch input * 27 input tokens * 10M jobs = $20.25 $0.300/1Mt batch output * 1 output token * 10M jobs = $3.00 It's a sub-$25 job. You'd need to be doing 20 times…

How do you validate these classifications?

Isn't it easier and cheaper to validate than to classify (requires expensive engineers)? I mean the skill is not as expensive - many companies do this at scale.

Re: Has LLM killed traditional NLP?

#53

Post by blindriver »

dartos wrote: post #39
blindriver wrote: post #29

Earlier quoted context omitted.

> Maybe take the word of domain experts rather than AI company marketing teams. Appeal to authority is a well known logical fallacy. I know how dead NLP is personally because I’ve never been able to get NLP working but once ChatGPT came around, I was able to classify texts extremely easily. It’s transformational. I was able to get ChatGPT to classify posts based on how political it was from a scale of 1 to 10 and whi…

> Appeal to authority is a well known logical fallacy. I did not make an appeal to authority. I made an appeal to expertise. It’s why you’d trust a doctor’s medical opinion over a child’s. I’m not saying “listen to this guy because their captain of NLP” I’m saying listen because experts have spent years of hands on experience with things like getting NLP working at all. > I know how dead NLP is personally because I’v…

> I did not make an appeal to authority. I made an appeal to expertise.

Lol

> I’m saying listen because experts have spent years of hands on experience with things like getting NLP working at all.

“It is difficult to get a man to understand something, when his salary depends on his not understanding it.”

Upton Sinclair

> Barely know anything about it, but you’re okay hand waving away expertise bc you got a toy NLP Demo working…

Yes that’s my point. I don’t know anything about implementing an NLP but got something that works pretty well using an LLM extremely quickly and easily.

> And I know you didn’t compare the results against classic NLP to see if there was any improvements because you don’t know NLP…

Do you cross reference all your Google searches to make sure they are giving you the best results vs Bing and DDG?

Do you cross reference the results from your NLP with LLMs to see if there were any improvements?

Re: Has LLM killed traditional NLP?

#54

Post by scarface_74 »

elicksaur wrote: post #44
alexwebb2 wrote: post #23

Earlier quoted context omitted.

I think your intuition on this might be lagging a fair bit behind the current state of LLMs. System message: answer with just "service" or "product" User message (variable): 20 bottles of ferric chloride Response: product Model: OpenAI GPT-4o-mini $0.075/1Mt batch input * 27 input tokens * 10M jobs = $20.25 $0.300/1Mt batch output * 1 output token * 10M jobs = $3.00 It's a sub-$25 job. You'd need to be doing 20 times…

How do you validate these classifications?

You need a domain expert either way. I mentioned in another reply that one of my niches is implementing call centers with Amazon Connect and Amazon Lex (the NLP engine).

https://news.ycombinator.com/item?id=42748189

I don’t know the domain beforehand they are working in, I do validation testing with them.

Re: Has LLM killed traditional NLP?

#55

Post by blindriver »

elicksaur wrote: post #41
blindriver wrote: post #29

Earlier quoted context omitted.

> Maybe take the word of domain experts rather than AI company marketing teams. Appeal to authority is a well known logical fallacy. I know how dead NLP is personally because I’ve never been able to get NLP working but once ChatGPT came around, I was able to classify texts extremely easily. It’s transformational. I was able to get ChatGPT to classify posts based on how political it was from a scale of 1 to 10 and whi…

I haven’t understood these types of uses. How do you validate the score that the LLM gives?

The same way you validate scores given by NLPs I assume. You run various tests and look at the results and see if they match what you would expect.

Re: Has LLM killed traditional NLP?

#56

Post by scarface_74 »

FridgeSeal wrote: post #37
blindriver wrote: post #29

Earlier quoted context omitted.

> Maybe take the word of domain experts rather than AI company marketing teams. Appeal to authority is a well known logical fallacy. I know how dead NLP is personally because I’ve never been able to get NLP working but once ChatGPT came around, I was able to classify texts extremely easily. It’s transformational. I was able to get ChatGPT to classify posts based on how political it was from a scale of 1 to 10 and whi…

“I couldn’t be bothered learning something, and now I don’t have to! Checkmate!” While LLM’s can have their uses, let’s not get carried away.

That’s true. I did avoid learning traditional NLP techniques because for my use case - call centers - LLMs do a much better job.

Context for the problem space:

https://dl.acm.org/doi/fullHtml/10.1145/3442381.3449870

Re: Has LLM killed traditional NLP?

#57

Post by WhitneyLand »

axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

For context, 10M would cost ~$27.

Say Gemini Flash 8B, allowing ~28 tokens for prompt input at $0.075/1M tokens, plus 2 output tokens at $0.30/1M. Works out to $0.0027 per classification. Or in other words, for 1 penny you could do this classification 3.7 times.

Re: Has LLM killed traditional NLP?

#58

Post by scarface_74 »

elicksaur wrote: post #51
scarface_74 wrote: post #14

For my use case, definitely. I have worked on AWS Connect (online call center) and Amazon Lex (the backing NLP engine) projects. Before LLMs, it was a tedious process of trying to figure out all of the different “utterances” that people could say and the various languages you had to support. With LLMs, it’s just prompting https://chatgpt.com/share/678bab08-f3a0-8010-82e0-32cff9c0b4... I used something like this using…

Thank you for sharing an actual prompt thread. So much of the LLM debate is washed in biases, and it is very helpful to share concrete examples of outputs.

The “cordele GA” example surprised me. I was expecting to get a value of “null” for the airport code since I knew that city had a population of 12K and no airport within its metropolitan statistical area. It returned an airport that was close.

Having world knowledge is a godsend. I also just tried a prompt with “Alpharetta, GA” a city north of Atlanta and it returned ATL. An NLP could never do that without a lot more work.

Re: Has LLM killed traditional NLP?

#59

Post by chaos_emergent »

dartos wrote: post #26
blindriver wrote: post #21

Earlier quoted context omitted.

That’s sort of like asking a horse and buggy driver whether automobiles are going to put them out of business. I think for the most part, casual nlp is dead because of LLMs. And LLM costs are going to plummet soon, so large scale nlp that you’re talking about is probably dead within 5 years or less. The fact that you can replace programmers with prompts is huge in my opinion so no one needs to learn an nlm API anymor…

> LLM costs Inference costs, not training costs. > The fact that you can replace programmers You can’t… not for any real project. For quick mockups they’re serviceable > That’s sort of like asking a horse and buggy driver whether automobiles Kind of an insult to OP, no? Horse and buggy drivers were not highly educated experts in their field. Maybe take the word of domain experts rather than AI company marketing teams…

> Inference costs, not training costs.

Why does training cost matter if you have a general intelligence that can do the task for you, that’s getting cheaper to run the task on?

> for quick mockups they’re serviceable

I know multiple startups that use LLMs as their core bread-and-butter intelligence platform instead of tuned but traditional NLP models

> take the word of domain experts

I guess? I wouldn’t call myself an expert by any means but I’ve been working on NLP problems for about 5 years. Most people I know in NLP-adjacent fields have converged around LLMs being good for most (but obviously not all) problems.

> kind of an insult

Depends on whether you think OP intended to offend, ig

Re: Has LLM killed traditional NLP?

#60

Post by otabdeveloper4 »

blindriver wrote: post #38
otabdeveloper4 wrote: post #31

Earlier quoted context omitted.

> The fact that you can replace programmers with prompts No, you can't. The only thing LLM's replace is internet commentators.

As I explained below, I avoided having to learn anything about ML, PyTorch or any other APIs when trying to classify posts based on how political they were and which affiliation they were. That was holding me back and it was easily replaced by an llm and a prompt. Literally took me minutes what would have taken days or weeks and the results are more than good enough.

> what would have taken days or weeks

Nah, searching Stackoverflow and Github doesn't take "weeks".

That said, due to how utterly broken internet search is nowadays, using an LLM as a search engine proxy is viable.

Post reply on HN