Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Has LLM killed traditional NLP?

medium.com

Post reply on HN
Print view View original

91–100 of 159 posts

Re: Has LLM killed traditional NLP?

#91

Post by scarface_74 »

LeafItAlone wrote: post #84
scarface_74 wrote: post #14

For my use case, definitely. I have worked on AWS Connect (online call center) and Amazon Lex (the backing NLP engine) projects. Before LLMs, it was a tedious process of trying to figure out all of the different “utterances” that people could say and the various languages you had to support. With LLMs, it’s just prompting https://chatgpt.com/share/678bab08-f3a0-8010-82e0-32cff9c0b4... I used something like this using…

That’s a great example and I understand it was intentionally simple but highlighted how LLMs need care with use. Not that this example is very related to NLP: My prompt: ` >` The response: ``` { "origin": ["PDX"], "destination": ["HAV"], "date": "2025-04-01", "departure_time": null, "preferences": null } ``` Of course I meant Portland Maine (PWM), there is more than one airport option in Cuba than HAV, and it got the…

If the business stakeholders came out with that scenario, I would modify the prompt like this. You would know the users address if they had an account.

https://chatgpt.com/share/678c1708-639c-8010-a6be-9ce1055703...

Re: Has LLM killed traditional NLP?

#92

Post by LeafItAlone »

scarface_74 wrote: post #91
LeafItAlone wrote: post #84

Earlier quoted context omitted.

That’s a great example and I understand it was intentionally simple but highlighted how LLMs need care with use. Not that this example is very related to NLP: My prompt: ` >` The response: ``` { "origin": ["PDX"], "destination": ["HAV"], "date": "2025-04-01", "departure_time": null, "preferences": null } ``` Of course I meant Portland Maine (PWM), there is more than one airport option in Cuba than HAV, and it got the…

If the business stakeholders came out with that scenario, I would modify the prompt like this. You would know the users address if they had an account. https://chatgpt.com/share/678c1708-639c-8010-a6be-9ce1055703...

OK, but that only fixed one of the three issues.

Re: Has LLM killed traditional NLP?

#93

Post by Palmik »

WildGreenLeave wrote: post #80
segmondy wrote: post #65

Earlier quoted context omitted.

You are not pushing it at 100. I can classify "Is 20 bottles of ferric chloride' a service or product in probably 2 seconds with a 4090. Something that most people don't realize is you can run multiple inference. So with something like a 4090, some solid few shots, and instead of having it classify one example at a time, you can do 5. We can probably run 100 parallel inference at 5 at a time. For about a rate of 250…

Correct me if I'm wrong, but, if you run multiple inferences at the same time on the same GPU you will need load multiple models in the vram and the models will fight for resources right? So running 10 parallel inferences will slow everything down 5 times right? Or am I missing something?

Inference for single example is memory bound. By doing batch inference, you can interleave computation with memory loads, without losing much speed (up until you cross the compute bound threshold).

Re: Has LLM killed traditional NLP?

#94

Post by macNchz »

derbaum wrote: post #88

One of the things I'm still struggling with when using LLMs over NLP is classification against a large corpus of data. If I get a new text and I want to find the most similar text out of a million others, semantically speaking, how would I do this with an LLM? Apart from choosing certain pre-defined categories (such as "friendly", "political", ...) and then letting the LLM rate each text on each category, I can't see…

I've used embeddings to define clusters, then passed sampled documents from each cluster to an LLM to create labels for each grouping. I had pretty impressive results from this approach when creating a category/subcategory labels for a collection of texts I worked on recently.

Re: Has LLM killed traditional NLP?

#95

Post by scarface_74 »

LeafItAlone wrote: post #92
scarface_74 wrote: post #91

Earlier quoted context omitted.

If the business stakeholders came out with that scenario, I would modify the prompt like this. You would know the users address if they had an account. https://chatgpt.com/share/678c1708-639c-8010-a6be-9ce1055703...

OK, but that only fixed one of the three issues.

While the first one is easy. I mean you could give it a list of holidays and dates. But the rest you would just ask the user to confirm the information and say “is this correct”? If they say “No” ask them which isn’t correct and let them correct it.

I would definitely assume someone wanted to leave from an airport close by if they didn’t say anything.

You don’t want the prompt to grow too much. But you do have analytics that you can use to improve your prompt.

In the case of Connect, you define your logic using a GUi flowchart builder called a contact flow.

BTW: with my new prompt, it did assume the correct airport “>”

Re: Has LLM killed traditional NLP?

#96

Post by gbnwl »

axegon_ wrote: post #87
segmondy wrote: post #65

Earlier quoted context omitted.

You are not pushing it at 100. I can classify "Is 20 bottles of ferric chloride' a service or product in probably 2 seconds with a 4090. Something that most people don't realize is you can run multiple inference. So with something like a 4090, some solid few shots, and instead of having it classify one example at a time, you can do 5. We can probably run 100 parallel inference at 5 at a time. For about a rate of 250…

FFS... "Lots of writers, few readers". Read again and do the math: 2 seconds, multiply that by 10 million records which contain this, as well as "alarm installation in two locations" and a whole bunch of other crap with little to no repetition (<2%) and where does that get you? 2 * 10,000,000 = 20,000,000 SECONDS!!!! A day has 86,400 seconds (24 * 3600 = 86,400). The data pipeline needs to finish in <24 hours. Everyo…

Why are you using 2 seconds? The commenter you are responding to hypothesized being able to do 250/s based on "100 parallel inference at 5 at a time". Not speaking to the validity of that, but find it strange that you ran with the 2 seconds number after seemingly having stopped reading after that line, while yourself lamenting people don't read and telling them to "read again".

Re: Has LLM killed traditional NLP?

#97

Post by jazzyjackson »

gbnwl wrote: post #96
axegon_ wrote: post #87

Earlier quoted context omitted.

FFS... "Lots of writers, few readers". Read again and do the math: 2 seconds, multiply that by 10 million records which contain this, as well as "alarm installation in two locations" and a whole bunch of other crap with little to no repetition (<2%) and where does that get you? 2 * 10,000,000 = 20,000,000 SECONDS!!!! A day has 86,400 seconds (24 * 3600 = 86,400). The data pipeline needs to finish in <24 hours. Everyo…

Why are you using 2 seconds? The commenter you are responding to hypothesized being able to do 250/s based on "100 parallel inference at 5 at a time". Not speaking to the validity of that, but find it strange that you ran with the 2 seconds number after seemingly having stopped reading after that line, while yourself lamenting people don't read and telling them to "read again".

OP said 2 seconds as if that wasn't an eternity...

Re: Has LLM killed traditional NLP?

#98

Post by LeafItAlone »

scarface_74 wrote: post #95
LeafItAlone wrote: post #92

Earlier quoted context omitted.

OK, but that only fixed one of the three issues.

While the first one is easy. I mean you could give it a list of holidays and dates. But the rest you would just ask the user to confirm the information and say “is this correct”? If they say “No” ask them which isn’t correct and let them correct it. I would definitely assume someone wanted to leave from an airport close by if they didn’t say anything. You don’t want the prompt to grow too much. But you do have analyt…

Sure, all the problems are “easy” once you identify them. As with most products. But the majority of Show HN posts here relying on LLMs that I see don’t account for simple things like my example. Flights finders in particular have been pretty bad.

>BTW: with my new prompt, it did assume the correct airport “>”

Not really. It chose the airport you put basically in the prompt. But I don’t live in MA, I live closer to PDX. And it didn’t suggest the multiple other Cuba airports. So you’ll end up with a lot of guiding rules.

Re: Has LLM killed traditional NLP?

#99

Post by dartos »

blindriver wrote: post #53
dartos wrote: post #39

Earlier quoted context omitted.

> Appeal to authority is a well known logical fallacy. I did not make an appeal to authority. I made an appeal to expertise. It’s why you’d trust a doctor’s medical opinion over a child’s. I’m not saying “listen to this guy because their captain of NLP” I’m saying listen because experts have spent years of hands on experience with things like getting NLP working at all. > I know how dead NLP is personally because I’v…

> I did not make an appeal to authority. I made an appeal to expertise. Lol > I’m saying listen because experts have spent years of hands on experience with things like getting NLP working at all. “It is difficult to get a man to understand something, when his salary depends on his not understanding it.” Upton Sinclair > Barely know anything about it, but you’re okay hand waving away expertise bc you got a toy NLP De…

> Lol

Great argument

> “It is difficult to get a man to understand something, when his salary depends on his not understanding it.”

NLP professionals are also LLM professionals. LLMs are tools in an NLP toolkit. LLMs don’t make the NLP professional obsolete the way it makes handwritten spam obsolete.

I was going to explain this further but you literally wouldn’t understand.

> Do you cross reference all your Google searches to make sure they are giving you the best results vs Bing and DDG?

…Yes I do…

That’s why I cancelled my kagi subscription. It was just as good as DDG.

> Do you cross reference the results from your NLP with LLMs to see if there were any improvements?

Yes I do… because I want to use the best tool for the job. Not just the first one I was able to get working…

Re: Has LLM killed traditional NLP?

#100

Post by gbnwl »

jazzyjackson wrote: post #97
gbnwl wrote: post #96

Earlier quoted context omitted.

Why are you using 2 seconds? The commenter you are responding to hypothesized being able to do 250/s based on "100 parallel inference at 5 at a time". Not speaking to the validity of that, but find it strange that you ran with the 2 seconds number after seemingly having stopped reading after that line, while yourself lamenting people don't read and telling them to "read again".

OP said 2 seconds as if that wasn't an eternity...

But then they said 250/second when running multiple inference? Again I don't know if their assertions about running multiple inference are correct but why focus on the wrong number instead of addressing the actual claim?
Post reply on HN