Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Has LLM killed traditional NLP?

medium.com

Post reply on HN
Print view View original

101–110 of 159 posts

Re: Has LLM killed traditional NLP?

#101

Post by dartos »

chaos_emergent wrote: post #59
dartos wrote: post #26

Earlier quoted context omitted.

> LLM costs Inference costs, not training costs. > The fact that you can replace programmers You can’t… not for any real project. For quick mockups they’re serviceable > That’s sort of like asking a horse and buggy driver whether automobiles Kind of an insult to OP, no? Horse and buggy drivers were not highly educated experts in their field. Maybe take the word of domain experts rather than AI company marketing teams…

> Inference costs, not training costs. Why does training cost matter if you have a general intelligence that can do the task for you, that’s getting cheaper to run the task on? > for quick mockups they’re serviceable I know multiple startups that use LLMs as their core bread-and-butter intelligence platform instead of tuned but traditional NLP models > take the word of domain experts I guess? I wouldn’t call myself a…

> Why does training cost matter if you have a general intelligence that can do the task for you, that’s getting cheaper to run the task on?

Assuming we didn’t need to train it ever again, it wouldn’t. But we don’t have that, so…

> I know multiple startups that use LLMs as their core bread-and-butter intelligence platform instead of tuned but traditional NLP models

Okay? Did that system write itself entirely? Did it replace the programmers that actually made it?

If so, they should pivot into a Devin competitor.

> Most people I know in NLP-adjacent fields have converged around LLMs being good for most (but obviously not all) problems.

Yeah LLMs are quite good at comming NLP tasks, but AFAIK are not SOTA at any specific task.

Either way, LLMs obviously don’t kill the need for the NLP field.

Re: Has LLM killed traditional NLP?

#102

Post by scarface_74 »

LeafItAlone wrote: post #98
scarface_74 wrote: post #95

Earlier quoted context omitted.

While the first one is easy. I mean you could give it a list of holidays and dates. But the rest you would just ask the user to confirm the information and say “is this correct”? If they say “No” ask them which isn’t correct and let them correct it. I would definitely assume someone wanted to leave from an airport close by if they didn’t say anything. You don’t want the prompt to grow too much. But you do have analyt…

Sure, all the problems are “easy” once you identify them. As with most products. But the majority of Show HN posts here relying on LLMs that I see don’t account for simple things like my example. Flights finders in particular have been pretty bad. >BTW: with my new prompt, it did assume the correct airport “ >” Not really. It chose the airport you put basically in the prompt. But I don’t live in MA, I live closer to…

A human would assume if you said “Portland” they would first assume you meant PDX unless they looked up your address and then they would assume Maine.

Just like if I said I wanted to fly to Albany, they would think I meant New York and not my parents city in south GA (ABY) which only has three commercial flights a day.

Even with a human agent, you ask for confirmation.

Also, I ask to speak to people on the ground - in this case it would be CSRs - to break it.

That’s another reason I think “side projects” are useless and they don’t have any merit on resumes. I want them to talk about real world implementations.

Re: Has LLM killed traditional NLP?

#103

Post by e12e »

aeternum wrote: post #81
WildGreenLeave wrote: post #80

Earlier quoted context omitted.

Correct me if I'm wrong, but, if you run multiple inferences at the same time on the same GPU you will need load multiple models in the vram and the models will fight for resources right? So running 10 parallel inferences will slow everything down 5 times right? Or am I missing something?

No, the key is to use the full context window so you structure the prompt as something like: For each line below, repeat the line, add a comma then output whether it most closely represents a product or service: 20 bottles of ferric chloride salesforce ...

Appreciate the concrete advice in this response. Thank you.

Re: Has LLM killed traditional NLP?

#104

Post by why_only_15 »

axegon_ wrote: post #87
segmondy wrote: post #65

Earlier quoted context omitted.

You are not pushing it at 100. I can classify "Is 20 bottles of ferric chloride' a service or product in probably 2 seconds with a 4090. Something that most people don't realize is you can run multiple inference. So with something like a 4090, some solid few shots, and instead of having it classify one example at a time, you can do 5. We can probably run 100 parallel inference at 5 at a time. For about a rate of 250…

FFS... "Lots of writers, few readers". Read again and do the math: 2 seconds, multiply that by 10 million records which contain this, as well as "alarm installation in two locations" and a whole bunch of other crap with little to no repetition (<2%) and where does that get you? 2 * 10,000,000 = 20,000,000 SECONDS!!!! A day has 86,400 seconds (24 * 3600 = 86,400). The data pipeline needs to finish in <24 hours. Everyo…

Assuming the 10M records is ~2000M input tokens + 200M output tokens, this would cost $300 to classify using llama-3.3-70b[1]. If using llama lets you do this in say one day instead of two days for a traditional NLP pipeline, it's worthwhile.

[1]: https://openrouter.ai/meta-llama/llama-3.3-70b-instruct

Re: Has LLM killed traditional NLP?

#105

Post by jcheng »

thangalin wrote: post #86

I created an NLP library to help curl straight quotes into curly quotes. Last I checked, LLMs struggled to curl the following straight quotation marks: ''E's got a 'ittle box 'n a big 'un,' she said, 'wit' th' 'ittle 'un 'bout 2'×6". An' no, y'ain't cryin' on th' "soap box" to me no mo, y'hear. 'Cause it 'tweren't ever a spec o' fun!' I says to my frien'. The library is integrated into my Markdown editor, KeenWrite (…

Great example. I just tried it with a few LLMs and got horrible results. GPT-4o got a ton of them wrong, GPT-1o got them all correct AFAICT but took 1m50s to do so, and Claude 3.5 Sonnet said “Here's the text with straight quotes converted to curly quotes” but then returned the text with all the straight quotes intact.

I’m very surprised all three models didn’t nail it immediately.

Re: Has LLM killed traditional NLP?

#106

Post by vletal »

The idea that we can solve "language" by breaking down and understanding sentences is naive and funny with the benefit of hindsight, is it not?

An equivalently funny attitude seems to be the "natural language will replace programming languages". Let's see how that one will work out when the hype is over.

Re: Has LLM killed traditional NLP?

#107

Post by bavell »

WildGreenLeave wrote: post #80
segmondy wrote: post #65

Earlier quoted context omitted.

You are not pushing it at 100. I can classify "Is 20 bottles of ferric chloride' a service or product in probably 2 seconds with a 4090. Something that most people don't realize is you can run multiple inference. So with something like a 4090, some solid few shots, and instead of having it classify one example at a time, you can do 5. We can probably run 100 parallel inference at 5 at a time. For about a rate of 250…

Correct me if I'm wrong, but, if you run multiple inferences at the same time on the same GPU you will need load multiple models in the vram and the models will fight for resources right? So running 10 parallel inferences will slow everything down 5 times right? Or am I missing something?

You will most likely be using the same model so just 1 to load into vram.

Re: Has LLM killed traditional NLP?

#108

Post by derbaum »

macNchz wrote: post #94
derbaum wrote: post #88

One of the things I'm still struggling with when using LLMs over NLP is classification against a large corpus of data. If I get a new text and I want to find the most similar text out of a million others, semantically speaking, how would I do this with an LLM? Apart from choosing certain pre-defined categories (such as "friendly", "political", ...) and then letting the LLM rate each text on each category, I can't see…

I've used embeddings to define clusters, then passed sampled documents from each cluster to an LLM to create labels for each grouping. I had pretty impressive results from this approach when creating a category/subcategory labels for a collection of texts I worked on recently.

That's interesting, it sounds a bit like those cluster graph visualisation techniques. Unfortunately, my texts seem to fall into clusters that really don't match the ones that I had hoped to get out of these methods. I guess it's just a matter of fine-tuning now.

Re: Has LLM killed traditional NLP?

#109

Post by derbaum »

thaumasiotes wrote: post #90
derbaum wrote: post #88

One of the things I'm still struggling with when using LLMs over NLP is classification against a large corpus of data. If I get a new text and I want to find the most similar text out of a million others, semantically speaking, how would I do this with an LLM? Apart from choosing certain pre-defined categories (such as "friendly", "political", ...) and then letting the LLM rate each text on each category, I can't see…

Take two documents. Feed one through an LLM, one word at a time, and keep track of words that experience greatly inflated probabilities of occurrence, compared to baseline English. "For" is probably going to maintain a level of likelihood close to baseline. "Engine" is not. Do the same thing for the other one. See how much overlap you get.

Wouldn't a simple comparison of the word frequency in my text against a list of usual word frequencies do the trick here without an LLM? Sort of a BM25?

Re: Has LLM killed traditional NLP?

#110

Post by axegon_ »

gbnwl wrote: post #96
axegon_ wrote: post #87

Earlier quoted context omitted.

FFS... "Lots of writers, few readers". Read again and do the math: 2 seconds, multiply that by 10 million records which contain this, as well as "alarm installation in two locations" and a whole bunch of other crap with little to no repetition (<2%) and where does that get you? 2 * 10,000,000 = 20,000,000 SECONDS!!!! A day has 86,400 seconds (24 * 3600 = 86,400). The data pipeline needs to finish in <24 hours. Everyo…

Why are you using 2 seconds? The commenter you are responding to hypothesized being able to do 250/s based on "100 parallel inference at 5 at a time". Not speaking to the validity of that, but find it strange that you ran with the 2 seconds number after seemingly having stopped reading after that line, while yourself lamenting people don't read and telling them to "read again".

Ok, let me dumb it down for you: you have a cockroach in your bathroom and you want to kill it. You have an RPG and you have a slipper. Are you gonna use the RPG or are you going to use the slipper? Even if your bathroom is fine after getting shot with an RPG somehow, isn't this an overkill? If you can code and binary classifier train a classifier in 2 hours that uses nearly 0 resources and gives you good enough results(in my case way above what my targets were) without having to use a ton of resources, libraries, rags, hardware and hell, even electricity? I mean how hard is this to comprehend really?

https://deviq.com/antipatterns/shiny-toy

Post reply on HN