Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Has LLM killed traditional NLP?

medium.com

Post reply on HN
Print view View original

41–50 of 159 posts

Re: Has LLM killed traditional NLP?

#41

Post by elicksaur »

blindriver wrote: post #29
dartos wrote: post #26

Earlier quoted context omitted.

> LLM costs Inference costs, not training costs. > The fact that you can replace programmers You can’t… not for any real project. For quick mockups they’re serviceable > That’s sort of like asking a horse and buggy driver whether automobiles Kind of an insult to OP, no? Horse and buggy drivers were not highly educated experts in their field. Maybe take the word of domain experts rather than AI company marketing teams…

> Maybe take the word of domain experts rather than AI company marketing teams. Appeal to authority is a well known logical fallacy. I know how dead NLP is personally because I’ve never been able to get NLP working but once ChatGPT came around, I was able to classify texts extremely easily. It’s transformational. I was able to get ChatGPT to classify posts based on how political it was from a scale of 1 to 10 and whi…

I haven’t understood these types of uses. How do you validate the score that the LLM gives?

Re: Has LLM killed traditional NLP?

#42

Post by elwebmaster »

dartos wrote: post #26
blindriver wrote: post #21

Earlier quoted context omitted.

That’s sort of like asking a horse and buggy driver whether automobiles are going to put them out of business. I think for the most part, casual nlp is dead because of LLMs. And LLM costs are going to plummet soon, so large scale nlp that you’re talking about is probably dead within 5 years or less. The fact that you can replace programmers with prompts is huge in my opinion so no one needs to learn an nlm API anymor…

> LLM costs Inference costs, not training costs. > The fact that you can replace programmers You can’t… not for any real project. For quick mockups they’re serviceable > That’s sort of like asking a horse and buggy driver whether automobiles Kind of an insult to OP, no? Horse and buggy drivers were not highly educated experts in their field. Maybe take the word of domain experts rather than AI company marketing teams…

Reply didn’t say that the expert is uneducated, just that their tool is obsolete. Better look at facts the way they are, sugar coating doesn’t serve anyone.

Re: Has LLM killed traditional NLP?

#43

Post by IanCal »

scarface_74 wrote: post #19
gtirloni wrote: post #16

Earlier quoted context omitted.

How about the the costs?

We measure savings in terms of call deflections. Clients we work with say that each time a customer talks to an agent it costs $2-$5. That’s not even taking into account call abandonments

My base thing while advising people is that if anyone you pay needs to read the output, or you are directly replacing any kind of work then even frontier llm model inference costs are irrelevant. Of course you need to work out of that's truly the case but people worry about the cost in places where it's just irrelevant. If it's $2 when you get to an agent, each case that's avoided there could pay for around a million words read/generated. That's expensive compared to most API calls but irrelevant when counting human costs.

Re: Has LLM killed traditional NLP?

#44

Post by elicksaur »

alexwebb2 wrote: post #23
axegon_ wrote: post #6

No, it has not and will not in the foreseeable future. This is one of my responsibilities at work. LLMs are not feasible when you have a dataset of 10 million items that you need to classify relatively fast and at a reasonable cost. LLMs are great at mid-level complexity tasks given a reasonable volume of data - they can take away the tedious job of figuring out what you are looking at or even come up with some basic…

I think your intuition on this might be lagging a fair bit behind the current state of LLMs. System message: answer with just "service" or "product" User message (variable): 20 bottles of ferric chloride Response: product Model: OpenAI GPT-4o-mini $0.075/1Mt batch input * 27 input tokens * 10M jobs = $20.25 $0.300/1Mt batch output * 1 output token * 10M jobs = $3.00 It's a sub-$25 job. You'd need to be doing 20 times…

How do you validate these classifications?

Re: Has LLM killed traditional NLP?

#45

Post by thaw13579 »

blindriver wrote: post #29
dartos wrote: post #26

Earlier quoted context omitted.

> LLM costs Inference costs, not training costs. > The fact that you can replace programmers You can’t… not for any real project. For quick mockups they’re serviceable > That’s sort of like asking a horse and buggy driver whether automobiles Kind of an insult to OP, no? Horse and buggy drivers were not highly educated experts in their field. Maybe take the word of domain experts rather than AI company marketing teams…

> Maybe take the word of domain experts rather than AI company marketing teams. Appeal to authority is a well known logical fallacy. I know how dead NLP is personally because I’ve never been able to get NLP working but once ChatGPT came around, I was able to classify texts extremely easily. It’s transformational. I was able to get ChatGPT to classify posts based on how political it was from a scale of 1 to 10 and whi…

Performance and cost are trade-offs though. You could just as well say that LLMs are dead in the water, except in terms of performance.

It does seem likely we’ll soon have cheap enough LLM inference to displace traditional NLP entirely, although not quite yet.

Re: Has LLM killed traditional NLP?

#46

Post by darepublic »

I remember using the open NLP library from Stanford around 2016. It would do parts of speech tagging of words in a sentence (labelling the words with their grammatical function). It was pretty good but reliably failed on certain words where context determined the tag. When for gpt 3 came out the first thing I tested it out on was parts of speech tagging. In particular those sentences open NLP had trouble with. And it aced everything I was impressed.

Re: Has LLM killed traditional NLP?

#47

Post by datadrivenangel »

blindriver wrote: post #38
otabdeveloper4 wrote: post #31

Earlier quoted context omitted.

> The fact that you can replace programmers with prompts No, you can't. The only thing LLM's replace is internet commentators.

As I explained below, I avoided having to learn anything about ML, PyTorch or any other APIs when trying to classify posts based on how political they were and which affiliation they were. That was holding me back and it was easily replaced by an llm and a prompt. Literally took me minutes what would have taken days or weeks and the results are more than good enough.

GPT 3.5 is more accurate at classifying tweets as liberal than it is at identifying posts that are conservative.

If you're going for rough approximation, LLMs are great, and good enough. More care and conventional ML methods are appropriate as the stakes increase though.

Re: Has LLM killed traditional NLP?

#48

Post by alexwebb2 »

otabdeveloper4 wrote: post #35
alexwebb2 wrote: post #28

Earlier quoted context omitted.

All software engineers are (or can be) prompt engineers, at least to the level of trivial jobs like this. It's just an API call and a one-liner instruction. Odds are very good at most companies that they have someone on staff who can knock this out in short order. No specialized hiring required.

> ..and validating the output? You glossed over the meat of the question.

Your validation approach doesn't really change based on the classification method (LLM vs NLP).

At that volume you're going to use automated tests with known correct answers + random sampling for human validation.

Re: Has LLM killed traditional NLP?

#50

Post by alexwebb2 »

datadrivenangel wrote: post #47
blindriver wrote: post #38

Earlier quoted context omitted.

As I explained below, I avoided having to learn anything about ML, PyTorch or any other APIs when trying to classify posts based on how political they were and which affiliation they were. That was holding me back and it was easily replaced by an llm and a prompt. Literally took me minutes what would have taken days or weeks and the results are more than good enough.

GPT 3.5 is more accurate at classifying tweets as liberal than it is at identifying posts that are conservative. If you're going for rough approximation, LLMs are great, and good enough. More care and conventional ML methods are appropriate as the stakes increase though.

GPT 3.5 has been very, very obsolete in terms of price-per-performance for over a year. Bit of a straw man.
Post reply on HN