Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advanced NLP with SpaCy

course.spacy.io

Post reply on HN
Print view View original

21–30 of 60 posts

Re: Advanced NLP with SpaCy

#21

Post by pletnes »

cantdutchthis wrote: post #5
tomohelix wrote: post #3

No offense, but isn't the NLP field effectively solved with the creation of LLMs, or at least for the majority of the tasks you would expect from an NLP application? I am sure you can find some special areas or niches where traditional NLP approaches would outcompete a black box like LLMs. But with the LLMs becoming much more efficient now after quantization to the point you can run them locally, I think there is a g…

> I think there is a good argument in saying simple NLP is basically solved In my experience LLMs can get about 70-80% accuracy on a bunch of NER and text classification tasks if you give it a reasonable prompt. That's not nothing and it's something that you can get started with super quickly. But you'll have slow responses and typically a 3rd party running the inference. Annotating data yourself to about 2000-3000 e…

Good points. This means you can use LLMs to create an 80% accurate dataset, then manually correct the last 20% to get all the way.

Re: Advanced NLP with SpaCy

#22

Post by john-radio »

pletnes wrote: post #21
cantdutchthis wrote: post #5

Earlier quoted context omitted.

> I think there is a good argument in saying simple NLP is basically solved In my experience LLMs can get about 70-80% accuracy on a bunch of NER and text classification tasks if you give it a reasonable prompt. That's not nothing and it's something that you can get started with super quickly. But you'll have slow responses and typically a 3rd party running the inference. Annotating data yourself to about 2000-3000 e…

Good points. This means you can use LLMs to create an 80% accurate dataset, then manually correct the last 20% to get all the way.

"Manually" anything just depends on the size of your dataset.

Re: Advanced NLP with SpaCy

#23

Post by rolisz »

tomohelix wrote: post #3

No offense, but isn't the NLP field effectively solved with the creation of LLMs, or at least for the majority of the tasks you would expect from an NLP application? I am sure you can find some special areas or niches where traditional NLP approaches would outcompete a black box like LLMs. But with the LLMs becoming much more efficient now after quantization to the point you can run them locally, I think there is a g…

Spacy has integration with LLMs, so it can leverage them.

But in general you can often get by with models that are 10-100 times smaller, but are not as general as an LLM.

Re: Advanced NLP with SpaCy

#24

Post by ATMLOTTOBEER »

syllogism wrote: post #19
tomohelix wrote: post #3

No offense, but isn't the NLP field effectively solved with the creation of LLMs, or at least for the majority of the tasks you would expect from an NLP application? I am sure you can find some special areas or niches where traditional NLP approaches would outcompete a black box like LLMs. But with the LLMs becoming much more efficient now after quantization to the point you can run them locally, I think there is a g…

(Original author of spaCy and Explosion CTO here) Okay so, first some terminology. LLMs can mean a bunch of different things, people call models the size of BERT LLMs sometimes. So let's talk specifically about in-context learning (ICL) with either zero or a few examples. So we'll say LLM ICL, and contrast that with techniques where you annotate enough data to train with, which might only be something like 10-40 hour…

/thread. Really complete explanation

Re: Advanced NLP with SpaCy

#25

Post by jgalt212 »

pletnes wrote: post #21
cantdutchthis wrote: post #5

Earlier quoted context omitted.

> I think there is a good argument in saying simple NLP is basically solved In my experience LLMs can get about 70-80% accuracy on a bunch of NER and text classification tasks if you give it a reasonable prompt. That's not nothing and it's something that you can get started with super quickly. But you'll have slow responses and typically a 3rd party running the inference. Annotating data yourself to about 2000-3000 e…

Good points. This means you can use LLMs to create an 80% accurate dataset, then manually correct the last 20% to get all the way.

Manually correcting the wrong 20% may be a reasonable amount of work, but you must examine all 100 to find the 20 that needs fixing. And that is most likely not a reasonable amount of work.

Re: Advanced NLP with SpaCy

#26

Post by jerrygenser »

janalsncm wrote: post #14
tomohelix wrote: post #3

No offense, but isn't the NLP field effectively solved with the creation of LLMs, or at least for the majority of the tasks you would expect from an NLP application? I am sure you can find some special areas or niches where traditional NLP approaches would outcompete a black box like LLMs. But with the LLMs becoming much more efficient now after quantization to the point you can run them locally, I think there is a g…

Usually businesses want fast, accurate, and cheap. LLMs are somewhat fast, somewhat accurate, and not cheap. Basic NLP techniques can be faster, more accurate, and cheaper.

LLM are not somewhat fast when measured against CPU models in spacy that are specialized for a task. They are multiple orders of magnitude slower for e.g. token classification/NER, text classification, etc.

that's not to mention the plumbing that spacy provides that are mostly bindings to C code for tokenization, lemmatization, etc. things like that which are more algorithms problems than machine learning problems.

Re: Advanced NLP with SpaCy

#27

Post by jgalt212 »

tomohelix wrote: post #3

No offense, but isn't the NLP field effectively solved with the creation of LLMs, or at least for the majority of the tasks you would expect from an NLP application? I am sure you can find some special areas or niches where traditional NLP approaches would outcompete a black box like LLMs. But with the LLMs becoming much more efficient now after quantization to the point you can run them locally, I think there is a g…

I wouldn't trust an LLM in any application where you need hi 90s% accuracy.

Re: Advanced NLP with SpaCy

#28

Post by markisus »

gattilorenz wrote: post #9
tomohelix wrote: post #3

No offense, but isn't the NLP field effectively solved with the creation of LLMs, or at least for the majority of the tasks you would expect from an NLP application? I am sure you can find some special areas or niches where traditional NLP approaches would outcompete a black box like LLMs. But with the LLMs becoming much more efficient now after quantization to the point you can run them locally, I think there is a g…

I don’t understand what it means to be “solved”. It’s like saying that “architecture is now solved”, “physics is solved”, or “programming is solved”. It’s a field of science and/or engineering, it’s not like we will ever run out of things to try/build/investigate. LLMs work… to a certain extent, with limitations and tradeoffs, and for some things. Would you spend days, money and Co2 to split a huge text corpus in sen…

I take it to mean that there is an effective generally accepted solution or methodology for problems in the field. Bridge building has been largely solved by methods of mathematical and computational structural analysis, manufacturing, and government regulation. We know how to build a bridge. Before the solution was known, designers would just go by intuition and we wouldn’t have any actual assurance that the bridge would hold. We can probably never solve larger domains like physics, programming, or architecture.

Re: Advanced NLP with SpaCy

#29

Post by pletnes »

jgalt212 wrote: post #25
pletnes wrote: post #21

Earlier quoted context omitted.

Good points. This means you can use LLMs to create an 80% accurate dataset, then manually correct the last 20% to get all the way.

Manually correcting the wrong 20% may be a reasonable amount of work, but you must examine all 100 to find the 20 that needs fixing. And that is most likely not a reasonable amount of work.

I’ve done image annotation for ML / computer vision. I found it extremely useful to use my first annotations to train a poor model, then use results from that to annotate new data. You get feedback on model quality as you go, and looking at 100 images with multiple objects is way less work than annotating 100 images.

If it’s worth it? I guess that depends on the project you’re working on.

Re: Advanced NLP with SpaCy

#30

Post by akgfab »

syllogism wrote: post #19
tomohelix wrote: post #3

No offense, but isn't the NLP field effectively solved with the creation of LLMs, or at least for the majority of the tasks you would expect from an NLP application? I am sure you can find some special areas or niches where traditional NLP approaches would outcompete a black box like LLMs. But with the LLMs becoming much more efficient now after quantization to the point you can run them locally, I think there is a g…

(Original author of spaCy and Explosion CTO here) Okay so, first some terminology. LLMs can mean a bunch of different things, people call models the size of BERT LLMs sometimes. So let's talk specifically about in-context learning (ICL) with either zero or a few examples. So we'll say LLM ICL, and contrast that with techniques where you annotate enough data to train with, which might only be something like 10-40 hour…

Also enjoyed your blog post on similar issues: https://explosion.ai/blog/against-llm-maximalism

Excited to see how curated transformers works as an alternative to hf!

Post reply on HN