I recently wrote an open source Python module to deidentify people's names and gender specific pronouns. It uses spaCy's Named Entity Recognition (NER) capabilities combined with custom pronoun handling. See the screenshot in the README.md file.
SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python
41–50 of 67 posts
Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python
#42At the risk of asking a naive question ... why would anyone still do traditional NLP today?
There is a need for good and easy NLP "structured" interfaces to traditional structured data software. This is a gaping hole in the tech right now. Most other NLP tasks can be handled by ML approaches but this one is a poor fit for those. I'm sure LLM true believers will disagree.
Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python
#43SpaCy is awesome - we have used it in a number of enterprise-grade applications and found it to hold up well.
Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python
#44SpaCy is criminally underrated. I expect to see it experience a new wave of growth as folks new to AI start to realize all of the language tooling they need to build more reliable "traditional" ML pipelines. API surface is designed well and it's still actively maintained almost 10 years after it initially went public.
Is there any use case for "traditional" NLP in the age of LLMs?
I used to work a lot with those pipelines, I think the truth is that LLMs (and LLM embeddings) have surpassed pretty much all traditional NLP. I guess if speed is more important than accuracy? but even then, like with small embedded LLMs they still outperform "traditional NLP" on pretty much every task probably. So it doesn't make a lot of sense to not use it nowadays.
Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python
#45A friend, who also has a background in NLP, was asking me the other day "Is there still even a need for traditional NLP in the age of LLMs?" This is one of the under-discussed areas of LLMs imho. For anything that would have have required either word2vec embeddings of a tf-idf representation (classification tasks, sentiment analysis, etc) there are rare exceptions where it wouldn't just be better to start with a sema…
How about expense? LLMs do dramatically more computations doing simple tasks, and only run on relatively exotic, expensive hardware. You have to trust an LLM provider, and keep paying them. If a traditional NLP solution can run under your control, and tackle the task at hand, it can be plainly much cheaper at scale.
thats absurd, there are thousands of open-source LLMs you can run yourself of all shapes and sizes
Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python
#46Earlier quoted context omitted.
I have been working on text classification tasks at work, and I have found that for my particular use-case, LLMs are not performing well at all. I have spent a few thousand dollars trying, and I have tried everything from few-shot to asking simple binary yes/no questions, and I have had mixed success. I have stopped trying to use LLMs for this project and switched to discriminative models (Logistic Regression with TF…
It depends on a lot of things but to add to your possible setups you can potentially improve results by using simpler systems for first answers and falling back afterwards. For example: If contains cafe and not internet/cyber/etc -> restaurant No -> (tfidf) -> yes, no, unsure unsure -> embeddings -> yes, no, unsure unsure -> llm -> yes, no, unsure unsure -> human queue ->...
I think the idea of backoff by ratcheting up complexity here is a very good idea, thanks for your suggestions.
Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python
#47Earlier quoted context omitted.
I have been working on text classification tasks at work, and I have found that for my particular use-case, LLMs are not performing well at all. I have spent a few thousand dollars trying, and I have tried everything from few-shot to asking simple binary yes/no questions, and I have had mixed success. I have stopped trying to use LLMs for this project and switched to discriminative models (Logistic Regression with TF…
At my work, we still prefer to use distilbert for text classification. It almost always does well with a little bit of fine tuning. In very rare cases, we use LLMs/Agentic setup when the task involves refering both images and text and the same time.
I can confirm that Distillbert has worked well when I have used it for classification, especially on shortish sequences. I'm really interested in trying out ModernBert, or a smaller variant due to the larger context window (8192 tokens).
Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python
#48Earlier quoted context omitted.
I have been working on text classification tasks at work, and I have found that for my particular use-case, LLMs are not performing well at all. I have spent a few thousand dollars trying, and I have tried everything from few-shot to asking simple binary yes/no questions, and I have had mixed success. I have stopped trying to use LLMs for this project and switched to discriminative models (Logistic Regression with TF…
Are your categories fixed? If so you could constrain the output using enums in structured outputs. re: inconsistencies in output, OpenAI provide a seed and system_fingerprint options to (mostly) produce deterministic output.
The outputs are working correctly in terms of formatting, but the answers themselves may be inconsistent. I have experimented with varying the prompt and the answers can change dramatically. I could experiment with lowering temperature, but I just don't think generative models were a good fit for the problem. The appeal is the speed of prototyping and no need for training data, but it honestly didn't take much for my problem: one afternoon and ~1000 samples labeled got me to a good baseline.
Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python
#49Earlier quoted context omitted.
Is there any use case for "traditional" NLP in the age of LLMs?
Most definitely! LLMs are amazing tools for generating synthetic datasets that can be used alongside traditional NLP to train things like decision trees with libraries like cat/xgboost. I have a search background so learning to rank is always top of mind for me, but there other places like sentiment analysis, intent detection, and topic classification where it's great too.
But for the analysis use cases you mentioned, can't you just ask an LLM to read the text and output the answer as JSON, and you're done? Is it just because running LLMs is expensive?
Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python
#50Earlier quoted context omitted.
At my work, we still prefer to use distilbert for text classification. It almost always does well with a little bit of fine tuning. In very rare cases, we use LLMs/Agentic setup when the task involves refering both images and text and the same time.
I can confirm that Distillbert has worked well when I have used it for classification, especially on shortish sequences. I'm really interested in trying out ModernBert, or a smaller variant due to the larger context window (8192 tokens).
I was thinking of trying ModernBERT for one of my projects. But I can only conclude after seeing the performance for my usecase. Do you think ModernBERT will be capable of expanding abbreviated sentences?