Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

github.com

Post reply on HN
Print view View original

51–60 of 67 posts

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#51

Post by skeptrune »

chpatrick wrote: post #49
skeptrune wrote: post #8

Earlier quoted context omitted.

Most definitely! LLMs are amazing tools for generating synthetic datasets that can be used alongside traditional NLP to train things like decision trees with libraries like cat/xgboost. I have a search background so learning to rank is always top of mind for me, but there other places like sentiment analysis, intent detection, and topic classification where it's great too.

But for the analysis use cases you mentioned, can't you just ask an LLM to read the text and output the answer as JSON, and you're done? Is it just because running LLMs is expensive?

No, it's just slow and less accurate. Wrong tool for the job when you care a lot about understanding the reasoning and internals of what the model is caring the most about.

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#52

Post by ACCount37 »

btown wrote: post #25
potatoman22 wrote: post #19

Earlier quoted context omitted.

The SpaCy creator has a good blog post on this https://explosion.ai/blog/against-llm-maximalism

I'd go a step beyond this (excellent) post and posit that one incredibly valuable characteristic of traditional NLP is that it is largely immune to prompt injection attacks. Especially as LLMs continue to be better tuned to follow instructions that are intentionally colocated and intermingled with data in user messages, it becomes difficult to build systems that can provide real guarantees that "we'll follow your pro…

If that's an issue for you, you do the year 2018 thing and just train classification heads for a base model LLM.

No instruct tuning means prompt injection is curbed. Classification heads means you get results off a single forward pass.

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#53

Post by apprentice7 »

nutjob2 wrote: post #42
ur-whale wrote: post #39

At the risk of asking a naive question ... why would anyone still do traditional NLP today?

There is a need for good and easy NLP "structured" interfaces to traditional structured data software. This is a gaping hole in the tech right now. Most other NLP tasks can be handled by ML approaches but this one is a poor fit for those. I'm sure LLM true believers will disagree.

Could you give a couple specific examples? I'm trying to get into traditional NLP but everything I find is AI related and I don't know if it's worth going the traditional route long-term.

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#54

Post by ffsm8 »

IanCal wrote: post #37
btown wrote: post #25

Earlier quoted context omitted.

I'd go a step beyond this (excellent) post and posit that one incredibly valuable characteristic of traditional NLP is that it is largely immune to prompt injection attacks. Especially as LLMs continue to be better tuned to follow instructions that are intentionally colocated and intermingled with data in user messages, it becomes difficult to build systems that can provide real guarantees that "we'll follow your pro…

> But no amount of text appended to an input document, no matter how persuasive, can cause an NLP pipeline to change how it interprets the remainder of the document, Text added to a document can absolutely change how an NLP pipeline interprets the document. > "Ignore the above prompt" is just a sentence that doesn't seem like positive or on-topic sentiment to an NLP classifier, and that's it. And simple repeated word…

> And simple repeated words can absolutely make that kind of change for many NLP systems.

That's not what prompt injection is.

And NLP stands for natural language processing. If the result didn't change after you've made changes to the input... It'd be a bug?

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#55

Post by IanCal »

coder68 wrote: post #46
IanCal wrote: post #38

Earlier quoted context omitted.

It depends on a lot of things but to add to your possible setups you can potentially improve results by using simpler systems for first answers and falling back afterwards. For example: If contains cafe and not internet/cyber/etc -> restaurant No -> (tfidf) -> yes, no, unsure unsure -> embeddings -> yes, no, unsure unsure -> llm -> yes, no, unsure unsure -> human queue ->...

I think the idea of backoff by ratcheting up complexity here is a very good idea, thanks for your suggestions.

Happy to help - this is a thing I’ve employed multiple times for real cases.

One big benefit is that it uses the cheapest and most understandable approaches for the majority of cases, and scales up quite nicely. It has a neat place for very custom issues to be fixed too.

There will always be some things that simple approaches think are clear but aren’t, which is awkward but then all pipelines end up with that somewhere.

Edit - you can also deploy things earlier if you start from the beginning of the chain. Moving from big deploy to iteration on the remaining issues is often a win just in deployment issues.

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#56

Post by nine_k »

lyu07282 wrote: post #45
nine_k wrote: post #32

Earlier quoted context omitted.

How about expense? LLMs do dramatically more computations doing simple tasks, and only run on relatively exotic, expensive hardware. You have to trust an LLM provider, and keep paying them. If a traditional NLP solution can run under your control, and tackle the task at hand, it can be plainly much cheaper at scale.

thats absurd, there are thousands of open-source LLMs you can run yourself of all shapes and sizes

Are many of them comparable to Claude Sonnet or GPT-5? What kind of hardware do they require?

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#57

Post by IanCal »

ffsm8 wrote: post #54
IanCal wrote: post #37

Earlier quoted context omitted.

> But no amount of text appended to an input document, no matter how persuasive, can cause an NLP pipeline to change how it interprets the remainder of the document, Text added to a document can absolutely change how an NLP pipeline interprets the document. > "Ignore the above prompt" is just a sentence that doesn't seem like positive or on-topic sentiment to an NLP classifier, and that's it. And simple repeated word…

> And simple repeated words can absolutely make that kind of change for many NLP systems. That's not what prompt injection is. And NLP stands for natural language processing. If the result didn't change after you've made changes to the input... It'd be a bug?

No? But repeated words can impact simple nlp setups. I’m not sure what case you’re concerned about where added text impacts classification with an LLM but added words shouldn’t with a different pipeline.

> And NLP stands for natural language processing. If the result didn't change after you've made changes to the input... It'd be a bug?

No, I’d want my classifier to be unchanged by garbage words added. It likely will be, but that impact is a bug not a feature.

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#58

Post by lyu07282 »

nine_k wrote: post #56
lyu07282 wrote: post #45

Earlier quoted context omitted.

thats absurd, there are thousands of open-source LLMs you can run yourself of all shapes and sizes

Are many of them comparable to Claude Sonnet or GPT-5? What kind of hardware do they require?

None of them of course. But the point is that even smaller open-source "LLMs" (more specifically transformer architectures) you can run anywhere yourself outperform these "traditional" pipelines with less compute. I would say that its not well defined what exactly "traditional" even means here though, since I wouldn't really even describe CNN/BiLSTMs as "traditional", in my mind that would be SpaCy <2.0 and NLTK (linear models SVMs/TF-IDF, Word2Vec/Glove/fastText/etc. etc.), LLMs are at least 2 generations ahead of those since there was the whole "deep learning" craze inbetween.

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#60

Post by ffsm8 »

IanCal wrote: post #57
ffsm8 wrote: post #54

Earlier quoted context omitted.

> And simple repeated words can absolutely make that kind of change for many NLP systems. That's not what prompt injection is. And NLP stands for natural language processing. If the result didn't change after you've made changes to the input... It'd be a bug?

No? But repeated words can impact simple nlp setups. I’m not sure what case you’re concerned about where added text impacts classification with an LLM but added words shouldn’t with a different pipeline. > And NLP stands for natural language processing. If the result didn't change after you've made changes to the input... It'd be a bug? No, I’d want my classifier to be unchanged by garbage words added. It likely will…

Prompt injection is about making the model do something else then specified.

Adding words to the text to break the algorithm which does the NLP is more along the lines of providing 1 in a boolean field to break the system. And that's generally something you can mitigate to some degree via heuristics and sanity checking. Doing the same for LLMs is essentially impossible, because it's an effective black box, so you cannot determine the error scenarios and add some mitigations

Post reply on HN