Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

github.com

Post reply on HN
Print view View original

21–30 of 67 posts

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#21

Post by ok_dad »

binarymax wrote: post #4

I’ve been a user of SpaCy since 2016. I haven’t touched it in years and I just picked it up again to develop a new metric for RAG using part of speech coverage. The API is one of the best ever, and really set the bar high for language tooling. I’m glad it’s still around and getting updates. I had a bit of trouble integrating it with uv, but nothing too bad. Thanks to the explosion team for making such an amazing proj…

What’s great about the API that you enjoy and do you have anything you hate about it?

I’m writing a small library at work for some NLP tasks and I haven’t got a whole lot of experience in writing libraries for NLP, so I’m interested in what would make my library the best for the user.

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#22

Post by [deleted] »

roadside_picnic wrote: post #7

A friend, who also has a background in NLP, was asking me the other day "Is there still even a need for traditional NLP in the age of LLMs?" This is one of the under-discussed areas of LLMs imho. For anything that would have have required either word2vec embeddings of a tf-idf representation (classification tasks, sentiment analysis, etc) there are rare exceptions where it wouldn't just be better to start with a sema…

[deleted]

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#24

Post by roadside_picnic »

coder68 wrote: post #20
noosphr wrote: post #18

Earlier quoted context omitted.

When you say llms do you mean decoder only models, gpt et al, or encoder only models, bert et al? I've found encoder only models to be vastly better for anything that doesn't require natural language responses and the majority of them are small enough that _pretraining_ a model for each task costs a few hundred dollars.

By LLMs I meant decoder only, e.g. Gemini, Claude, etc. Can you go into more detail on how you're using the encoder models? I'm curious. Typically I have used them for embedding text or for fine-tuning after attaching a classifier head. What are you pre-training on, and for what task?

> how you're using the encoder models?

In my original comment this is what I was referring to: using the embeddings produced by these models, not using something like GPT to classify text (that's wildly inefficient and in my experience gets subpar results).

To answer your question: you simply use the embedding vector as the features in whatever model you're trying to train. I've found this to get significantly superior results with significantly less examples than any traditional NLP approach to vector representation.

> What are you pre-training on, and for what task?

My experience has been that you don't need to pretrain at all. The embeddings are more information rich than anything you could attempt to achieve with other vector representations you might come up with using the set of data you have. This might not be true at extreme scales, but for nearly all traditional nlp classification tasks I've found this to be so much easier to implement and so much better performing there's really not a good reason to start with a "simpler" approach.

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#25

Post by btown »

potatoman22 wrote: post #19
roadside_picnic wrote: post #7

A friend, who also has a background in NLP, was asking me the other day "Is there still even a need for traditional NLP in the age of LLMs?" This is one of the under-discussed areas of LLMs imho. For anything that would have have required either word2vec embeddings of a tf-idf representation (classification tasks, sentiment analysis, etc) there are rare exceptions where it wouldn't just be better to start with a sema…

The SpaCy creator has a good blog post on this https://explosion.ai/blog/against-llm-maximalism

I'd go a step beyond this (excellent) post and posit that one incredibly valuable characteristic of traditional NLP is that it is largely immune to prompt injection attacks.

Especially as LLMs continue to be better tuned to follow instructions that are intentionally colocated and intermingled with data in user messages, it becomes difficult to build systems that can provide real guarantees that "we'll follow your prompt, but not prompts that are in the data you provided."

But no amount of text appended to an input document, no matter how persuasive, can cause an NLP pipeline to change how it interprets the remainder of the document, or to leak its own system instructions, or anything of that nature. "Ignore the above prompt" is just a sentence that doesn't seem like positive or on-topic sentiment to an NLP classifier, and that's it.

There's an even broader discussion to be had about the relative reliability of NLP pipelines, outside of a security perspective. As always, it's important to pick the right tools for the job, and the SpaCy article linked in the parent puts this quite well.

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#26

Post by coder68 »

roadside_picnic wrote: post #24
coder68 wrote: post #20

Earlier quoted context omitted.

By LLMs I meant decoder only, e.g. Gemini, Claude, etc. Can you go into more detail on how you're using the encoder models? I'm curious. Typically I have used them for embedding text or for fine-tuning after attaching a classifier head. What are you pre-training on, and for what task?

> how you're using the encoder models? In my original comment this is what I was referring to: using the embeddings produced by these models, not using something like GPT to classify text (that's wildly inefficient and in my experience gets subpar results). To answer your question: you simply use the embedding vector as the features in whatever model you're trying to train. I've found this to get significantly superi…

Ah yes this does make sense. We are definitely in agreement on the point of "wildly inefficient and subpar". I'll try out decoder model embeddings soon, e.g. Qwen/Qwen3-Embedding-8B. I'm working with largish amounts of data (200M records), so I tried to pick a good balance between size:perf:cost, using BAAI/bge-base-en-v1.5 to start (384 dim).

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#27

Post by cantdutchthis »

bratao wrote: post #3

I'm really curious about the history of spaCy. From my PoV: it grew a lot during the pandemic era, hiring a lot of employees. I remember something about raising money for the first time. It was very competitive in NLP tasks. Now it seems that it has scaled back considerably, with a dramatic reduction in employees and a total slowdown of the project. The v4 version looks postponed. It isn't competitive in many tasks a…

former employee here, Matt wrote a blogpost with pretty much all of the details here: https://honnibal.dev/blog/back-to-our-roots

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#29

Post by joshdavham »

I’ve been using SpaCy for many of my projects for 5 years now. The library has incredible ergonomics and allows you to reuse the same API across languages as different as French and Japanese! I also appreciate that they allow you to install different model sizes (I usually go with small).

Re: SpaCy: Industrial-Strength Natural Language Processing (NLP) in Python

#30

Post by microtonal »

cantdutchthis wrote: post #27
bratao wrote: post #3

I'm really curious about the history of spaCy. From my PoV: it grew a lot during the pandemic era, hiring a lot of employees. I remember something about raising money for the first time. It was very competitive in NLP tasks. Now it seems that it has scaled back considerably, with a dramatic reduction in employees and a total slowdown of the project. The v4 version looks postponed. It isn't competitive in many tasks a…

former employee here, Matt wrote a blogpost with pretty much all of the details here: https://honnibal.dev/blog/back-to-our-roots

:wave:

Also: https://explosion.ai/blog/back-to-our-roots-company-update

(Interesting tidbit: I got hired by Explosion after a HN comment on model distillation :))

Post reply on HN