Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advanced NLP with spaCy v3

course.spacy.io

Post reply on HN
Print view View original

21–30 of 40 posts

Re: Advanced NLP with spaCy v3

#21

Post by kulikalov »

artembugara wrote: post #2

We've been using spaCy a lot for the past few months. Mostly for non-production use cases, however, I can say that it is the most robust framework for NLP at the moment. V3 added support for transformers: that's a killer feature as many models from https://huggingface.co/docs/transformers/index work great out of the box. At the same time, I found NER models provided by spaCy to have a low accuracy while working with…

Are you using the high accuracy eng model for NER? I’ve been very happy with orgs recognition, it actually did way better than any other open source model in my case.

Re: Advanced NLP with spaCy v3

#22

Post by brd »

artembugara wrote: post #20
brd wrote: post #19

Earlier quoted context omitted.

I really appreciate how accessible SpaCy has made NLP work but their NER is definitely low accuracy. Where stem/lem felt critical to successful NLP processing a few years ago, we've found stem/lem work to be much less important for downstream tasks when transformer based models are involved. For topic extraction stem/lem still seems to do a lot to improve accuracy and for rules based approaches I can still see how it…

Did you try something like autoNLP by huggingface?

No, we've got our own fine tuning pipeline and initial tests showed better performance without traditional stem/lem processing so we dropped it from our classification pipelines and haven't seen a need to revisit.

Re: Advanced NLP with spaCy v3

#23

Post by Vetch »

minimaxir wrote: post #4

A relatively underdiscussed quirk of the rise of superlarge language models like GPT-3 for certain NLP tasks is that since those models have incorporated so much real world grammar, there's no need to do advanced preprocessing and can just YOLO and work with generated embeddings instead without going into spaCy's (excellent) parsing/NER features. OpenAI recently released an Embeddings API for GPT-3 with good demos an…

While you make sensible points, in the case of GPT-3, not everyone will be willing to route their data through OpenAI's servers.

> Just use DistilBERT uncased/cased (which is fast enough to run on consumer CPUs)

This can still be impractical, at least in my case of regularly needing to process hundreds of pages of text. Simpler systems can be much faster for an acceptable loss and you can get more robustness by working with label distributions instead of just picking argmax.

Fast simpler classifiers can also help decide where the more resource intensive models should focus attention.

Another reason for preprocessing is rule systems. Even if not glamorous to talk about, they still see heavy use in practical settings. While dependency parses are hard to make use of, shallow parses (chunking) and parts of speech data can be usefully fed into rule systems.

Re: Advanced NLP with spaCy v3

#24

Post by Vetch »

minimaxir wrote: post #13
hooande wrote: post #11

Earlier quoted context omitted.

lol. a rough translation is that the new super language models are good enough that you don't have to keep track of specific parts of speech in your programming. if you look at the arrays of floating point weights that underlie gpt-3 etc, you can use them to match present participle phrases with other present participle phrases and so forth this is of course a correct and prescient observation. minimaxir is kind of a…

I don't think it's more of a final boss thing: IMO working with embeddings/word vectors is easier, even in the basest case such as word2vec/GloVe, to understand than some of the more conventional NLP techniques (e.g. bag of words/TF-IDF). The spaCy tutorials in the submission also have a section on word vectors.

Ah, although, TF-IDF is still good to know. Semantic search hasn't eliminated the need for classical retrieval techniques. It can also be used to select a subset of words to use to create an average of word vectors for a document signature, a quick and dirty method for document embeddings.

Bag of word co-occurrences in matrix format is also a nice to know, factorizing such matrices were the original vector space model for distributional semantics and provide historical context for GloVe and the like.

Re: Advanced NLP with spaCy v3

#25

Post by artembugara »

kulikalov wrote: post #21
artembugara wrote: post #2

We've been using spaCy a lot for the past few months. Mostly for non-production use cases, however, I can say that it is the most robust framework for NLP at the moment. V3 added support for transformers: that's a killer feature as many models from https://huggingface.co/docs/transformers/index work great out of the box. At the same time, I found NER models provided by spaCy to have a low accuracy while working with…

Are you using the high accuracy eng model for NER? I’ve been very happy with orgs recognition, it actually did way better than any other open source model in my case.

Try it on a sentence where all tokens are lower/upper case. It just doesn’t really work.

Re: Advanced NLP with spaCy v3

#26

Post by Eridrus »

artembugara wrote: post #2

We've been using spaCy a lot for the past few months. Mostly for non-production use cases, however, I can say that it is the most robust framework for NLP at the moment. V3 added support for transformers: that's a killer feature as many models from https://huggingface.co/docs/transformers/index work great out of the box. At the same time, I found NER models provided by spaCy to have a low accuracy while working with…

I feel like NER is a poorly designed task in general. You're eventually trying to link the entities to some kind of KB, so you should be injecting that entity information into your system for detecting mentions.

Re: Advanced NLP with spaCy v3

#28

Post by nefitty »

artembugara wrote: post #8
wyldfire wrote: post #7

Earlier quoted context omitted.

I assume your product does some kind of entity disambiguation and/or link to an ontology? Spacy doesn't provide this out of the box either, AFAICT. Can you share more info about how you do it?

We don't provide entity disambiguation out of a box. It's more of a on request for Enterprise clients. But overall, entity disambiguation is one of the most useful and difficult tasks in the NLP. SpaCy supports entity linking via knowledge base: https://spacy.io/api/entitylinker

That might be the killer feature from what I've heard.

Re: Advanced NLP with spaCy v3

#29

Post by Der_Einzige »

dang wrote: post #16
master_yoda_1 wrote: post #10

Quoted post unavailable.

" Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something. " https://news.ycombinator.com/newsguidelines.html

As usual, dang is wrong and not moderating effectively. This is not a shallow comment but a legitimate concern about spaCy, and to a lesser extent other NLP tools such as NLTK. Most of the tooling around them that people end up using really is nothing more than wrappers around other tools. See the default tokenizers or models utilized by these tools.

And yes, even if spaCy is not making money itself, you can bet that the other paid for tools that they sell are.

Re: Advanced NLP with spaCy v3

#30

Post by dang »

Der_Einzige wrote: post #29
dang wrote: post #16

Earlier quoted context omitted.

" Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something. " https://news.ycombinator.com/newsguidelines.html

As usual, dang is wrong and not moderating effectively. This is not a shallow comment but a legitimate concern about spaCy, and to a lesser extent other NLP tools such as NLTK. Most of the tooling around them that people end up using really is nothing more than wrappers around other tools. See the default tokenizers or models utilized by these tools. And yes, even if spaCy is not making money itself, you can bet that…

Actually if the GP had posted this critique instead of a shallow, reductionist internet dismissal ("just want to sell the hype"), that would have been fine. Thoughtful critique is welcome—it just requires higher-quality comments than that.
Post reply on HN