SpaCy 3.0 • Hacker News Board

Re: SpaCy 3.0

#31

Post by tedivm » Mon, Feb 01, 2021, 5:55 PM UTC

syllogism wrote: post #25

williamtrask wrote: post #22

Earlier quoted context omitted.

Man I wish they could be compensated remotely in proportion to that. Matthew Honnibal and team are wizards who have been working really hard for a really long time. Maybe there's a comp strategy i don't know about - but they've created SO much value for the world.

Thanks for the love :). For the record yes we've been working hard, but also yes, we've been doing well from it. I will say that people are using spaCy for free because that is what we asked them to do. I chose to make the library free and open-source when I first released it because I had the idea that I would be able to make that work out for me, if I could make this thing that would be useful to people and if they…

The only thing that Prodigy is missing is a team based workflow. We've been on the beta list for awhile for it, and are excited for it to come out- but without having a concept of users we've had to use other tools that aren't as polished on the annotation side but which hit our compliance needs.

Re: SpaCy 3.0

#32

Post by number6 » Mon, Feb 01, 2021, 6:05 PM UTC

ZeroCool2u wrote: post #11

SpaCy and HuggingFace fulfill practically 99% of all our needs for NLP project at work. Really incredible bodies of work. Also, my team chat is currently filled with people being extremely stoked about the SpaCy + FastAPI support! Really hope FastAPI replaces Flask sooner rather than later.

As someone who never used NLP, what is it used for? Or better what is SpaCy used for - I know that it can generate Texts... but how would I use it in a business?

Re: SpaCy 3.0

#33

Post by t-vi » Mon, Feb 01, 2021, 6:15 PM UTC

Abishek_Muthian wrote: post #5

mark_l_watson wrote: post #3

Thanks to the SpaCy team! I spent a lot of time over about 20 years working on my own NLP tools. I stopped doing that and mostly now just use SpaCy (and sometimes Huggingface and Apple’s NLP models).

Have you compared SpaCy with Apple's NLP on M1? I presume GPU is not being used by SpaCy on M1.

Given that SpaCy uses PyTorch, that is being worked on.

Re: SpaCy 3.0

#34

Post by tiangolo » Mon, Feb 01, 2021, 6:23 PM UTC

gavinray wrote: post #23

ZeroCool2u wrote: post #11

SpaCy and HuggingFace fulfill practically 99% of all our needs for NLP project at work. Really incredible bodies of work. Also, my team chat is currently filled with people being extremely stoked about the SpaCy + FastAPI support! Really hope FastAPI replaces Flask sooner rather than later.

There's some native SpaCy + FastAPI integration being created?! This sounds mindblowing, off to Google I go...

With the new "spaCy project" it's easy to generate the boilerplate for many spaCy related projects, including a FastAPI app to serve models.

But apart from that, there's actually not really much to integrate, as it all just works.

Both libraries are actually (well ) designed to be decoupled. So that you can mix them independently with anything else.

Note: I'm the creator of FastAPI, and also a developer at Explosion (spaCy's home).

Re: SpaCy 3.0

#35

Post by tiangolo » Mon, Feb 01, 2021, 6:23 PM UTC

thesehands wrote: post #24

gavinray wrote: post #23

Earlier quoted context omitted.

There's some native SpaCy + FastAPI integration being created?! This sounds mindblowing, off to Google I go...

The author of FastAPI https://twitter.com/tiangolo is a Spacy employee

Yep, that's me, I work at Explosion (spaCy's home)!

Re: SpaCy 3.0

#36

Post by t-vi » Mon, Feb 01, 2021, 6:24 PM UTC

syllogism wrote: post #18

pabs3 wrote: post #15

Please note that Explosion does not like redistribution of SpaCy, they expect everyone to only use the builds they produce, so it would not be a good idea to package it for your favourite distro.

I'm sorry that this conflicted with your plans, but I feel strongly that distributing Python libraries via system package managers such as apt is very bad for users. The pain is felt especially by users who are relatively new to Python, who will end up with their system Python in a confusing state that is difficult to correct. We of course encourage anyone to clone the repo or install from an sdist if they want to co…

That is sad, I would use SpaCy more if it had Debian packages (in particular in Debian). Python stuff packaged by Debian seems to work very well for me and has been for years.

Re: SpaCy 3.0

#37

Post by gillesjacobs » Mon, Feb 01, 2021, 6:44 PM UTC

langitbiru wrote: post #21

So with SpaCy 3.0, HuggingFace, do we still have a reason to use NLTK? Or they complement each other? Right now, I lost track of the progress in NLP.

NLTK is showing its age. In my information extraction pipelines, the heavy lifting for modelling is done by SpaCy, AllenNLP, and Huggingface (and Pytorch or TF ofc).

I only use NLTK since it has some base tools for low-resource languages for which noone has pretrained a transformer model or for specific NLP-related tasks. I still use their agreement metrics module, for instance. But that's about it. Dep parsing, NER, lemmatising and stemming is all better with the above mentioned packages.

Re: SpaCy 3.0

#38

Post by gillesjacobs » Mon, Feb 01, 2021, 6:50 PM UTC

Excited for this release and I will start integrating this in my own information extraction pipelines immediately. Thanks, Explosion team, got your stickers on my notebook!

The new configuration approach looks familiar to AllenNLP and that's great. Loose-coupling of model submodules with flexible config should be standard in NLP. I am happy that more libraries are integrating these concepts.

Re: SpaCy 3.0

#39

Post by Xenoamorphous » Mon, Feb 01, 2021, 7:31 PM UTC

I think I read somewhere that spaCy was going to have named entity disambiguation at some point, with named entities having links to knowledge bases like Wikidata or DBpedia. That’s something that paid NER services but that I haven’t found in open source libs, and would be really interesting IMO.

Re: SpaCy 3.0

#40

Post by ZeroCool2u » Mon, Feb 01, 2021, 7:38 PM UTC

number6 wrote: post #32

ZeroCool2u wrote: post #11

SpaCy and HuggingFace fulfill practically 99% of all our needs for NLP project at work. Really incredible bodies of work. Also, my team chat is currently filled with people being extremely stoked about the SpaCy + FastAPI support! Really hope FastAPI replaces Flask sooner rather than later.

As someone who never used NLP, what is it used for? Or better what is SpaCy used for - I know that it can generate Texts... but how would I use it in a business?

Obviously, it depends, but assuming you do have an NLP use case already, there are certain things that you will almost certainly have to do in your preprocessing regardless of your task. For example, sentence parsing. Writing your own basic sentence parser is fairly easy. Writing your own _good_ sentence parser is a nightmare akin to trying to parse HTML with regex. SpaCy provides a very good one for you. Down the line it will help you train a model on various tasks in a compute efficient manner as well. This is just a small example.