Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SpaCy v3.0 Nightly

explosion.ai

Post reply on HN
Print view View original

21–30 of 31 posts

Re: SpaCy v3.0 Nightly

#21

Post by syllogism »

ppod wrote: post #19

I love Spacy, but I was recently trying to embed documents with it and it seems that it uses word-vector averaging for that. Is it possible to do sentence or paragraph embedding like sentenceBert?

Yes, that's the sort of thing that's now much better. See here for discussion about using transformers in v3: https://nightly.spacy.io/usage/embeddings-transformers

Document similarity is a tricky thing to get into the main API though, which is organised around the idea of sending a Doc through a sequence of steps, each of which adds or updates annotations. The signature for document similarity is fundamentally different. In the end we decided to not try to shoe-horn it in. Not everything needs to be one function call. So document relation predictions (including document similarity) wouldn't be a standard pipeline component.

Re: SpaCy v3.0 Nightly

#22

Post by cjv »

I'm always impressed by SpaCy's web design, it's so distinctive and the documentation is a pleasure to navigate and read.

Re: SpaCy v3.0 Nightly

#23

Post by The_rationalist »

minimaxir wrote: post #9
The_rationalist wrote: post #4

Earlier quoted context omitted.

You can downvote but this is the cold hard truth

You are likely getting downvoted because your complaint doesn't make any sense. Autocompletion isn't a standard feature of any major NLP library, and blaming it on Python (currently the most popular language for NLP) isn't a good argument.

Autocompletion isn't a standard feature of any major NLP library, and blaming it on Python (currently the most popular language for NLP) isn't a good argument.

I install a package (spacy) I don't get any autocompletion of the API from the main spacy object and any other subsequent spacy data structure. Indeed autocompletion is usually on the language side but here it's a combination of the weaknesses of python and of the lazyness of spacy developers by making everything dynamic they broke the autocompletion that is normally working in python. Therefore it's not production ready, what else is there to say? I tried both pycharm and vscode

Re: SpaCy v3.0 Nightly

#24

Post by mjhirn »

cjv wrote: post #22

I'm always impressed by SpaCy's web design, it's so distinctive and the documentation is a pleasure to navigate and read.

Ines Montani is the woman behind the awesome web design. I worked with her previously - she is amazing!

Re: SpaCy v3.0 Nightly

#25

Post by The_rationalist »

youngNed wrote: post #16
The_rationalist wrote: post #3

Earlier quoted context omitted.

High quality what? Spacy doesn't has autocompletion and it's the 21st century, only a python package can achieve such prodigy

Now we're all playing "what is (s)he talking about?" I'm gonna go for: "spyder" Final answer

Ok so I'll break it down for you: Spacy doesn't has autocompletion means that when you use Spacy or any other data structure from spacy you won't have any autocompletion that allow you to know the API you're using. Not on any IDE I've tried and even on the simplest examples.

Was it too hard to understand or the reality is too ridiculous to even think about it being true? A friend of me had the same problem so I think everybody has it, if you have autocompletion when using Spacy then say it so you would add intellectual value to the thread by constraining the scope of this existential bug for the python Nlp industry.

Re: SpaCy v3.0 Nightly

#26

Post by syllogism »

The_rationalist wrote: post #23
minimaxir wrote: post #9

Earlier quoted context omitted.

You are likely getting downvoted because your complaint doesn't make any sense. Autocompletion isn't a standard feature of any major NLP library, and blaming it on Python (currently the most popular language for NLP) isn't a good argument.

Autocompletion isn't a standard feature of any major NLP library, and blaming it on Python (currently the most popular language for NLP) isn't a good argument. I install a package (spacy) I don't get any autocompletion of the API from the main spacy object and any other subsequent spacy data structure. Indeed autocompletion is usually on the language side but here it's a combination of the weaknesses of python and of…

> lazyness of spacy developers by making everything dynamic they broke the autocompletion that is normally working in python.

Actually it's the opposite of "making everything dynamic". spaCy is mostly implemented in Cython, which is a language for writing C extensions for Python. So things are statically typed and memory managed, which sometimes doesn't work well with inspect. We set whatever compilation flags we can to make the annotations passed through though. It's definitely not a question of laziness though...You could say we worked extremely hard to break things in this particular way.

In v3 we've been able to finally ditch Python 2 support, which means we're able to embrace type annotations. Our machine learning library, https://thinc.ai, is extensively type-annotated and actually you can get type hints and type errors for composing layers incorrectly.

Personally I use vim and don't use autocompletion, I find it makes the typing laggy and I've just never really wanted it. We do have developers on the team who use autocompletion though and I think spaCy actually supports it better than you suggest, I'm not sure whether you had a specific problem, but maybe see whether it works if you install spacy-nightly.

Btw I do think you've come across as a weird jerk in this thread though? You might want to read over your messages and think about that. Like, I (and I think most others) genuinely didn't get what you meant in your first few messages; I thought you were saying spaCy didn't let you _build_ autocomplete.

Re: SpaCy v3.0 Nightly

#27

Post by nl »

The_rationalist wrote: post #3
eob wrote: post #2

I’m constantly blown away at how high quality SpaCy is with such a small team. Everything: from API design, to documentation, to education, to tool ecosystem. Congrats on the (preview) release and thank you for the great software. Exciting to see SpaCy pull transformers into the mix!

High quality what? Spacy doesn't has autocompletion and it's the 21st century, only a python package can achieve such prodigy

I was sure I've had autocomplete work for me, so I just tested with this nightly release and VS.Code and it works fine for me.

Here's a screenshot: https://imgur.com/a/9bcgPkz

There's a lot that can go wrong with getting Python autocomplete working, and it's possible you are seeing something different. But I would suggest it is environmental rather than something Spacy is doing.

Re: SpaCy v3.0 Nightly

#28

Post by nl »

The_rationalist wrote: post #23
minimaxir wrote: post #9

Earlier quoted context omitted.

You are likely getting downvoted because your complaint doesn't make any sense. Autocompletion isn't a standard feature of any major NLP library, and blaming it on Python (currently the most popular language for NLP) isn't a good argument.

Autocompletion isn't a standard feature of any major NLP library, and blaming it on Python (currently the most popular language for NLP) isn't a good argument. I install a package (spacy) I don't get any autocompletion of the API from the main spacy object and any other subsequent spacy data structure. Indeed autocompletion is usually on the language side but here it's a combination of the weaknesses of python and of…

I mentioned this elsewhere in thread, but here is an image of it working with VS.Code: https://imgur.com/a/9bcgPkz

Re: SpaCy v3.0 Nightly

#29

Post by wokwokwok »

syllogism wrote: post #26
The_rationalist wrote: post #23

Earlier quoted context omitted.

Autocompletion isn't a standard feature of any major NLP library, and blaming it on Python (currently the most popular language for NLP) isn't a good argument. I install a package (spacy) I don't get any autocompletion of the API from the main spacy object and any other subsequent spacy data structure. Indeed autocompletion is usually on the language side but here it's a combination of the weaknesses of python and of…

> lazyness of spacy developers by making everything dynamic they broke the autocompletion that is normally working in python. Actually it's the opposite of "making everything dynamic". spaCy is mostly implemented in Cython, which is a language for writing C extensions for Python. So things are statically typed and memory managed, which sometimes doesn't work well with inspect. We set whatever compilation flags we can…

fwiw, I love spacy but the autocomplete has never work for me in pycharm either.

Many autocomplete implementations seem to struggle with anything that isn’t “pure python”; pyspark is notorious for it too.

...but, the type annotations do work! That’s definitely the right way to fix it.

Can’t wait~

Re: SpaCy v3.0 Nightly

#30

Post by Barrin92 »

The_rationalist wrote: post #25
youngNed wrote: post #16

Earlier quoted context omitted.

Now we're all playing "what is (s)he talking about?" I'm gonna go for: "spyder" Final answer

Ok so I'll break it down for you: Spacy doesn't has autocompletion means that when you use Spacy or any other data structure from spacy you won't have any autocompletion that allow you to know the API you're using. Not on any IDE I've tried and even on the simplest examples. Was it too hard to understand or the reality is too ridiculous to even think about it being true? A friend of me had the same problem so I think…

>Spacy doesn't has autocompletion

Python libraries don't implement autocompletion, language servers or completion engines do. I suggest you configure your editor correctly, because spacy works just fine with vscode vim, or whatever have you.

Post reply on HN