71–80 of 82 posts
I'm curious what sort of NLP use cases people are solving. How are people finding business value in these models and pipelines? We have looked at a number of uses and have found it hard to make a case for ROI. Wondering what's been working for folks.
Earlier quoted context omitted.
I've been using LASER from Facebook Research via https://github.com/yannvgn/laserembeddings to accept multi-lingual input in front of the the domain-specific models for recommendations and stuff (that are trained on English annotated examples).
This sounds interesting. Can you share more please? It sounds like there is some multilingual input text on the basis of which you make recommendations, but I think you would have called that a search engine rather than recommender.
My approach to AI is somewhat conservative because of working in a law-adjacent field where explainability is paramount. When it comes to getting "smart" I prefer forward-chaining logic over facts, and facts include predictions from models too. But at least there is a "judge"/engine to coordinate how the predictions from the ensemble of models maps to actions. I love me some pertained neural nets, but use them more as black box appliances.
I have been using Spacy3 nightly for a while now. This is game changing. Spacy3 practically covers 90% of NLP use-cases with near SOTA performance. The only reason to not use it would be if you are literally pushing the boundaries of NLP or building something super specialized. Hugging Face and Spacy (also Pytorch, but duh) are saving millions of dollars in man hours for companies around the world. They've been a rev…
Everything in the above paragraph sounds like a hyped overstatement. None of it is. As someone that's worked on some rather intensive NLP implementations, Spacy 3.0 and HuggingFace both represent the culmination of a technological leap in NLP that started a few years ago with the advent of transfer learning in NLP. The level of accessibility to the masses these libraries offer is game-changing and democratizing.
Earlier quoted context omitted.
Everything in the above paragraph sounds like a hyped overstatement. None of it is. As someone that's worked on some rather intensive NLP implementations, Spacy 3.0 and HuggingFace both represent the culmination of a technological leap in NLP that started a few years ago with the advent of transfer learning in NLP. The level of accessibility to the masses these libraries offer is game-changing and democratizing.
I use HuggingFace for most of the NLP modeling work, it seems like Spacy 3.0 provides a framework for many different libraries including HuggingFace, StreamLit, Ray and WnB, but if you know how to use these individually, what does Spacy add to this?
Let’s say you wanted to build a system where you used an efficient bag-of-words text classifier to select paragraphs that might have information of interest, and then you wanted to run an entity recognizer and recognise relation triples between predicates and pairs of entities. When extracting the triples, you want to use the lemma of the relation word, so you want to map “dove” to “dive” when it’s a verb etc. It’s possible to build this system directly using PyTorch modules for the various model parts, but you’ll need to write the various bits of logic to string together the model predictions yourself, and for tasks like lemmatization that are pretty easy, you’ll struggle to find existing systems that you’ll actually want to use. A lemmatization system that’s published for PyTorch will probably be designed for languages where lemmatization is really hard, but for English it’s really easy.
spaCy has a good architecture and API for this system level stuff, where you’re putting together models into practical solutions. It also has a Doc object that makes it really easy to actually work with the system outputs, especially to relate multiple levels of annotation to each other.
Partly because orchestrating a number of models is kind of a hassle in lower-level frameworks, a lot of guides will encourage you to take entirely joint approaches to this type of system. In theory you can bypass problems like the lemmatization and NER if you take a sequence-to-sequence approach, and generate the relation triples as just arbitrary data. But this has a lot of limitations. It’s difficult to express structural constraints that you know should hold about the triples, the system will be much much slower, and might require vastly more training data. It’s also difficult to divide the task up between different people, and it’ll be difficult to analyse the system errors, iterate on individual parts, or inject rule logic to ensure certain invariants about the output. All these facts about joint approaches increase the risk of the project failing; they add large uncertainties that keep projects from getting out of the prototype stage.
I think I read somewhere that spaCy was going to have named entity disambiguation at some point, with named entities having links to knowledge bases like Wikidata or DBpedia. That’s something that paid NER services but that I haven’t found in open source libs, and would be really interesting IMO.
There's a component for Entity Linking available in spaCy, but you have to train it yourself, as the use-cases (type of entities, type of knowledge base etc) can vary greatly. See more here: https://spacy.io/api/entitylinker
Earlier quoted context omitted.
This sounds interesting. Can you share more please? It sounds like there is some multilingual input text on the basis of which you make recommendations, but I think you would have called that a search engine rather than recommender.
That's true, I'm making recommendations based on Multinomial Naive Bayes (and SGDClassifier) over custom TF-IDF bags of words, so it is like search plus text classification. And some endpoints do just check the cosine or Jaccard distance between things. There is a lot of overlap between search and NLP. My approach to AI is somewhat conservative because of working in a law-adjacent field where explainability is paramo…
I've been using the v3 nightly version for 2 months and it works like a charm. I'm now training models with v3 and using them in production without any issue.
Great job!
Earlier quoted context omitted.
That's true, I'm making recommendations based on Multinomial Naive Bayes (and SGDClassifier) over custom TF-IDF bags of words, so it is like search plus text classification. And some endpoints do just check the cosine or Jaccard distance between things. There is a lot of overlap between search and NLP. My approach to AI is somewhat conservative because of working in a law-adjacent field where explainability is paramo…
Which forward-chaining engine do you use? Something based on prolog?
There's also https://github.com/jruizgit/rules but haven't tried it yet.
Earlier quoted context omitted.
Which forward-chaining engine do you use? Something based on prolog?
Currently https://github.com/nilp0inter/experta but https://github.com/noxdafox/clipspy seems nice, I just shied away from using it due to uneasiness about FFI and debugging, even though the original CLIPS is still awesome and has a very interesting manual. There's also https://github.com/jruizgit/rules but haven't tried it yet.