Earlier quoted context omitted.
Huggingface fills the need for task based prediction when you have a GPU. With model distillation, you can make models that annotate hundreds of sentences per second on a single CPU with a library like Huggingface Transformers. For instance, one of my distilled Dutch multi-task syntax models (UD POS, language-specific POS, lemmatization, morphology, dependency parsing) annotates 316 sentences per second with 4 thread…
Interesting. Did you start from a Distilled base model (like DistilRoBerta), or did you distill your fine-tuned model?
Sorry for the late reply. I distilled from my own finetuned XLM-RoBERTa model.