Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SpaCy PyTorch Transformers

explosion.ai

Post reply on HN
Print view View original

11–12 of 12 posts

Re: SpaCy PyTorch Transformers

#11

Post by p1esk »

3wolf wrote: post #3

I'm surprised transfer learning via fine-tuning large transformer models hasn't taken off more in the public consciousness a la image recognition models. In my experience, the results can be staggering with very small amounts of training data.

It has taken off. Pretty much everyone and their grandma has fine tuned gpt-2 to generate all kinds of stuff, even poetry: https://www.gwern.net/GPT-2

Re: SpaCy PyTorch Transformers

#12

Post by bigmit37 »

ma2rten wrote: post #8
3wolf wrote: post #3

I'm surprised transfer learning via fine-tuning large transformer models hasn't taken off more in the public consciousness a la image recognition models. In my experience, the results can be staggering with very small amounts of training data.

It's a very new thing. Pretrained ImageNet models were first released around 2011. Pretrained transformer models have only been released recently. Also, transformers are only useful for short text, not full documents (AFAIK).

are transformer models are the text version of imagenet model? The first time I am hearing this term.
Post reply on HN