LightlyTrain X DINOv2: Smarter Self-Supervised Pretraining, Faster

Nice to see that distillation also works well. That means I can take the huge pretrained dinov2 models and distill them to my custom model architecture. Very nice!