Add LightlyTrain Integration for Pretraining Support

[yutong-xiang-97]

LightlyTrain is a novel pretraining framework built with PyTorch. It lets you pretrain any computer vision model on your unlabeled data, by leveraging distillation from powerful vision models and using self-supervised learning. With only a few lines of code, the community can pretrain domain-specific backbones for any downstream task with a TIMM backbone and beyond.

LightlyTrain supports all TIMM models out-of-the-box. You can start pretraining simply by:

import lightly_train

if __name__ == "__main__":
    lightly_train.train(
        out="out/my_experiment",                # Output directory.
        data="my_data_dir",                     # Directory with images.
        model="timm/resnet18",                  # Pass the timm model.
    )

and reload the exported model by TIMM directly for your own use

import timm

model = timm.create_model(
  model_name="resnet18",
  checkpoint_path="out/my_experiment/exported_models/exported_last.pt",
)

You can also check our docs and product page for more details.

Changes

This PR contains the link to our GitHub Repo in the "Awesome PyTorch Resources - Training / Frameworks" section.