Code, data, and model weights for paper A General Framework for Jersey Number Recognition in Sports (Maria Koshkina, James H. Elder).
Image-level detection, localization and recognition (experiments on Hockey dataset):
- legibility classifier
- scene text recognition for jersey numbers
Tracklet-level detection, localization and recognition (experiments on SoccerNet dataset):
- occlusion/outlier removal using re-id features and fitting a Gaussian
- legibility classifier
- pose-guided RoI cropping
- scene text recognition for jersey numbers
- tracklet prediction consolidation
- pytorch 1.9.0
- opencv
Clone current repo. Create conda environment and install requirements. Code makes use of the several repositories. Run
python3 setup.py
to automatically clone, setup a separate conda environment for each and fetch models.
Alternatively, clone each of the following repo, setup conda environments for each following documentation in corresponding repo, and download models:
Should be in jersey-number-pipeline/sam. Repo: https://github.com/davda54/sam
Should be in jersey-number-pipeline/reid/centroids-reid. Repo: https://github.com/mikwieczorek/centroids-reid. Download centroid-reid model weights and place them under jersey-number-pipeline/reid/centroids-reid/models.
Should be in jersey-number-pipeline/pose/ViTPose. Repo: https://github.com/ViTAE-Transformer/ViTPose. Download ViTPose model weights and place them under jersey-number-pipeline/pose/ViTPose/checkpoints/.
We include the version of the PARSeq code that was used to fine-tune the jersey number model as part of this repo. The original PARSeq repo is https://github.com/baudm/parseq. Model weights should be downloaded and placed under jersey-number-pipeline/models/.
SoccerNet Jersey Number Recognition: https://github.com/SoccerNet/sn-jersey Download and save under /data subfolder.
- Weakly-labelled player images used to train legibility classifier can be downloaded here.
- Weakly-labelled jersey number crops used to fine-tune STR in LMDB format can be downloaded here.
Hockey (comprised of legibility dataset and jersey number dataset):
- Request access by contacting Maria Koshkina. Extract under data/Hockey subfolder.
Download and place under jersey-number-pipeline/models/.
Update configuration.py if required to set custom path to data or dependencies.
To run the full inference pipeline for SoccerNet:
python3 main.py SoccerNet test
To run legibility and jersey number inference for hockey:
python3 main.py Hockey test
Update actions in main.py actions list to run steps selectively.
Train legibility classifier:
python3 legibility_classifier.py --train --arch resnet34 --sam --data <new-dataset-directory> --trained_model_path ./experiments/hockey_legibility.pth
Fine-tune PARSeq STR for hockey number recognition:
python3 main.py Hockey train --train_str
Trained model will be under str/parseq/outputs
To train legibility classifier and jersey number recognition for SoccerNet, we first generate weakly labelled datasets and then use them to fine-tune. Weak labels are obtained by using models trained on hockey data.
Train legibility classifier for it:
python3 legibility_classifier.py --finetune --arch resnet34 --sam --data <new-dataset-directory> --full_val_dir
<new-dataset-directory>/val --trained_model_path ./experiments/hockey_legibility.pth --new_trained_model_path ./experiments/sn_legibility.pth
Fine-tune PARSeq on weakly-labelled SoccerNet data:
python3 main.py SoccerNet train --train_str
Trained model will be under str/parseq/outputs.
@InProceedings{Koshkina_2024_CVPR,
author = {Koshkina, Maria and Elder, James H.},
title = {A General Framework for Jersey Number Recognition in Sports Video},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2024},
pages = {3235-3244}
}
We would like to thank authors of the following repositories:
This work is licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported License.
