Skip to content

Update references to use the new Model Registration API #6369

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Aug 8, 2022

Conversation

datumbox
Copy link
Contributor

@datumbox datumbox commented Aug 4, 2022

Depends on #6364

This PR updates all the __dict__[model_name] occurrences within our code with a get_model() call.

To confirm the reference scripts still work we run the same experiments as #5618 and #6241. Minor variations on accuracy are expected because we use multiple GPUs and batch-sizes > 1. The important think to verify is that the models are initialized properly and provide similar accuracies.

torchrun --nproc_per_node=2 train.py --model efficientnet_b0 --test-only --weights EfficientNet_B0_Weights.IMAGENET1K_V1
Test:  Acc@1 77.684 Acc@5 93.536

python -u train_quantization.py --test-only --weights ResNet18_QuantizedWeights.IMAGENET1K_FBGEMM_V1 --model resnet18 --device cpu
Test:  Acc@1 69.494 Acc@5 88.882

torchrun --nproc_per_node=2 train.py --model retinanet_resnet50_fpn --test-only --weights RetinaNet_ResNet50_FPN_Weights.COCO_V1
IoU metric: bbox
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.363
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.557
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.382
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.193
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.400
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.490
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.314
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.500
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.540
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.340
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.581
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.696

torchrun --nproc_per_node=2 train.py --model lraspp_mobilenet_v3_large --test-only --weights LRASPP_MobileNet_V3_Large_Weights.COCO_WITH_VOC_LABELS_V1
global correct: 91.3
average row correct: ['94.6', '84.8', '68.5', '75.6', '58.3', '38.4', '78.8', '54.1', '90.1', '37.9', '80.6', '60.7', '79.5', '77.7', '81.9', '87.5', '51.5', '81.1', '60.8', '84.2', '54.3']
IoU: ['90.3', '68.5', '56.2', '57.7', '47.8', '33.0', '72.1', '46.1', '79.2', '31.3', '66.0', '35.5', '61.7', '63.1', '71.4', '76.7', '28.9', '62.7', '44.3', '69.9', '45.7']
mean IoU: 57.5

torchrun --nproc_per_node 1 --nnodes 1 train.py --val-dataset sintel --batch-size 1 --dataset-root $dataset_root --model raft_large --weights Raft_Large_Weights.C_T_SKHT_V2
Batch-processed 1040 / 1041 samples. Going to process the remaining samples individually, if any.
Sintel val clean epe: 0.6502	1px: 0.9255	3px: 0.9701	5px: 0.9805	per_image_epe: 0.6501	f1: 2.6801
Sintel val final Total time: 0:02:40
Batch-processed 1040 / 1041 samples. Going to process the remaining samples individually, if any.
Sintel val final epe: 1.0195	1px: 0.8898	3px: 0.9493	5px: 0.9660	per_image_epe: 1.0190	f1: 4.5747

torchrun --nproc_per_node=8 train.py \
    --batch-size=64 --test-only \
    --data-path="/datasets/clean_kinetics_400/" \
    --clip-len 16 --frame-rate 15 --clips-per-video 5 \
    --cache-dataset \
    --model mc3_18 --weights="MC3_18_Weights.DEFAULT" 
 * Clip Acc@1 52.820 Clip Acc@5 75.228
 * Video Acc@1 63.950 Video Acc@5 84.125

@datumbox datumbox requested a review from jdsgomes August 8, 2022 13:02
Copy link
Contributor

@jdsgomes jdsgomes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@datumbox datumbox merged commit 1d0786b into pytorch:main Aug 8, 2022
@datumbox datumbox deleted the references/adopt_new_api branch August 8, 2022 15:02
facebook-github-bot pushed a commit that referenced this pull request Aug 23, 2022
Summary:
* Expose on Hub the public methods of the registration API

* Limit methods and update docs.

* Update references to use the new Model Registration API

Reviewed By: datumbox

Differential Revision: D38824242

fbshipit-source-id: 8898a56115b356ef70f03d347550412fc816e0e0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants