cocotools breakes when loading results in version 2.0.9, after PR that added info dict to results

[tsahiasher]

🐛 Describe the bug

With ppwwyyxx/cocoapi#26 pycocotools is copying info dict to results, so it has to be created in coco_utils.py::convert_to_coco_api while creating the dataset

Versions

pycocotools==2.0.9

[NicolasHug]

Hi @tsahiasher , can you provide a minimal reproducing example of the issue you are mentioning?

[tsahiasher]

Here is a minimal example to reproduce the problem:

The call to COCO.loadRes depends on the model finding results, and since this is random data there are no results, what i did was to change line 37 in coco_eval.py from

coco_dt = COCO.loadRes(self.coco_gt, results) if results else COCO()

to

coco_dt = COCO.loadRes(self.coco_gt, results)

running this script produced the error:

Traceback (most recent call last):
  File ".\test.py", line 31, in <module>
    coco_evaluator = evaluate(model, data_loader_test, device)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\AppData\Local\Programs\Python\Python312\Lib\site-packages\torch\utils\_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File ".\engine.py", line 96, in evaluate
    coco_evaluator.update(res)
  File ".\coco_eval.py", line 37, in update
    coco_dt = COCO.loadRes(self.coco_gt, results)# if results else COCO()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\AppData\Local\Programs\Python\Python312\Lib\site-packages\pycocotools\coco.py", line 314, in loadRes
    res.dataset['info'] = copy.deepcopy(self.dataset['info'])
                                        ~~~~~~~~~~~~^^^^^^^^
KeyError: 'info'

---

import torch
import torchvision
from engine import evaluate
from torch.utils.data import Dataset, DataLoader

def collate_fn(batch):
    return tuple(zip(*batch))

class ClassDataset(Dataset):
    def __getitem__(self, idx):
        target = {}
        img, boxes = torch.rand(3, 600, 1200), torch.rand(1, 4)
        boxes[:, 2:4] = boxes[:, 0:2] + boxes[:, 2:4]
        target['boxes'] = boxes
        target['labels'] = torch.randint(1, 100, (1,))
        target['area'] = torch.randint(0, 100, (1,))
        target['iscrowd'] = torch.zeros(1, dtype=torch.int64)
        target['image_id'] = torch.randint(0, 100, (1,))
        return img, target

    def __len__(self):
        return 1

dataset_test = ClassDataset()
data_loader_test = DataLoader(dataset_test, batch_size=1, collate_fn=collate_fn)
device = torch.device('cpu')
model = torchvision.models.detection.fasterrcnn_resnet50_fpn()
model.eval()
coco_evaluator = evaluate(model, data_loader_test, device)

[chengolivia]

I can also replicate this with a fake model that generates one result to trigger loadRes().

class PredictiveModel(torch.nn.Module):
    def __init__(self, base_model):
        super().__init__()
        self.base_model = base_model
        
    def forward(self, images, targets=None):
        if self.training:
            return self.base_model(images, targets)
        else:
            with torch.no_grad():
                predictions = self.base_model(images)
            for i, pred in enumerate(predictions):
                if len(pred['boxes']) == 0:
                    # Add at least one fake detection to trigger loadRes
                    pred['boxes'] = torch.tensor([[100, 100, 200, 200]], dtype=torch.float32)
                    pred['labels'] = torch.tensor([1], dtype=torch.int64)
                    pred['scores'] = torch.tensor([0.5], dtype=torch.float32)
                
                # Ensure all required fields are present
                if 'scores' not in pred:
                    pred['scores'] = torch.rand(len(pred['boxes']))
                    
            return predictions
        
dataset_test = ClassDataset()
data_loader_test = DataLoader(dataset_test, batch_size=1, collate_fn=collate_fn, shuffle=False)
device = torch.device('cpu')
base_model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)
model = PredictiveModel(base_model)
model.eval()
coco_evaluator = evaluate(model, data_loader_test, device)

  File "/Users/oliviacheng/projects/oss/pytorch/venv/lib/python3.13/site-packages/pycocotools/coco.py", line 314, in loadRes
    res.dataset['info'] = copy.deepcopy(self.dataset['info'])
                                        ~~~~~~~~~~~~^^^^^^^^
KeyError: 'info'

Happy to contribute a PR for this, if desired.