num_workers in VideoClips initialization

[ekosman]

I've been trying to create a VideoClips object from a very large dataset, however during the process my system restarts for some unknown reason.
After looking in here I figured out that I never get into the loop (line 85).
Running torch.get_num_threads() gives me 44, so I changed this to 16 manually and it works fine now.

  import torch.utils.data
        dl = torch.utils.data.DataLoader(
            DS(self.video_paths),
            batch_size=16,
            num_workers=torch.get_num_threads(), # changed this to 16
            collate_fn=lambda x: x)

I think it would be better to let the user choose the num_workers parameter by passing it to VideoClips constructor.

[fmassa]

Yes, I agree that this option should be exposed, but I was not very convinced that it should be an argument to VideoClips constructor.

But I don't see any better solution than that, so exposing it to the constructor might be the best thing we can do, with a default value of 1 maybe?

[ekosman]

@fmassa Another option is to use torch.set_num_workers(), but I don't know how this would affect the behavior of other functions. Therefore I think the best thing to do is exposing it to the constructor with the default value as you suggested.

[fmassa]

Yes, I agree. I'd be happy to accept a PR adding the constructor argument to VideoClips

[fmassa]

Fixed in #1359