change call_delegate_autograd (#3073)

Summary:
Pull Request resolved: #3073

Some changes angela told me to make 😂

Reviewed By: angelayi

Differential Revision: D56222503

fbshipit-source-id: ab1e5194492df439effab550781f056d12eaba53

Author: mcr229

exir/delegate.py

@@ -102,7 +102,7 @@ def fake_requires_grad(var):
                         var.requires_grad = True
                 return var
 
-            return pytree.tree_map(fake_requires_grad, res)
+            return pytree.tree_map_only(torch.Tensor, fake_requires_grad, res)
 
         return res