If all the gradient inputs of some grad_op are marked as no_gradient, which means all of them can be considered as zeros. For grad_ops are in essence the propagation of gradients, all the outputs are definitely zeros when all gradient inputs are zeros. Therefore the grad_op can also be skipped.