-
Notifications
You must be signed in to change notification settings - Fork 5.9k
Closed
Description
The current backward algorithm in backward.cc has several problems:
- There are two kinds of operators, computational and control flow. The backward logic of these two kinds could be significantly different.
- We should make users be able to customize while backwards a neural network.
- The
error clippingwill addcropoperators when backward.
- The
- The variable name is not the only key of the gradient in
no_grad_set. It should be a pair of variable name and its block id. - We should propagate
no_gradafter renaming the variable.