Closed
Description
I would like to introduce an elegant API for different learning rates in different layers. It will be an optimizer wrapper going like this
input = Input(input_shape)
conv = Conv2D(filters, strides=strides, padding=padding)
x = conv(input)
x = Dense(num_class, activation="softmax")(x)
model = Model(input, x)
MultiLRAdam = MultiLR(Adam)
lr_multiplier = {conv.kernel: 0.5, conv.bias: 0.1}
model.compile(loss="categorical_crossentropy", optimizer=MultiLRAdam(lr=1e-4, lr_multiplier=lr_multiplier))
Details can be found in this proposal.
Metadata
Metadata
Assignees
Labels
No labels