Migration of keras-team/keras-contrib to tensorflow/addon

[gabrieldemarmiesse]

Hello all,

Keras-contrib host many good features. We'd like to keep them. Since we're focusing on tensorflow, and will deprecate the multi-backend Keras, we need to think about what will happen to other keras-related projets.

The goal of keras-contrib and tensorflow/addons are very similar, but tensorflow/addons benefits from a better support (read workforce) from google. With many processes already in place to make the maintenance easier.

As such, we'll deprecate keras-contrib. People who would like to keep a feature and are ready to maintain it should do a pull request to tensorflow/addons.

We don't have a timeline yet as of when we get this repository out of github.com/keras-team but it will happen eventually.

If you have any questions concerning the migration or if you need help with it, feel free to comment below.

[lingvisa]

hi, I would like to be able to do this: "from keras_contrib.layers import CRF", as before.

[gabrieldemarmiesse]

The CRF is available in tf.addons. See tensorflow/addons#314

[lingvisa]

Hi, @gabrieldemarmiesse, why doesn't it have a CRF layer implementation in tf.addons.text. The original one has this, as you can see: https://github.com/keras-team/keras-contrib/blob/master/keras_contrib/layers/crf.py

[seanpmorgan]

Hi @lingvisa we currently have CRF layer under review:
https://github.com/tensorflow/addons/pull/377/files but as I mentioned in the other thread the under review implementation seems overly complicated due to using ported tf.contrib CRF functions. I've made a note in that PR to consider using keras-contrib version.

Separately, TF Addons is happy to review any keras-contrib functionality that has proved useful and well tested as it should most likely align with our API formats. Looking forward to merging a lot of the great functionality of this repo with the ability to write custom TF ops/kernels in our library.

[lingvisa]

@gabrieldemarmiesse That would be great, if the layer can be added. Thanks.

[gabrieldemarmiesse]

@lingvisa We would need to find a maintainer for the CRF layer in tf addons to add the layer. I'm too busy with keras to do that at the moment. Hopefully, someone else will volunteer to do it.

[RaphaelMeudec]

@gabrieldemarmiesse Is there a list of features in keras-contrib that should be moved to tensorflow/addons? Can dedicate some time to migrate stuff over the next few weeks

[gabrieldemarmiesse]

I think the problem is that none of knows exactly which functions/classes are in keras-contrib and which functions/classes are in tensorflow-addons. I didn't have the time to look into it. If we had two lists we could compare easily and then know what to migrate. That's not super fun to do sadly :(

[seanpmorgan]

Generated components using the __init__ files of each repo:

TensorFlow Addons:

activations.gelu
activations.hardshrink
activations.lisht
activations.softshrink
activations.sparsemax
activations.tanhshrink	
activations.rrelu

image.dense_image_warp
image.interpolate_bilinear
image.euclidean_dist_transform
image.adjust_hsv_in_yiq
image.random_hsv_in_yiq
image.mean_filter2d
image.median_filter2d
image.rotate
image.transform
image.sparse_image_warp
image.interpolate_spline
image.translate
image.connected_components

layers.GeLU
layers.Maxout
layers.GroupNormalization
layers.InstanceNormalization
layers.CorrelationCost
layers.PoincareNormalize
layers.Sparsemax
layers.WeightNormalization

losses.ContrastiveLoss
losses.SigmoidFocalCrossEntropy
losses.LiftedStructLoss
losses.NpairsLoss
losses.NpairsMultilabelLoss
losses.SparsemaxLoss
losses.TripletSemiHardLoss

metrics.CohenKappa
metrics.F1Score
metrics.FBetaScore
metrics.RSquare
metrics.MultiLabelConfusionMatrix

optimizers.ConditionalGradient
optimizers.LazyAdam
optimizers.Lookahead
optimizers.MovingAverage
optimizers.RectifiedAdam
optimizers.AdamW
optimizers.SGDW

rnn.LayerNormLSTMCell
rnn.NASCell

seq2seq.AttentionMechanism
seq2seq.AttentionWrapper
seq2seq.AttentionWrapperState
seq2seq.BahdanauAttention
seq2seq.BahdanauMonotonicAttention
seq2seq.LuongAttention
seq2seq.LuongMonotonicAttention
seq2seq.hardmax
seq2seq.monotonic_attention
seq2seq.safe_cumprod
seq2seq.BasicDecoder
seq2seq.BasicDecoderOutput
seq2seq.BeamSearchDecoder
seq2seq.BeamSearchDecoderOutput
seq2seq.BeamSearchDecoderState
seq2seq.FinalBeamSearchDecoderOutput
seq2seq.gather_tree
seq2seq.gather_tree_from_array
seq2seq.tile_batch
seq2seq.BaseDecoder
seq2seq.Decoder
seq2seq.dynamic_decode
seq2seq.SequenceLoss
seq2seq.sequence_loss
seq2seq.CustomSampler
seq2seq.GreedyEmbeddingSampler
seq2seq.InferenceSampler
seq2seq.SampleEmbeddingSampler
seq2seq.Sampler
seq2seq.ScheduledEmbeddingTrainingSampler
seq2seq.ScheduledOutputTrainingSampler
seq2seq.TrainingSampler

text.crf_binary_score
text.crf_decode
text.crf_decode_backward
text.crf_decode_forward
text.crf_forward
text.crf_log_likelihood
text.crf_log_norm
text.crf_multitag_sequence_score
text.crf_sequence_score
text.crf_unary_score
text.viterbi_decode
text.skip_gram_sample
text.skip_gram_sample_with_text_vocab
text.parse_time

Keras-Contrib

activations.squash

applications.DenseNet
applications.ResNet
applications.ResNet18
applications.ResNet34
applications.ResNet50
applications.ResNet101
applications.ResNet152
applications.WideResidualNetwork
applications.NASNet
applications.NASNetLarge
applications.NASNetMobile

callbacks.SnapshotCallbackBuilder
callbacks.SnapshotModelCheckpoint
callbacks.DeadReluDetector
callbacks.CyclicLR
callbacks.TensorBoardGrouped

contraints.Clip

initializers.ConvolutionAware

layers.PELU
layers.SReLU
layers.Swish
layers.SineReLU
layers.CosineConv2D
layers.CosineConvolution2D
layers.SubPixelUpscaling
layers.CosineDense
layers.CRF
layers.Capsule
layers.InstanceNormalization
layers.GroupNormalization

losses.DSSIMObjective
losses.jaccard_distance
losses.crf_loss
losses.crf_nll

metrics.crf_accuracy
metrics.crf_marginal_accuracy
metrics.crf_viterbi_accuracy

optimizers.FTML
optimizers.Padam
optimizers.Yogi
optimizers.LARS

[gabrieldemarmiesse]

Thanks a bunch @seanpmorgan, I'll go over each of the items and then make a list with a proposition of what is relevant. Also since there are a lot of classes in keras-contrib that are not relevant anymore (e.g. TensorboardGrouped).

[gabrieldemarmiesse]

Can be migrated and have maintainers

callbacks.SnapshotCallbackBuilder      -> @titu1994

activations.squash                     -> @SriRangaTarun

layers.Capsule                         -> @SriRangaTarun
layers.SubPixelUpscaling               -> @titu1994
layers.Swish                           -> @gabrieldemarmiesse
layers.SineReLU (can be an activation) -> @wilderrodrigues

optimizers.Padam                       -> @MFreidank
optimizers.Yogi                        -> @MarcoAndreaBuchmann

---

Can be migrated but no maintainers

callbacks.DeadReluDetector
callbacks.CyclicLR

initializers.ConvolutionAware

layers.PELU
layers.SReLU
layers.CosineConv2D
layers.CosineDense
layers.CRF

optimizers.FTML
optimizers.LARS

losses.DSSIMObjective
losses.jaccard_distance
losses.crf_loss
losses.crf_nll

metrics.crf_accuracy
metrics.crf_marginal_accuracy
metrics.crf_viterbi_accuracy

@seanpmorgan do you want to make a selection? What do we do about features without maintainers?

[RaphaelMeudec]

@gabrieldemarmiesse imo, CyclicLR shouldn't be migrated as a callback, but rather as a LearningRateScheduler. Could also be split into different schedulers to avoid the mode argument. What's your opinion on this?