Implement complex-valued constraint parameter

[j-peetz]

Hello @NEGU93, thank you for making this library! I'm attempting to use the arguments "use_bias" and "kernel_constraint" within "ComplexDense" as so:

self.d1 = complex_layers.ComplexDense(dim, use_bias=False, kernel_constraint=ConstantWeights())

However, these arguments seem to have no effect -- there are still biases, and the constraints on the weights are not enforced. That is, the layer performs identically to this:

self.d1 = complex_layers.ComplexDense(dim)

I tried the same arguments with tf.keras.layers.Dense and got the desired behaviors. Please let me know if I misunderstand how to use these. Thanks in advance!

Here's the relevant context, in case it helps:

class ConstantWeights(tf.keras.constraints.Constraint):
    def __call__(self, w):
        tf.keras.backend.set_value(w[0, 0], 3)
        tf.keras.backend.set_value(w[1, 2], 3)
        return w

class MyModel(Model):
    def __init__(self):
        super(MyModel, self).__init__()
        self.d1 = complex_layers.ComplexDense(dim, use_bias=False, kernel_constraint=ConstantWeights())

    def call(self, x):
        x = self.d1(x)
        return x

model = MyModel()

[NEGU93]

Hello and thank you for your interest.
It is possible that the kernel_contraint is not implemented but the bias parameter should be working...
Could you confirm me which version of the library are you using? Could you try using the last version and see if the issue is solved?

[j-peetz]

Thank you for the response. I updated from v1.0.0 (via conda) to v1.2.14 (via pip). Like you said, the bias feature is now working properly. However, the kernel constraint still seems to have no effect.

[NEGU93]

Yes sorry, I need to update conda's version. I will do so next week.

Indeed, the kernel_constraint is not yet implemented.
I may add this feature in the future. I will let you know when (if) I do.

[NEGU93]

I have been investigating about this. I could easily do:

• A real-valued kernel_contraint that applies to both real and imaginary parts (but separately)
• Two real-valued real_contraint and imag_contraint that applies to them separately.

This is a problem because it will prevent (I think) the idea of making a contraint on the phase or amplitude.
I understand that the ideal would be to be able to define a complex-valued contraint but that will require more work and I am not free.

Does this still work for you? If so, which feature do you think would be better suit for you?

[j-peetz]

I think having two separate real_constraint and imag_constraint options would be especially useful.

In my specific application, I just needed to set certain weights to 0. And I actually made some modifications that worked for me here: master...j-peetz:master

But I only changed the ComplexDense class. And in general, it'd be nice to be able to specify separate real and imaginary constraints.

Thank you so much for being responsive! Having complex NNs opens up a lot of possibilities, and this library is super helpful.

[NEGU93]

Congrats! I did a pull request of your code.
I leave this issue open as feature requests of:

• Add Bias contraint
• Add constrain to other layers (such as Conv)
• Add separate real_constraint and imag_constraint
• Best case scenario as complex_constraint. Related to issue 56217

[j-peetz]

Ok great! Thanks again.