Support for annotating Optional inputs

[justinchuby]

We would like to make this work:

@torch_op("aten::clamp")
def aten_clamp(
    self: TReal, min_: Optional[TReal] = None, max_: Optional[TReal] = None
) -> TReal:
    # clamp(Tensor self, Scalar? min=None, Scalar? max=None) -> Tensor

    if op.OptionalHasElement(min_):
        min_ = op.OptionalGetElement(min_)
        min_clamp = op.CastLike(min_, self)
    else:
        # TODO: get the correct min number for the dtype
        min_clamp = op.Constant(value_float=float("-inf"))

    if op.OptionalHasElement(max_):
        max_ = op.OptionalGetElement(max_)
        max_clamp = op.CastLike(max_, self)
    else:
        max_clamp = op.Constant(value_float=float("inf"))

    # Enforce the lower and upper bounds
    clamped = op.Max(op.Min(self, max_clamp), min_clamp)
    return clamped

onnxruntime complains that

E onnxruntime.capi.onnxruntime_pybind11_state.InvalidGraph: [ONNXRuntimeError] : 10 : INVALID_GRAPH : This is an invalid model. In Node, ("", OptionalHasElement, "", -1) : () -> ("output0",) , Error Node () has input size 0 not in range [min=1, max=1].

Reference

• Optional input draft #76

[gramalingam]

Question: when do you get this error message? When the script is processed, or when you try to execute it with some input, etc.?

[justinchuby]

That's during eager execution

[gramalingam]

With which opset? The op OptionalHasElement was updated in opset 18 to make the argument optional. I believe the error is valid for earlier opsets, but if it happens with opset 18, there is an issue. Am trying to run it, but as you noted elsewhere, it looks like we need some other changes before we can use the latest ONNX and opset (I am running into the same issue with default_opset you noted elsewhere.)

[gramalingam]

Ok. with the recent changes. I am able to run onnxscript. But onnxruntime still doesn't support the latest opset. We may need to wait for that, hopefully not too long.

[justinchuby]

I see! That makes sense. I did notice a few ops not runnable by ort

[justinchuby]

So this function is fine?

[justinchuby]

Was #76 still relevant?

[gramalingam]

So this function is fine?

The usage of OptionalHasElement/OptionalGetElement looks fine to me. But we need to wait for onnxruntime to be able run it end-to-end to validate everything works.

As for PR 76, we are unlikely to use it.

But there are still some features to be implemented. For example, our type-annotation support doesn't yet handle Optional, etc. I am looking into this for now.

[gramalingam]

BTW: I see in the code-base that min_ / max_ are attributes rather than inputs. Which is it?

The two (optional attributes vs. optional inputs) are different in terms of how we can handle them in ONNX. Optional attributes requires some design decisions (may be easier to discuss this in a meeting).

[justinchuby]

They can be inputs if needed