Conv with auto_pad="SAME_UPPER" does not work as expected.

[lynic]

Describe the bug
When set strides > 1 with auto_pad="SAME_UPPER", the output shape is not the same as input.
if x.shape is {1,1,7,5}, strides={2,2}, then the output should be {1,1,7,5}, but I got {1,1,4,3}

(shapes (1, 1, 7, 5), (1, 1, 4, 3) mismatch)

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu16.04
• ONNX Runtime installed from (source or binary): pip install onnxruntime
• ONNX Runtime version: 0.2.1
• Python version: 3.6
• GCC/Compiler version (if compiling from source):
• CUDA/cuDNN version:
• GPU model and memory:

To Reproduce
Describe steps/code to reproduce the behavior:

import onnx
# import onnxruntime as onnxrt
import onnxruntime.backend as backend 
# from onnx_tf import backend

import numpy as np

def _extract_value_info(arr, name):  # type: (np.ndarray, Text) -> onnx.ValueInfoProto
    return onnx.helper.make_tensor_value_info(
        name=name,
        elem_type=onnx.mapping.NP_TYPE_TO_TENSOR_TYPE[arr.dtype],
        shape=arr.shape)


def expect(node,  # type: onnx.NodeProto
           inputs,  # type: Sequence[np.ndarray]
           outputs,  # type: Sequence[np.ndarray]
           name,  # type: Text
           **kwargs  # type: Any
           ):  # type: (...) -> None
    present_inputs = [x for x in node.input if (x != '')]
    present_outputs = [x for x in node.output if (x != '')]
    inputs_vi = [_extract_value_info(arr, arr_name)
                 for arr, arr_name in zip(inputs, present_inputs)]
    outputs_vi = [_extract_value_info(arr, arr_name)
                  for arr, arr_name in zip(outputs, present_outputs)]
    graph = onnx.helper.make_graph(
        nodes=[node],
        name=name,
        inputs=inputs_vi,
        outputs=outputs_vi)
    kwargs[str('producer_name')] = 'backend-test'
    model_def = onnx.helper.make_model(graph, **kwargs)
    onnx.checker.check_model(model_def)
    pm = backend.prepare(model_def)
    outs = list(pm.run(inputs))
    for ref_o, o in zip(outputs, outs):
        np.testing.assert_almost_equal(ref_o, o)

def export_conv_with_strides():  # type: () -> None

        x = np.array([[[[0., 1., 2., 3., 4.],  # (1, 1, 7, 5) input tensor
                        [5., 6., 7., 8., 9.],
                        [10., 11., 12., 13., 14.],
                        [15., 16., 17., 18., 19.],
                        [20., 21., 22., 23., 24.],
                        [25., 26., 27., 28., 29.],
                        [30., 31., 32., 33., 34.]]]]).astype(np.float32)
        W = np.array([[[[1., 1., 1.],  # (1, 1, 3, 3) tensor for convolution weights
                        [1., 1., 1.],
                        [1., 1., 1.]]]]).astype(np.float32)

        # Convolution with strides=2 and padding only along one dimension (the H dimension in NxCxHxW tensor)
        node_with_asymmetric_padding = onnx.helper.make_node(
            'Conv',
            inputs=['x', 'W'],
            outputs=['y'],
            kernel_shape=[3, 3],
            # pads=[1, 0, 1, 0],
            auto_pad="SAME_UPPER",
            strides=[2, 2],  # Default values for other attributes: dilations=[1, 1], groups=1
        )
        y_with_asymmetric_padding = np.array([[[[0., 1., 2., 3., 4.],  # (1, 1, 7, 5) input tensor
                        [5., 6., 7., 8., 9.],
                        [10., 11., 12., 13., 14.],
                        [15., 16., 17., 18., 19.],
                        [20., 21., 22., 23., 24.],
                        [25., 26., 27., 28., 29.],
                        [30., 31., 32., 33., 34.]]]]).astype(np.float32)
        expect(node_with_asymmetric_padding, inputs=[x, W], outputs=[y_with_asymmetric_padding],
               name='test_conv_with_strides_and_asymmetric_padding')


if __name__ == "__main__":
    export_conv_with_strides()

Expected behavior
A clear and concise description of what you expected to happen.
output shape should be {1,1,7,5}

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

[hariharans29]

Hi @lynic - is the output y_with_asymmetric_padding correct ? It seems like it's a copy of x which I would imagine isn't correct given all 1's kernel ?

[hariharans29]

Hi @lynic

It turns out the output shape generated by the runtime is quite valid according to the ONNX spec. For a precedent, please check out the test averagepool_2d_precomputed_same_upper in ONNX. It has auto_pad = "SAME_UPPER" and strides > 1 (non-default) and the expected output dimension value contains a value lower than the corresponding input dimension value.

The rationale is like this -

1. When an autopad value of "SAME_UPPER" or "SAME_LOWER", pads are first computed so that the entire kernel can be accommodated even when operating on a border pixel ("SAME_UPPER" and "SAME_LOWER" only differ in minor details dictating the distribution of pads to the front and end of the dimension)

2. The output shape is then computed as this: floor(((in_size - kernel + total_padding) / stride)) +1. For the special case of stride == 1, the output shape is the same as the input shape, not otherwise.

References: https://stackoverflow.com/questions/37674306/what-is-the-difference-between-same-and-valid-padding-in-tf-nn-max-pool-of-t (I particluary liked the third answer :))

Please re-open this issue if further clarification is required.

Also, please follow the answers here - onnx/onnx#1921. I raised this question after seeing the test mentioned above, but I later did some research to find that the test is quite valid, but it might be worthwhile to follow the explanation given by the Deep Learning experts.

cc: @pranavsharma