spec-first/connection does not respond correctly if the openapi are added in a certain order

[damienklotz77]

Hi,

Let's consider the 3 following files :

openapi1.yaml

openapi: 3.0.4
info:
  title: My title
  version: '1.0'
servers:
  - url: http://127.0.0.1:8000
paths:
  /endpoint1:
    get:
      operationId: operationId1
      responses:
        '200':
          description: My description

openapi2.yaml

openapi: 3.0.4
info:
  title: My title
  version: '1.0'
servers:
  - url: http://127.0.0.1:8000/path2
paths:
  /endpoint2:
    get:
      operationId: operationId2
      responses:
        '200':
          description: My description

script.py

from connexion import AsyncApp
from connexion.resolver import Resolver


class MockResolver(Resolver):
    def __init__(self, mock):
        super().__init__()
        self.mock = mock
        self.operations = {}

    def resolve_operation_id(self, operation):
        self.operations[operation.operation_id] = operation
        return operation.operation_id

    def resolve_function_from_operation_id(self, operation_id):
        def _call_mock(*args, **kwargs):
            return self.mock[operation_id](*args, operation_id=operation_id, **kwargs)

        return _call_mock


def my_function1(**kwargs):
    return "OK1"


def my_function2(**kwargs):
    return "OK2"


mock = {
    "operationId1": my_function1,
    "operationId2": my_function2
}
resolver = MockResolver(mock)
app = AsyncApp(__name__)
app.add_api(specification="openapi1.yaml", resolver=resolver, strict_validation=True)
app.add_api(specification="openapi2.yaml", resolver=resolver, strict_validation=True)
app.run()

When calling endpoints, its is working properly

curl http://127.0.0.1:8000/endpoint1
"OK1"
curl http://127.0.0.1:8000/path2/endpoint2
"OK2"

Now, in the python script, we swap the order in which the openapis are added.

From

app.add_api(specification="openapi1.yaml", resolver=resolver, strict_validation=True)
app.add_api(specification="openapi2.yaml", resolver=resolver, strict_validation=True)

To

app.add_api(specification="openapi2.yaml", resolver=resolver, strict_validation=True)
app.add_api(specification="openapi1.yaml", resolver=resolver, strict_validation=True)

Then, we call the endpoints the same way than previously.

Expected behavior

curl http://127.0.0.1:8000/endpoint1
"OK1"
curl http://127.0.0.1:8000/path2/endpoint2
"OK2"

Obtained behavior

curl http://127.0.0.1:8000/endpoint1
"OK1"
curl http://127.0.0.1:8000/path2/endpoint2
{"type": "about:blank", "title": "Not Found", "detail": "Not Found", "status": 404}

Why does the order in which the two openapis are added affect the behavior of spec-first/connection when we calls the 2 endpoints ?

Thank you :-)

[chrisinmtown]

I don't have an answer. I see that the server URLs are different in your two OpenAPI spec files. Would you please try using URL "http://127.0.0.1:8000/path1" in your openapi1.yaml file and see if the results change? I am wondering if there's some matching or folding going on here; in this case, the first URL is a proper prefix of the second URL.

Also, forgive if I'm missing something really obvious; is the problem only visible when a custom resolver is used? What happens if you use operation IDs in the spec that are just the typical module_name.function_name style?

[damienklotz77]

Thank you for you quick answer :-)

I don't have an answer. I see that the server URLs are different in your two OpenAPI spec files. Would you please try using URL "http://127.0.0.1:8000/path1" in your openapi1.yaml file and see if the results change? I am wondering if there's some matching or folding going on here; in this case, the first URL is a proper prefix of the second URL.

Using "http://127.0.0.1:8000/path1" in openapi1.yaml, it is working but I cannot change the openapi (given by another team).

Also, forgive if I'm missing something really obvious; is the problem only visible when a custom resolver is used? What happens if you use operation IDs in the spec that are just the typical module_name.function_name style?

With typical module_name.function_name style like this :

openapi: 3.0.4
info:
  title: My title
  version: '1.0'
servers:
  - url: http://127.0.0.1:8000
paths:
  /endpoint1:
    get:
      operationId: script.myfunction1
      responses:
        '200':
          description: My description

openapi: 3.0.4
info:
  title: My title
  version: '1.0'
servers:
  - url: http://127.0.0.1:8000/path2
paths:
  /endpoint2:
    get:
      operationId: script.myfunction2
      responses:
        '200':
          description: My description

from connexion import AsyncApp


def myfunction1():
    return "OK1"


def myfunction2():
    return "OK2"


if __name__ == "__main__":
    app = AsyncApp(__name__)
    app.add_api(specification="openapi2.yaml", strict_validation=True)
    app.add_api(specification="openapi1.yaml", strict_validation=True)
    app.run()

It produces the same problem so the issue is not coming from the resolver.

[chrisinmtown]

Thanks for simplifying the example. Sorry I wasn't clear, I'm not asking you to change what you do in real life! I was only asking if you can still demonstrate the problem if the two specs have URLs that are a bit more different. Just trying to pinpoint the conditions that trigger the failure.

[damienklotz77]

Ok ;-)

1/ Using "http://127.0.0.1:8000/path1" in openapi1.yaml, it is working in both examples (with resolver and simplified).
I have done the test, sorry I was not clear too ;-)

2/

I am wondering if there's some matching or folding going on here; in this case, the first URL is a proper prefix of the second URL.

What do you mean saying that ?

3/ To summarize, I have the impression that declaring the openapi without a path after declaring the one with path, overwrites the one with a path declared previously.

[chrisinmtown]

I was making a SWAG, a stupid wild-a$$ guess, about how Connexion's add_api call works. My wild guess was basically that add_api might perform special handling if the new base URL matches the base URL for an API already registered. However, I dug into the code for add_api and I see that it just creates an object and adds it to a list of registered API objects, without any use of the base URL. So the problem of not finding the path/endpoint/whatever is probably in the feature that tries to find a match for the request path against a base path in the list of registered API objects.

The list of API objects is ordered (of course). I am starting to think that what happens when the matcher receives the request path "http://127.0.0.1:8000/path2" is that it matches the base URL "http://127.0.0.1:8000" in the first API object nicely, so Connexion latches on to that API object. Then it searches but fails to find the registered path path2 within that first API object, and it gives up. If you register APIs in the other order, then the search will find a match; but probably something else fails.

If I'm right, that means the current implementation requires the base URLs of all registered APIs to be different, and not share a prefix. Perhaps the search could be extended to check all registered APIs, instead of stopping at the first failure?

Please tell me if I'm making any sense here.

[damienklotz77]

Thank you chrisinmtown for the detailled explanation.

To summarize, you are saying that spec-first/connexion currently does not support the combinaison of the two following statements :

• the basepath of one openapi includes the basepath of another openapi
• the openapi with basepath included is declared in second position

?

[chrisinmtown]

I think you have accurately described the current implementation. Now that you and I understand the situation a bit better, you might want to reword this issue, or open a new issue, to explain the current behavior and your desired change. Maybe state it as an enhancement request, something like "Search all registered APIs when matching a request path". Please consider.

[damienklotz77]

Hi chrisinmtown,

I have just create the enhancement request :
#2108
(I cannot add "enhancement" label)

Regarding the pre-analysis, I was wondering : if the order in which APIs are added matters, does that mean the API list is unordered ?

[chrisinmtown]

@damienklotz77 you can inspect the code for add_api yourself :) you will see that method modifies a plain old Python list. I expect the list contents reflect the order in which the APIs were added. The order only matters today because of how the search algorithm works, when it tries to find a matching endpoint when a request arrives. If the enhancement that you requested is made, then I think the order will not matter anymore.