Is OpenAPI an Open World or a Closed World Contract?

[darrelmiller]

Forgive me for the nebulous issue, but as we head towards a v.Next document, I believe that understanding the scope of what OpenAPI is trying to define will help in making the many decisions that need to be made to get to v.Next.

The terms Open World and Closed World seem to have reasonable definitions on Wikipedia. Let me try and apply the concepts to Open API.

Consider any arbitrary Open API document that describes an API. It is assumed that the document accurately describes the API. We sometimes use the word contract to define the role that the Open API document plays between the client and the server.

Is the OpenAPI document still valid if

• there exists a resource in the API that is not described?
• a resource accepts a parameter that is not described?
• a resource supports a representation format that is not described?
• a resource returns a status code that is not described?

In a closed world, the API Description describes exactly what is supported, if it is not described, then it must not exist. In an open world we cannot make assumptions about what is not described. There may be other resources, other supported formats, other parameters.

I get the feeling from the conversations I have had with people who use OpenAPI that many perceive it to be a closed world description. Certainly I believe that many people would say that returning an unspecified status code would be a contract violation.

Which brings us to the challenge. Closed world systems work well when the entire system is under the control of the creator. However, in the world of network based HTTP systems, many parts of the system are not under the control of the API designer. It is quite common for intermediary components to return status codes and media types that the API designer may have no knowledge of. It is quite common for APIs to change in ways that are additive and non-breaking and should not be considered a contract violation.

For this reason I do not believe that OpenAPI can be a truly closed world specification. My fear is that we continue to pretend that it is a closed world specification and client developers continue building clients based on a false assumption that the only thing that is true is what is described in the specification document. I am convinced this is a common cause of buggy client applications.

Another less than ideal outcome would be if we try and hold some middle ground where some parts of the specification must be exhaustively described, whereas other parts only describe some subset of what is possible. This is going to further challenge us when it comes to identifying what should be considered a breaking change and therefore require a version change.

I would like to see wording in the Open API specification that states that what is being described is what can happen, but the behavior of the API is not limited by the description document. It would be reasonable to state that behaviour that is described will not be removed without a corresponding version change. Obviously no such guarantee is made about the availability of undescribed behavior!

My guess is that many people reading this will not object to undocumented resources, undocumented parameters and undocumented response media types. However, I expect that many will feel that it is unreasonable to allow undocumented response status codes. How can a client be expected to consume an API if the API can return status codes that are not documented?

Well, first let us recall that there are an unknown number of intermediaries that may sit between your client and your API that may return statuses like 407, 502, 504, 511. The reality is that it already happens.

Second, and more importantly, let me quote RFC 7231,

HTTP clients are not required to
understand the meaning of all registered status codes, though such
understanding is obviously desirable. However, a client MUST
understand the class of any status code, as indicated by the first
digit, and treat an unrecognized status code as being equivalent to
the x00 status code of that class

The HTTP specification says that ALL clients of an HTTP API must be designed to deal with all status codes. If a client gets a 413 and it doesn't have any special handling code for that status code, then it can treat it as if it were a 400. This means that clients need to handle 200,300,400, and 500 at a minimum to be compliant with the HTTP specification.

Once you can accept this as a requirement of building a HTTP client, then the ability for an API request to return a unexpected status code gets a whole lot less concerning.

If the OpenAPI specification can be accepted as an Open World specification, then we gain a significant amount of flexibility. If we want to implement rate limiting in an API, we are no longer required to add a 429 description to every response object. And we can have a certain amount of confidence that existing client applications will not crash because if they have followed the rules, then they already know how to treat the 429 as if it were a 400.

Being open world doesn't prevent someone choosing to explicitly describe every operation in detail if they choose to. However, it opens the possibility for those of us that wish to build a more self-descriptive API, can limit what is being explicitly described. We may even be able to lift the requirement that a response object must be present with at least one response.

Taking this path makes it easier to take the decision that adding a capability to an OpenAPI description should never be considered a breaking change.

I'm sure there many other impacts of committing to this approach. Perhaps some of them may be negative. I look forward to hearing all the opinions.

[earth2marsh]

Historically IIRC, Swagger would have considered it to be perfectly acceptable to change the representation of a spec based on the authenticated user. In that case, an admin might see more than a regular consumer. In this case, at least from the less privileged user’s perspective, that seems like an Open World.

IMO, an OpenAPI spec always describes a possible truth, and it may be wrong or incomplete. If the spec is used in a contract-driven service, then the description can be considered be a definition. A definition does seem to edge closer to a closed world, but even then the tooling driven by that spec may make some assumptions that aren’t explicit in the spec, such as handling OPTIONS requests so they don’t clutter up the spec itself. I also find it a bit tedious to document all possible response codes—I'd much rather assume the implementation follows HTTP standards and only documents exceptions, for example.

I think I’m arguing that OpenAPI allows for an open world perspective, though it doesn't proscribe it. For example, tooling could choose to treat a spec strictly in a more closed approach.

Will need to think through the implications and your points further, but this is a very interesting question to me.

[fehguy]

Hi guys, good thoughts. It is important to put forth the goal of a project or we would never really hit it!

Historically, Swagger was intended to be a contract between client and API. That has many benefits, including the ability for a client to understand what an API is capable of, and that parameters, payloads, etc. are well understood. How can one know that a query parameter exists if it's not documented... somehow?

That goal--being a contract vs. a modeling or documentation format--is what I believe has helped Swagger's adoption through the ability to have practical and useful tooling. Many of us don't mind using wireshark to decipher what a server can understand. However, the general population won't find that acceptable!

Back to the question though, because I don't feel the answer is strictly black or white. I do believe that the (now) OAS needs to continue to provide the contract for end users to consume. That contract can and should become more flexible than it has been historically.

How can this happen in a practical sense? I can imagine a couple approaches.

• Explicit flexibility. What does that mean? Perhaps arbitrary query params? Input or response payloads which are "undefined" (i.e. type: object)?
• Clarity around undocumented operations. Sure, add them! But don't expect anyone to use them. That's certainly not a philosophical issue historically, nor should it be moving forward.
• Response codes. It was never really the intent in Swagger that every single response code be documented--only those with payloads and messages which would be deemed extraordinary or especially useful for the client to understand. What's the payload definition in a 401 message? It is often very valuable to know. But do we need to say "server might return a 401?" My guess is that it doesn't add much value

These are just some thoughts, and only to a small number of points. But I do think flexibility is important while we consider the different proposed changes. If the goal is a contract, there is some specificity required. One wouldn't want to enter into an agreement to be paid without knowing the denomination of payment, right?

[tomchristie]

@darrelmiller - Very well made point, yes!

Well, first let us recall that there are an unknown number of intermediaries that may sit between your client and your API that may return statuses like 407, 502, 504, 511. The reality is that it already happens.

Indeed. Moreover, intermediaries may return differing content-types. Intermediary error cases might reasonably be expected to return any of text/html, text/plain or application/json. (Even without intermediaries this can be true - a server error will often result in a simple HTML error page, whereas application errors might be expected in JSON)

[ePaul]

Interesting discussion.

I guess in the spirit of both evolvability + compatibility neither the closed nor a fully open world is the right approach.

Consider the case where a client works with a swagger description of last year's "state of the API", but the server during that time got changed.

• It doesn't hurt any existing client if there are additional resource types (paths) or operations on those resources available, which were not defined in the API. (It might hurt if those paths are referenced by URIs contained in responses or by redirects, but the HATEOAS support is another topic).
• It also doesn't hurt any existing client if an operation accepts additional non-required parameters, as long as the operation when using the old parameters does "the same thing".
• It also doesn't hurt if the server accepts additional content types, or offers additional content type as results, as long as it still accepts the old input, and still offers the old output format (assuming the client uses a suitable Accept header).
• A maybe a bit dubious case is the addition of fields in the input or output JSON objects. For inputs, as long as the old input (without these fields) is still accepted and works the same, it is okay. For outputs, at Zalando, we decided that the client has to ignore any additional fields in objects returned by the server (see our Restful API guidelines, section compatibility).
  I think this is consistent with the JSON schema meaning of properties.

For "additional status codes", the others said already enough – I guess the general guideline from HTTP "handle unknown xyz as x00" (with potentially different body payload) is quite fine.

Examples for changes/additions which are not okay:

• new required parameters
• new required fields in input parameters
• new enum values in return objects (we introduced an x-extensible-enum which allows that)
• redirect to an URL which does not comply with the same API

[inadarei]

Thank you, Darrel, for raising this issue.

This one is crucial. The approach Darrel is talking about is also known as Postel's Law or Robustness Principle, has its genesis in TCP/IP spec of 1981 and has quite literally enabled the whole web thing. Distributed architectures need Open World approach, it's not really a choice but a requirement.

The principle is echoed in W3C's Architecture of the WWW from 2004, dubbed MUST IGNORE: https://www.w3.org/TR/2004/WD-webarch-20040816/

It is also known as Forward Compatibility in MSDN's architecture documents: https://msdn.microsoft.com/en-us/library/ms954726.aspx

Simply put: stringent "contracts" lead to brittle systems and brittle systems don't survive long on the open web. If Swagger wants to grow up to become Open API Specification, it has to be compatible with open web and Open World approach. That isn't much of a choice, what is a choice is - identifying this need early and evaluating everything that will be put in the spec, through that lense, which I believe is exactly what Darrel is proposing and I think it is very important.

[fehguy]

@inadarei conceptually that's fine, but let's put things in more concrete terms. I do think that having a philosophy that "the server may do x, y, z" is an open definition but not very useful to anyone.

To make this more clear, and to avoid getting into an overly philosophical discussion where we all agree but in different words, I suggest we tackle a list of items which need clarification--response codes is one of them. We can discuss how we define and approach them, how things need to be more open and then see how that fits into the spirit of this effort?

[inadarei]

@fehguy iterative and concrete sounds great. No need for philosophical vaporware, indeed.

@darrelmiller correct me if I am wrong, but since you already started making the general principle much more specific to the API space, I would like to take a shot at further specifying what Postel's may mean for Open API Spec. How about:

"Documentation is not a contract, it's a guide. There is no 1-1 relationship between what is in the API and what is documented. Specifically:

1. There may be things that are not in the documentation that API affords because they may be documented somewhere else (e.g. HTTP Codes in HTTP RFCs)
2. There may be things that are not in the documentation because your client may be of ver v1.1 using v1.1 documentation and it may come across API v 1.2 (Forward Compatibility)"

In general, once you start distilling these, a lot of rules come to three principles of evolvable systems:

1. Do not take anything that system already supports away (you will break existing clients)
2. Do not change the meaning of things already supported (you will confuse existing clients)
3. Anything new added should be optional (existing clients don't know about them, yet)

APIs need to be evolvable systems, because they operate in a distributed environment. Documentation that is created for APIs, therefore should respect this fact and accommodate it.

The big step is: stopping viewing documentation as a contract and single source of truth. WS-* standards made that mistake and created problems. If that main principle can be agreed upon, everything else is: details that should be figured-out specifically as the standard evolves and matures.

Makes sense?

[earth2marsh]

I'm not sure I agree, @inadarei. An OpenAPI spec can be a description (possibly true) or a definition (must be true). Either way it is a formal description of the available services and at least aspires to represent a contract.

If the spec drives the implementation (like the swagger-inflector, swagger-node, and connexion projects) then isn't it a true contract? If the documentation is then generated from that spec, it would be an HTML representation of that contract (NB: it could be a subset or have description inaccuracies). Similarly, if the spec is generated from code annotations. It may not be an exhaustive contract, such as your example of HTTP Codes, but it can still be a contract that is about as close to truth as you can get.

Once you generate clients from a spec, isn't that essentially leveraging the spec as a contract? Maybe your point was that the spec is the contract but the artifacts derived from specs are somewhat suspect?

[inadarei]

@earth2marsh,

centralized systems don't scale. By making OpenAPI spec central point of "all truth" it will be doomed for inflexibility and eventual failure. There are enough historical examples of this to at least be worried.

The examples you listed are very helpful. Discussing how they can be implemented in the Open World vs Closed World is a healthy discussion, I think. Assuming that turning spec into a stringent contract is the only way, on the other hand - feels like starting with a pre-baked solution in search for the problems it may satisfy.

I don't think that strict, central contract that needs to explicitly enumerate all known "facts" and "truths" about an API is the only way to solve the needs you have listed.

Being too demanding on conformity, for the declared benefit of clients, is exactly what killed XHTML (the strictly XML-conforming HTML), was it not? Eventually they had to completely abandon it and move onto HTML5.

Just my 2c.

[tomchristie]

I think this is just a case of a little more subtly in the language used. The schema definition necessary cannot always be a complete description of the API, because:

• We cannot know what intermediaries may exist between the client and server, and what status codes, content types, or content they response with.
• We cannot know that if a client is operating against an older (eg incomplete but still version-compatible) copy of the schema.

I don't think there's any fundamental problem here, just that the language ought to reflect the open nature of the possible responses that might result from any given request, and the "correct but not necessarily strictly complete" nature of the available endpoints and parameters.

This means that clients need to handle 200,300,400, and 500 at a minimum to be compliant with the HTTP specification.

Indeed. Having a somewhat related discussion on why response status codes, content types and content schemas don't actually need to be known ahead of time in order to implement a dynamic client library. Available over here.

[fehguy]

I agree with @tomchristie. As I said, we are making a contract. That doesn't mean it's the only thing a server can perform. As an example, I may run a car wash service. If you agree to have your car washed and I instead rotate the tires, I have broken the contract. We don't want that. However, I may have additional capabilities which may not be advertised in the contract (like tipping). That's OK.

So yes to contract. We are not simply documenting the API. A contract can be used to document an API but not necessarily vice-versa but I fear those terms have different meanings to different people, and arguing over "it's a contract" or "it's documentation" is probably not worth it.

We should, as has been suggested, be very specific about the language to explain why one documents response codes. It should not in my opinion reflect the intent that "all response codes which can be emitted from the server are documented". It should explain the default response payload at a minimum (assuming it is intended to be consumed) and any payloads on other response codes, or descriptions of why you would receive them.

Speaking of response codes, I do believe that one can say "there's no response payload" for an operation but it sends something--conceptually that means "the client shouldn't care about the response payload, period" if not documented. That's not a failure of a contract IMHO.

[inadarei]

Thank you, Tony. This was the main thing for me:

I may have additional capabilities which may not be advertised in the contract (like tipping). That's OK.

As long as that is the case and "contract" is not meant in a strict sense of the likes of XML Schema (that do demand: only things we said should be present) – everything is great.

Thank you!

[fehguy]

Awesome, I'm glad it's just wording!

[darrelmiller]

@fehguy I'm actually really glad I brought this up now. I was concerned it might rat-hole and not lead to more clarity. However, this quote made me very happy.

Response codes. It was never really the intent in Swagger that every single response code be documented--only those with payloads and messages which would be deemed extraordinary or especially useful for the client to understand

Knowing this is very useful as I know this is not understood by many. I wonder whether in addition to the spec whether a design guidelines document would be valuable for clarifying these kind of perspectives.

@inadarei I have no issue with OpenAPI being defined as a contract, and I have no issue with OpenAPI being strict about things that are explicitly stated. I have concerns about the application of Postel's law that leads to the implementation of code that says "well I think the user meant to say x, so we will accept the invalid input by making assumptions based on common mistakes".

I am satisfied with the responses to my concerns and I think there is a general consensus on the goals even though we may use some of the words differently.

[fehguy]

I'd really love help explaining this in the documentation. If it's confusing then it's not good enough! Thank you for bringing this up and helping make the intent + documentation more clear.