Add client code generation from OpenAPI documents

[dougbu]

Is this a Bug or Feature request?:

Feature request

Steps to reproduce (preferably a link to a GitHub repo with a repro project):

See comments in aspnet/specs#159 and aspnet/specs#161 about client code generation for services described using OpenAPI. The most specific comments are in the API client generation (C# & TypeScript) section of aspnet/specs#161

Description of the problem:

With a focus on MSBuild integration, build one or more wrappers for tools that can generate client code for a service described using OpenAPI. This includes:

• OpenAPI document available at a URI
• OpenAPI document available in a file on disk
  • This may be treated as a derivative case of URIs. Could also require some special-casing. For example, though not required initially, we may eventually want to refresh clients in a running application when the OpenAPI document changes i.e. file watching / cache flushes similar to how we handle .cshtml file changes at runtime.
• Project-to-project references that indicate the "other" project can provide an OpenAPI document
  • Most general case i.e. something independent of how that document is produced may be a way to start a project, use a described URI to download the OpenAPI document, then generate a client for that document.
  • As mentioned in aspnet/specs#161, an "inside man" approach may also be useful when tighter integration with whatever generates the OpenAPI document may be helpful.

Code generation itself

To the extent possible, will outsource the exact OpenAPI to C# or TypeScript conversion. This interface is likely to be behind a small MSBuild Task that can be easily overridden or a set of properties that control the exact Exec task used to perform code generation.

Inspirations to investigate

• https://github.com/aspnet/scaffolding
  • Scaffolding provides a relatively simple approach for wrapping very different code generators into a common command-line UI. Though the lower (Razor-based) code generators are likely to be irrelevant, this high-level integration may provide an intuitive CLI for our users. Likely not needed until we're done providing MSBuild itegration.
• Add Service Reference
  • What artifacts remain in the project after a service reference is added? Can we reuse some of these artifacts for generator-to-project and generator-to-URI integration?
• <Compile> and <EmbeddedResource> links
  • Can we reuse some of these artifacts and their specific metadata for generator-to-project and generator-to-URI integration?
  • If we decide to do something completely different on one side or the other, what inspiration can we take from the <Compile> and <EmbeddedResource> linkage approach

[dougbu]

This is a tracking issue for work in the area of client code generation from OpenAPI documents. So far, the main lower-level work is covered in aspnet/Mvc#6788. Will add another issue shortly to describe the parallel investigations mentioned above.

[bugproof]

OpenAPI documents provide weak descriptions of types, examples of such types are enums and generic types. It would be nice if such generator supported generics and proper enum generation(especially when targeting C# & TypeScript). From my experience, it's much easier to generate client code using ApiExplorer instead of parsing OpenAPI (using something like OpenAPI.NET)

I've already experimented with client generation using OpenAPI.NET and Razor. The problem of loading OpenAPI documents is trivial, code generation itself is much more complex to solve because first, you have to parse the document into some type representations you can generate code from.

Existing client generators like autorest, swagger-codegen, NSwag exist but they don't solve problems mentioned above (enums and generics)

Summary of the problems:

• API response wrappers, like ApiResult<T> should have equal representation on the client-side
• Enums as integers should have correct names on the client-side, the only way is using OpenAPI extensions

It would be nice if such generator would be customizable to the point you can edit the template to generate client that will use some library like Refit or RestSharp.

One disadvantage(or not) of generating using ApiExplorer is that you can't generate remotely unless you provide some remote interface that downloads generated client(as a user of an API without access to the server source code).

Anyways this approach works really well and I have made usable code generator (based on ApiExplorer) that works much better than other public generators. I use RazorLight and generate code from razor templates :)

[jods4]

Sorry if this is not the right place to comment.
I saw a mention of "API client generation" in the 2.2 roadmap but there was no link.

This will be a great addition but could please base this on a lower-level templating that would be reusable?

Currently, we're using the Typewriter VS extension to generate TS code based on our own templates.
We generate our own service client but not only that. For example we also generate client-side validation from the server-side model validation.

So it would be great if you had tasks to generate client code based on a meta-model + template; and the Swagger or OpenAPI generators were simply providing the meta-model part and maybe a default template.
That way we could re-use those task to generate other stuff, assuming we provide a different meta-model and template (e.g. data model and validation).

Customisable templates is quite important. Our network calls are highly opiniated, there are additional headers, automatic handling of specific status codes...
In our service template we basically import and call a custom http function that wraps fetch with lots of app-specific code.

Neat thing about Typewriter is that it updates the client code live, as soon as you save the C# code. Hopefully live refresh will work the same with your MS build tasks.

[rynowak]

Hi @LeCoque @jods4 - thanks for the feedback. This is certainly an appropriate place for a discussion.

We're intentionally light on details about what the in-the-box code generator will do that this point because our plans aren't fixed yet.

This issue covers some of the things we do know - like:

• we want to have deep MSBuild integration to support Project-to-Project code generation
• we want to support code generation using arbitrary swagger/openapi documents
• we want to be able to plug in any code generation tool that can be driven from the command line - this can include dotnet global tools, things like autorest, or tools delivered via nuget

We'll announce more details once we have a plan to discuss

[dougbu]

aspnet/Mvc#6788 (the prototype) is done. Will soon add new issues covering additional work items related to this big rock.

[dougbu]

Coming soon

Will open additional issues covering:

• Ensure all non-generated targets have license headers - Ensure all client code generation source has appropriate license headers aspnet/Mvc#8415
• Remove CodeAnnotations and any related use of JetBrains features - Remove CodeAnnotations and related JetBrains use from client code generation source aspnet/Mvc#8416
• Rationalize the code sharing between the three code generation projects - Rationalize the code sharing between the three code generation projects aspnet/Mvc#8417
  • Sharing is mostly from the bottom up: GetDocumentInsider -> dotnet-getdocument -> Microsoft.Extensions.ApiDescription.Client but a couple of oddball cases exist and those links should be reversed
• Add batching in Microsoft.Extensions.ApiDescription.Client.targets and use it to simplify the DefaultDocumentGenerator target - Add batching in Microsoft.Extensions.ApiDescription.Client.targets aspnet/Mvc#8419
  • This will reduce the MSBuild requirements placed on code generation providers as well as 3rd party document generator tooling
• Flesh out the GetProjectReferenceMetadata task and use it to simplify the ServiceProjectReferenceGenerator target - Flesh out the GetProjectReferenceMetadata task #4912
• Extend the GetDocumentInsider code to avoid overwriting the API description when it's already up-to-date - GetDocumentInsider code should not overwrite up-to-date API descriptions #4913
  • Right now, the inside man only does this check when it downloads the document from a URI
• Add the actual Microsoft.Extensions.ApiDescription.Client package, containing the tasks, generic document generation tool, and MSBuild infrastructure for the Add client code generation from OpenAPI documents #4896 feature - Create single Microsoft.Extensions.ApiDescription.Client package aspnet/Mvc#8428
  • Currently these pieces include a separate global tool but that is not the chosen design
  • Also, package generation is currently disabled in both of the Microsoft.Extensions.ApiDescription.Client and dotnet-getdocument projects
• Track work we'll submit to the https://github.com/RSuter/NSwag repo - Implement fix for RSuter/NSwag#1587 aspnet/Mvc#8423, Implement fix for RSuter/NSwag#1588 aspnet/Mvc#8424 and Implement fix for RSuter/NSwag#1626 aspnet/Mvc#8425
  • Specific issues are Add package supporting proposed ASP.NET Core's MSBuild code generation features RicoSuter/NSwag#1587, Extend proposed ASP.NET Core's MSBuild code generation features using aspnetcore2swagger RicoSuter/NSwag#1588 and one liner to host the client generation as a service RicoSuter/NSwag#313 or a new issue specifically covering an IDocumentProvider service
• Add tests (!!) - Add tests of client code generation components #4914
• Announce and document the new end-user features - Describe new client code generation Swagger features AspNetCore.Docs#8462, Add documentation to support code generator MSBuild AspNetCore.Docs#8461, and Add documentation for API description providers in web sites AspNetCore.Docs#8465
  • /cc @glennc 😈

(Checkboxes above track creating the issues. Issues will track the actual work.)

[RicoSuter]

@necronux nswag already supports enums pretty well:
https://github.com/RSuter/NJsonSchema/wiki/Enums

Generics however are a different story: It could be implemented but only the nswag toolchain would understand them and other tools probably wont (not implemented yet):
RicoSuter/NJsonSchema#23

[dougbu]

Clearing old needs design label…

[mkArtakMSFT]

Moved to 3.0 as this is the uber tracking issue. The work is still happening in preview milestones.

[dougbu]

@mkArtakMSFT though I've reset the checkboxes above to reflect what work is complete, this issue overlaps the Client Code Generation project. Suggest we close this and just track the work in that project. Any objection?

[Eilon]

@dougbu / @mkArtakMSFT - is this a PRI-0 bug, and is it in the correct milestone?

[dougbu]

This is a tracking bug for the overall service reference feature. I suggest we close it because we now have a project that makes it more obvious what's going on: https://github.com/orgs/aspnet/projects/21

@mkArtakMSFT any objections to closing this?

[StevenRasmussen]

Hmmm... I get a 404 when trying to go to the new project. Should I be able to see it? Or is it internal or something?

[dougbu]

@StevenRasmussen the project is visible only to those in the organization. What information were you looking for?

[StevenRasmussen]

@dougbu, nothing in particular. I was just tracking this issue and now that it’s closed due to this other project I was wanting to see if there was a better issue to track in that project. Ultimately I’m just trying to track any progress regarding client generation with .Net Core (which I thought this issue was about). Is there a better issue to be tracking? Thanks

[dougbu]

Is there a better issue to be tracking?

I suggest following the issue or issues of most interest to you. For example, #7639 may be worth following if you're looking for tooling features or #8242 if you want to generate Open API documents in your ASP.NET Core MVC projects.