Request timeouts middleware

[Tratcher]

Background and Motivation

A common customer request is to be able to apply timeouts to their requests. AspNetCore servers don't do this by default since request times vary widely by scenario and we don't have good ways to predict that. E.g. WebSockets, static files, expensive APIs, etc..

• RequestTimeout in ASP.NET Core 3.1 is not availablewhen using InProcess Hosting #23160 - Out-of-proc had request timeouts.
• Kestrel execution timeout support #10079
• [Needs Tests] Add timeouts aspnet/KestrelHttpServer#485 (comment)
• How to specify / modify request timeout aspnet/KestrelHttpServer#611

To provide more control we can provide timeouts via middleware that are configured per-endpoint, as well as a global timeout if desired. These timeouts would link to the RequestAborted CancellationToken and be entirely cooperative. E.g. we won't call Abort when the timeout fires. It's up to the application logic to consume RequestAborted and decide how to handle the cancellation.

Proposed API

Project/Assembly: Microsoft.AspNetCore.Http

namespace Microsoft.Extensions.DependencyInjection;

+ public static class RequestTimeoutsIServiceCollectionExtensions
+ {
+     public static IServiceCollection AddRequestTimeouts(this IServiceCollection services) { }
+     public static IServiceCollection AddRequestTimeouts(this IServiceCollection services, Action<RequestTimeoutOptions> configure) { }
+     // We need to consider how these policies would integrate with IConfiguration, but that may be substantially different from this API.
+     // public static IServiceCollection AddRequestTimeouts(this IServiceCollection services, IConfigurationSection section) { }
+ }

namespace Microsoft.AspNetCore.Builder;

+ public static class RequestTimeoutsIApplicationBuilderExtensions
+ {
+     public static IApplicationBuilder UseRequestTimeouts(this IApplicationBuilder builder) { }
+ }

+ public static class RequestTimeoutsIEndpointConventionBuilderExtensions
+ {
+     public static IEndpointConventionBuilder WithRequestTimeout(this IEndpointConventionBuilder builder, TimeSpan timeout)  { }
+     public static IEndpointConventionBuilder WithRequestTimeout(this IEndpointConventionBuilder builder, string policyName) { }
+     public static IEndpointConventionBuilder WithRequestTimeout(this IEndpointConventionBuilder builder, RequestTimeoutPolicy policy) { }
+     public static IEndpointConventionBuilder DisableRequestTimeout(this IEndpointConventionBuilder builder) { }
+ }

+ namespace Microsoft.AspNetCore.Http.Timeouts;

+ public class RequestTimeoutOptions
+ {
+     // Applied to any request without a policy set. No value by default.
+     public TimeSpan? DefaultTimeout { get; set; }
+     public RequestTimeoutOptions AddPolicy(string policyName, TimeSpan timeout) { }
+     public RequestTimeoutOptions AddPolicy(string policyName, RequestTimeoutPolicy policy) { }
+     public bool TryGetPolicy(string policyName, out RequestTimeoutPolicy policy) { }
+     public void RemovePolicy(string policyName) { }
+ }

+ public class RequestTimeoutPolicy
+ {
+     public TimeSpan? Timeout { get; }
+     public RequestTimeoutPolicy(TimeSpan? timeout) { }
+ }

+ // Overrides the global timeout, if any.
+ [AttributeUsage(AttributeTargets.Method | AttributeTargets.Class)]
+ public sealed class RequestTimeoutAttribute : Attribute
+ {
+    public TimeSpan? Timeout { get; }
+    public string? PolicyName { get; }
+    public RequestTimeoutAttribute(int seconds) { }
+    public RequestTimeoutAttribute(string policyName) { }
+ }

+ // Disables all timeouts, even the global one.
+ [AttributeUsage(AttributeTargets.Method | AttributeTargets.Class)]
+ public sealed class DisableRequestTimeoutAttribute : Attribute
+ {
+    public DisableRequestTimeoutAttribute() { }
+ }

Usage Examples

services.AddRequestTimeout(options =>
{
    options.DefaultTimeout = TimeSpan.FromSeconds(10);
    options.DefaultTimeoutLocator = context => TimeSpan.Parse(context.Request.Query["delay"]);
    options.AddPolicy("MyTimeoutPolicy", TimeSpan.FromSeconds(1));
    options.AddPolicy("DynamicPolicy", new RequestTimeoutPolicy(context => TimeSpan.Parse(context.Request.Query["timeout"])));
});
...
app.UseRequestTimeouts();
...
app.UseEndpoints(endpoints => {
    endpoints
        .MapGet("/", (context) => { ... })
        .WithRequestTimeout(TimeSpan.FromSeconds(1));
app.UseEndpoints(endpoints => {
    endpoints
        .MapGet("/", (context) => { ... })
        .WithRequestTimeout("MyTimeoutPolicy");
...
    [RequestTimeout(seconds: 1)]
    public ActionResult<TValue> GetHello()
    {
        return "Hello";
    }
    [RequestTimeout("MyTimeoutPolicy")]
    public ActionResult<TValue> GetHello()
    {
        return "Hello";
    }
    [DisableRequestTimeout]
    public ActionResult<TValue> ImVerySlow()
    {
        Thread.Sleep(TimeSpan.FromHours(1);
        return "Hello";
    }

Alternative Designs

Risks

• We need to ensure this system is flexible enough to cover a wide variety of scenarios, while being easy enough for people to not mis-configure.
• What about components that don't use endpoints?
• What if the middleware is placed in the wrong location?

Thanks for contacting us.

We're moving this issue to the .NET 8 Planning milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s).
If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues.
To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[davidfowl]

1. What is the purpose of DefaultTimeoutLocator?
2. There should be an overload of WithRequestTimeout that directly takes a policy instance.
3. AddRequestTimeouts that takes the IConfigurationSection should be removed.

It feels a bit odd to have a mix of endpoint specific metadata that further filters the request to discover the timeout. The point of metadata is to bake in the decision about which endpoints have which metadata at startup so cheap decision can be made per request.

[Tratcher]

1. What is the purpose of DefaultTimeoutLocator?

This supports determining the timeout for any non-endpoint-aware resources like static files. It's optional for now, but I anticipate customers asking for it.

2. There should be an overload of WithRequestTimeout that directly takes a policy instance.

So that policy instance would get stored in the endpoint metadata and then checked for along with the attributes?

3. AddRequestTimeouts that takes the IConfigurationSection should be removed.

@sebastienros is working on an effort to make more of our components config aware, reloadable, etc. since this is a common requirement from real apps. They don't want to recompile to change timeouts. The API shape for that experience is negotiable. I don't think this can be removed without a proposed alternative.

It feels a bit odd to have a mix of endpoint specific metadata that further filters the request to discover the timeout. The point of metadata is to bake in the decision about which endpoints have which metadata at startup so cheap decision can be made per request.

I don't follow. Where is there further filtering? Endpoints store a specific timeout, a policy name for a centrally configured timeout, or an opt-out. Or are you expecting the policy name to be resolved to a timeout when building the endpoint?

[davidfowl]

This supports determining the timeout for any non-endpoint-aware resources like static files. It's optional for now, but I anticipate customers asking for it.

Whenever we add middleware that has predicate as part of the options, I question the need. We have routing matching and pipeline branching, why do we need middleware specific filters on top of that? This also came up when we were discussing output caching.

So that policy instance would get stored in the endpoint metadata and then checked for along with the attributes?

Right, much like the authorization policies

@sebastienros is working on an effort to make more of our components config aware, reloadable, etc. since this is a common requirement from real apps. They don't want to recompile to change timeouts. The API shape for that experience is negotiable. I don't think this can be removed without a proposed alternative.

I think that's fine, but I'm not a fan of making one off changes like this to new middleware. This proposal needs to be all encompassing, and we shouldn't add this API to this one place. As an example, the hosting filtering middleware supports config reloading and it doesn't take IConfiguration as input (which is good, it should be using IOptionsMonitor<T>). I don't want to end up in a situation like this again.

I don't follow. Where is there further filtering? Endpoints store a specific timeout, a policy name for a centrally configured timeout, or an opt-out. Or are you expecting the policy name to be resolved to a timeout when building the endpoint?

Endpoints have already been routed to (routing is the filtering). Routing can applies policies. e.g.

app.MapGet("/long", (Cancellation token) => client.GetSomethingAsync(token))
     .RequireHost("*:80");
      .WithTimeout(TimeSpan.FromSeconds(5));

The above endpoint is already filtered to GET requests with /long in the route and port 80.

[Tratcher]

I don't follow. Where is there further filtering? Endpoints store a specific timeout, a policy name for a centrally configured timeout, or an opt-out. Or are you expecting the policy name to be resolved to a timeout when building the endpoint?

Endpoints have already been routed to (routing is the filtering). Routing can applies policies. e.g.

app.MapGet("/long", (Cancellation token) => client.GetSomethingAsync(token))
     .RequireHost("*:80");
      .WithTimeout(TimeSpan.FromSeconds(5));

The above endpoint is already filtered to GET requests with /long in the route and port 80.

I'm still not sure which part of the design you're talking about? Which API is this for?

[Tratcher]

I've updated the proposal to remove the TimeoutLocator from RequestTimeoutPolicy, but left the one on RequestTimeoutOptions for further discussion.

I've commented out the IConfigurationSection API and left a note about considering the IConfiguration integration story.

I've also added a WithRequestTimeout overload that directly takes a RequestTimeoutPolicy.

[sebastienros]

1. If a RequestTimeoutPolicy's locator returns null, does the DefaultTimeoutLocator apply?
2. If the DefaultTimeoutLocator returns null, does the DefaultTimeout apply?
3. Named can be added and fetched. Should we be able to delete them? Probably fine not to be, since I assume we'd only want to be able to change one using TryGet, if nothing is using it it can remain in the list.
4. Duration in seconds in the attribute seems limiting. Would keep TimeSpan at least too.
5. You mention DefaultTimeoutLocator to handle things like static files. Something that was discussed in Output Caching too, is how we should be able to trigger these middleware when there is no specific endpoint, for specific routing patterns that would be combined with the selected endpoint. For instance be able to do app.AddEndpointMetadata("/static/css/*"}).WithTimeout(TimeSpan.FromSeconds(5)). Could even be using regexes if required. The metadata would then be added to the resolved endpoint's metadata. The same way metadata can be applied on endpoint "groups".

[Tratcher]

1. If a RequestTimeoutPolicy's locator returns null, does the DefaultTimeoutLocator apply?

Removed.

2. If the DefaultTimeoutLocator returns null, does the DefaultTimeout apply?

No?

3. Named can be added and fetched. Should we be able to delete them? Probably fine not to be, since I assume we'd only want to be able to change one using TryGet, if nothing is using it it can remain in the list.

We should consider Update/Remove as part of the reloadable configuration story.

4. Duration in seconds in the attribute seems limiting. Would keep TimeSpan at least too.

TimeSpan isn't a supported input for attributes. I wanted to avoid string input as both error prone and it would conflict with the policy overload.

5. You mention DefaultTimeoutLocator to handle things like static files. Something that was discussed in Output Caching too, is how we should be able to trigger these middleware when there is no specific endpoint, for specific routing patterns that would be combined with the selected endpoint. For instance be able to do app.AddEndpointMetadata("/static/css/*"}).WithTimeout(TimeSpan.FromSeconds(5)). Could even be using regexes if required. The metadata would then be added to the resolved endpoint's metadata. The same way metadata can be applied on endpoint "groups".

👍

[davidfowl]

You mention DefaultTimeoutLocator to handle things like static files. Something that was discussed in Output Caching too, is how we should be able to trigger these middleware when there is no specific endpoint, for specific routing patterns that would be combined with the selected endpoint. For instance be able to do app.AddEndpointMetadata("/static/css/*"}).WithTimeout(TimeSpan.FromSeconds(5)). Could even be using regexes if required. The metadata would then be added to the resolved endpoint's metadata. The same way metadata can be applied on endpoint "groups".

#43642

[Tratcher]

Removed DefaultTimeoutLocator.

Added RemovePolicy, though at this point maybe we should be using a thread safe collection instead.