Is there an existing issue for this?
Describe the bug
OutputCacheMiddleware can cache empty or truncated responses when the request that populates the cache is canceled. The invalid cached response is then served to subsequent clients until the cache entry expires.
We encountered this in a production service that was using both OutputCache and ResponseCompression middleware, where clients occasionally received truncated responses from the cache.
Expected Behavior
OutputCache should not cache truncated responses.
Steps To Reproduce
This code shows a repro involving both OutputCache and ResponseCompression middleware. It prints "*** TRUNCATED ***" when a request gets a truncated response from the cache:
const int PayloadSize = 8 * 1024;
const string BaseUrl = "http://127.0.0.1:5050";
const string Url = BaseUrl + "/item";
var payload = new byte[PayloadSize];
var builder = WebApplication.CreateBuilder();
builder.WebHost.UseUrls(BaseUrl);
builder.Services.AddOutputCache();
builder.Services.AddResponseCompression(options =>
{
options.Providers.Add<GzipCompressionProvider>();
options.MimeTypes = new[] { "application/octet-stream" };
});
var app = builder.Build();
app.UseOutputCache();
app.UseResponseCompression();
app.MapGet("/item", async ctx =>
{
await Task.Delay(TimeSpan.FromSeconds(2));
ctx.Response.ContentType = "application/octet-stream";
try
{
await ctx.Response.Body.WriteAsync(payload, ctx.RequestAborted);
}
catch (OperationCanceledException)
{
ctx.Abort();
}
}).CacheOutput();
await app.StartAsync();
var handler = new HttpClientHandler { AutomaticDecompression = DecompressionMethods.GZip };
using var client = new HttpClient(handler);
client.DefaultRequestHeaders.AcceptEncoding.Add(new StringWithQualityHeaderValue("gzip"));
// Request 1: cancel after 1s (server is still in its 2s delay).
using (var cts = new CancellationTokenSource(TimeSpan.FromSeconds(1)))
{
try { await client.GetAsync(Url, cts.Token); }
catch (OperationCanceledException) { }
}
// Make sure that the first request is complete so that its result is cached.
await Task.Delay(TimeSpan.FromSeconds(3));
// Request 2: normal GET — should return PayloadSize bytes (after gzip decoding by HttpClient).
using var resp = await client.GetAsync(Url);
byte[] body = await resp.Content.ReadAsByteArrayAsync();
Console.WriteLine($"status={(int)resp.StatusCode} received={body.Length} expected={PayloadSize}{(body.Length < PayloadSize ? " *** TRUNCATED ***" : "")}");
Exceptions (if any)
No response
.NET Version
10.0.7
Anything else?
OutputCacheMiddleware has several checks intended to prevent incomplete responses from being cached. These include:
- If a
Content-Length header is present, the number of bytes written must match it.
- If
OutputCacheStream detects an exception while writing to the underlying stream, it sets BufferingEnabled = false, preventing the response from being cached.
When the Content-Length header is absent, code like the following can still result in a truncated response being cached when httpContext.RequestAborted is triggered:
try
{
// no Content-Length header set before writing
await httpContext.Response.Body.WriteAsync(payload, httpContext.RequestAborted);
}
catch (OperationCanceledException)
{
// abort the request since the response may be incomplete
httpContext.Abort();
}
In this case, the first check is bypassed because no Content-Length header is present. The second check may also fail to observe the error because a decorator stream layered above OutputCacheStream can throw before the write reaches OutputCacheStream.
In the repro above, placing ResponseCompressionMiddleware after OutputCacheMiddleware creates both conditions:
- Response compression removes any existing
Content-Length header.
- Response compression layers a decorator stream (for example,
GZipStream) above OutputCacheStream.
As a result, a canceled request can populate the cache with a truncated response that is later served to subsequent clients.
There is also an issue related to SetLocking(true). When locking is enabled, concurrent requests waiting for another request to generate the response receive the in-flight response without running the normal cacheability checks. As a result, even responses with a Content-Length header are susceptible to truncation in this mode.
This occurs because this line returns a non-null cacheEntry even in cases where the response fails the cacheability checks.
Possible fixes:
- Do not cache responses when
httpContext.RequestAborted is canceled. Many of the IActionResult types use code like the try/catch block above when writing the response, and call httpContext.Abort() to prevent serving an invalid response. This means that httpContext.RequestAborted is a strong signal that the response is invalid.
- Use the same checks when sharing a response between concurrent requests in the
SetLocking(true) case that are used when deciding if a response should be cached.
Is there an existing issue for this?
Describe the bug
OutputCacheMiddlewarecan cache empty or truncated responses when the request that populates the cache is canceled. The invalid cached response is then served to subsequent clients until the cache entry expires.We encountered this in a production service that was using both
OutputCacheandResponseCompressionmiddleware, where clients occasionally received truncated responses from the cache.Expected Behavior
OutputCacheshould not cache truncated responses.Steps To Reproduce
This code shows a repro involving both
OutputCacheandResponseCompressionmiddleware. It prints "*** TRUNCATED ***" when a request gets a truncated response from the cache:Exceptions (if any)
No response
.NET Version
10.0.7
Anything else?
OutputCacheMiddlewarehas several checks intended to prevent incomplete responses from being cached. These include:Content-Lengthheader is present, the number of bytes written must match it.OutputCacheStreamdetects an exception while writing to the underlying stream, it setsBufferingEnabled = false, preventing the response from being cached.When the
Content-Lengthheader is absent, code like the following can still result in a truncated response being cached whenhttpContext.RequestAbortedis triggered:In this case, the first check is bypassed because no
Content-Lengthheader is present. The second check may also fail to observe the error because a decorator stream layered aboveOutputCacheStreamcan throw before the write reachesOutputCacheStream.In the repro above, placing
ResponseCompressionMiddlewareafterOutputCacheMiddlewarecreates both conditions:Content-Lengthheader.GZipStream) aboveOutputCacheStream.As a result, a canceled request can populate the cache with a truncated response that is later served to subsequent clients.
There is also an issue related to
SetLocking(true). When locking is enabled, concurrent requests waiting for another request to generate the response receive the in-flight response without running the normal cacheability checks. As a result, even responses with aContent-Lengthheader are susceptible to truncation in this mode.This occurs because this line returns a non-null
cacheEntryeven in cases where the response fails the cacheability checks.Possible fixes:
httpContext.RequestAbortedis canceled. Many of theIActionResulttypes use code like the try/catch block above when writing the response, and callhttpContext.Abort()to prevent serving an invalid response. This means thathttpContext.RequestAbortedis a strong signal that the response is invalid.SetLocking(true)case that are used when deciding if a response should be cached.