AspNetCoreModuleV2 should be able to restart executable if startup fails

[AnkHansen]

Is there an existing issue for this?

• I have searched the existing issues

Is your feature request related to a problem? Please describe the problem.

I am hosting our .Net 6.0 API on IIS on Windows server using the AspNetCoreModuleV2 handler.

During startup, we need to connect to a SQL server to load some customer specific configuration details.

If the SQL server is unavailable at the time the API starts, it throws an unhandled exception, and the executable exits.

This works fine when using Kestrel and running from command prompt.

When executed in IIS, it seems like the AspNetCoreModuleV2 handler tries to start the executable, it crashes, and the handles returns 500 - which is fair as it crashes.

But the handler then stays in this state, and never seems to try and restart the executable.

The restart logic must be there anyway as it can handle the process crashing later on, so seems like a flag that detects that it is during initialization it fails - and then it is decided to not retry.

If the executable crashes (or manually killed) later on after startup sequence is completed the AspNetCoreModuleV2 successfully boots a new instance of the executable.

Best regards
/Anders

Describe the solution you'd like

I would like a feature where it would be possible to configure how to handle startup errors.

In our case it happens rarely, but is 99% of the time due to the API is starting in a SQL server service window, connection failure or similar. And then we manually have to go through all API's and restart these in the faulty state. (or by scripting).

I am aware of the problems that can arise if 100's, 1000's or more requests hits the IIS - and all instances keeps crashing.

So maybe something about a window in which it keeps returning 500 for e.g. 1, 2, 5 minutes - and then tries to restart an instance of the executable again, so we don't try to restart every second - but only a few times an hour.

<aspNetCore processPath=".\myApi.exe" hostingModel="outofprocess" startupFailureWindow="00:05:00"/>

The name of the attribute is obviously up to you to decide so it matches convention of other attributes.

Additional context

Have posted the question on StackOverflow with no good result: https://stackoverflow.com/questions/75578644/how-to-handle-if-asp-net-core-api-crashed-during-building-host/75589373?noredirect=1#comment133379495_75589373

From our investigations a reproduction of the issue would just need an API that in Startup.cs throws an exception, and then using it in IIS using web.config ith AspNetCoreModuleV2.

[BrennanConroy]

This is the expected behavior with hostingModel="inprocess".
Reference: #22507 (comment)

When using hostingModel="outofprocess", IIS will restart the app automatically.
You can also catch exceptions in startup and return a negative exit code which will restart the app. This is also mentioned in the above issue. You may need to configure Rapid-Fail Protection to suit your needs.

I gave this a quick try and it worked well.

[AnkHansen]

We already are running with hostingModel="outofprocess".

I don't fully understand from your comment if I am supposed to run with hostingModel inprocess to get what I want, or outofprocess with Rapid-Fail-Protection enabled to get to the point where you would expect it to work.

I did not have try-catch in main, but even after changing my main method to the snippet below, IIS still doesn't retry to start my application.

If I run it by command line, now it exits with exitcode -1 if database can't be reached. But inside IIS, running HostingModel OutOfProcess, and Rapid-Fail-Protection set to 1 error in 1 minute - the api process that IIS starts remains having the same processId in Task Manager - so it is never restarted.

Can you please elaborate on what you tried that worked? with/without try-catch, which rapid-fail-protection settings and which HostingModel, and which .Net version?

My Main looks like this at the moment.

public static void Main(string[] args)
{
    try { 
        var host = BuildWebHost(args).Build();
        host.Run();
    }
    catch
    {
        Environment.Exit(-1);
    }
}

[AnkHansen]

Hi again

Got it working by using inprocess hostingmodel, and exiting with Environment.Exit(-1).

Rapid-Fail-Protection set to 10 minutes and 100 failures - then it retries up to 100 times and then gives up.

At some point when we moved from .Net 2.1 to .Net 6 we kept the hostingmodel as outofprocess - will have to go back and see why.

It seems like inprocess is a better fit for this case at least.

We are running hundreds of identical API's for various customers within same IIS, seperate apppools though. So it ought to work.

[adityamandaleeka]

Glad it works with the in-process hosting model. That is our recommended path these days.

[ghost]

This issue has been resolved and has not had any activity for 1 day. It will be closed for housekeeping purposes.

See our Issue Management Policies for more information.