Time To Frist Response regression on NativeAOT

[sebastienros]

Windows and Linux

Before:

crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/goldilocks.benchmarks.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/steadystate.profile.yml --scenario todosapipublishaot --profile intel-win-app --profile intel-lin-load --profile amd-lin2-db --application.packageReferences Microsoft.DotNet.ILCompiler=8.0.0-rc.2.23470.7 --application.aspNetCoreVersion 8.0.0-rtm.23470.10 --application.runtimeVersion 8.0.0-rc.2.23470.7 --application.sdkVersion 8.0.100-rtm.23470.21 --application.environmentVariables DOTNET_GCDynamicAdaptationMode=1 --application.framework net8.0 --application.options.collectCounters true

After:

crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/goldilocks.benchmarks.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/steadystate.profile.yml --scenario todosapipublishaot --profile intel-win-app --profile intel-lin-load --profile amd-lin2-db --application.packageReferences Microsoft.DotNet.ILCompiler=8.0.0-rc.2.23471.30 --application.aspNetCoreVersion 8.0.0-rtm.23471.8 --application.runtimeVersion 8.0.0-rc.2.23471.30 --application.sdkVersion 8.0.100-rtm.23471.13 --application.environmentVariables DOTNET_GCDynamicAdaptationMode=1 --application.framework net8.0 --application.options.collectCounters true

[sebastienros]

@MichalStrehovsky @eerhardt

[eerhardt]

The runtime diff: dotnet/runtime@49bf70a...b1dbbb9

There were 2 other runtime builds in between those commits - "8.0.0-rc.2.23471.3" and "8.0.0-rc.2.23471.9". Trying to isolate this, it appears the regression came in between 8.0.0-rc.2.23470.7 and 8.0.0-rc.2.23471.3. Here is the commit sha of each of the 4 builds:

8.0.0-rc.2.23470.7- 49bf70a429a6122217f2c88e778ce8115ceac3bd
8.0.0-rc.2.23471.3- 1757497f37245d1756870fb1c0b3221c0d235d54
8.0.0-rc.2.23471.9- 12fc9044307e32e143eb7cbaf1d68badf61b192c
8.0.0-rc.2.23471.30 - b1dbbb964a4daba07fe13c4150133f1d5d41ceb0

The diff between the builds that seem to cause the regression is: dotnet/runtime@49bf70a...1757497

[MichalStrehovsky]

Given the full chart with the workstation GC mode looks like this (i.e. startup with workstation GC unaffected):

My bet is on the DATAS change in that commit range. Cc @Maoni0

Btw @sebastienros any plans to start measuring main? Looks like we also have a size regression in main already and being able to pinpoint a commit range would be helpful.

[sebastienros]

@MichalStrehovsky We are already measuring main but display these in KPIs only. So we have the whole data for the past month on about 10 benchmarks. You could try a dichotomy to find the commit that regressed it if you know the benchmark that is affected (I can explain how if you have never done that with crank). But it's limited to the most common ones because the SDK doesn't support anything that has a package reference right now when targeting .net90.

[sebastienros]

Was due to switch to anoother server.