[release/5.0] Update dependencies from dotnet/arcade #36838

dotnet-maestro · 2021-09-22T14:24:16Z

This pull request updates the following dependencies

From https://github.com/dotnet/arcade

Subscription: f918d9d3-5a78-4954-eaff-08d86aff023e
Build: 20210921.4
Date Produced: September 21, 2021 7:50:02 PM UTC
Commit: f2ebec887288367b18d3299cce38ae3365215555
Branch: refs/heads/release/5.0

Updates:
- Microsoft.DotNet.Build.Tasks.Installers: from 5.0.0-beta.21427.7 to 5.0.0-beta.21471.4
- Microsoft.DotNet.Arcade.Sdk: from 5.0.0-beta.21427.7 to 5.0.0-beta.21471.4
- Microsoft.DotNet.Helix.Sdk: from 5.0.0-beta.21427.7 to 5.0.0-beta.21471.4

Updates to .NET SDKs:
- Updates sdk.version to 5.0.401
- Updates tools.dotnet to 5.0.401

…921.4 Microsoft.DotNet.Build.Tasks.Installers , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Helix.Sdk From Version 5.0.0-beta.21427.7 -> To Version 5.0.0-beta.21471.4

ghost

Auto-approving dependency update.

mmitche · 2021-09-22T17:30:55Z

@dougbu @wtgodbe Can you take a look at these failures? I think we may need this to ship 5.0 in Oct

wtgodbe · 2021-09-22T17:53:38Z

All template test failures:

System.InvalidOperationException : Failed to uninstall previous templates. The template 'web' could still be found.

@pranavkm @javiercn @SteveSandersonMS could you take a look?

wtgodbe · 2021-09-22T17:54:35Z

Looks like it's here:

aspnetcore/src/ProjectTemplates/Shared/TemplatePackageInstaller.cs

Line 112 in 493ed8b

await VerifyCannotFindTemplateAsync(output, "web");

dougbu · 2021-09-22T23:15:17Z

I think we may need this to ship 5.0 in Oct

Because the VS on our agents may be too new for the 5.0.207 SDK❔ Or, is there something else in the newer Arcade we likely need❔

mmitche · 2021-09-22T23:21:44Z

I think we may need this to ship 5.0 in Oct

Because the VS on our agents may be too new for the 5.0.207 SDK❔ Or, is there something else in the newer Arcade we likely need❔

Ahh I misread the SDK in use. This repo should be fine. Carry on.

dougbu · 2021-09-24T21:07:45Z

It should hurt to update the SDK once our VS agents target v16.11 but the SDK change is breaking, bringing in something that regresses our required template removals in tests. I'm very surprised to have something like this required in a "regular" Arcade update. Was this announced in some forum❔

@wli3 @dsplaisted @marcpopMSFT what reaction is necessary to handle the breaking change in template removal (and perhaps installation as well)❔

/fyi we keep ahead of Arcade's minimum SDK in every round of branding updates. I'm not thrilled to have the two tied together. (Yeah, I'm agreeing w/ @ViktorHofer here.)

/cc @markwilkie for awareness of the breaking change in the Arcade SDK

dsplaisted · 2021-09-24T21:12:39Z

@dougbu What is the breaking change? Previously you could remove (built-in?) project templates, but now you can't?

I know there are changes to the template system in 6.0, but I wouldn't expect something to have changed in 5.0.

@phenning @KathleenDollard @dotnet/templating-engine-maintainers

dougbu · 2021-09-24T21:34:46Z

Error message

System.InvalidOperationException : Failed to uninstall previous templates. The template 'web' could still be found.

Sample stack

   at Templates.Test.Helpers.TemplatePackageInstaller.VerifyCannotFindTemplateAsync(ITestOutputHelper output, String templateName) in /_/src/ProjectTemplates/Shared/TemplatePackageInstaller.cs:line 185
   at Templates.Test.Helpers.TemplatePackageInstaller.InstallTemplatePackages(ITestOutputHelper output) in /_/src/ProjectTemplates/Shared/TemplatePackageInstaller.cs:line 138
   at Templates.Test.Helpers.TemplatePackageInstaller.EnsureTemplatingEngineInitializedAsync(ITestOutputHelper output) in /_/src/ProjectTemplates/Shared/TemplatePackageInstaller.cs:line 69
   at Templates.Test.Helpers.ProjectFactoryFixture.GetOrCreateProject(String projectKey, ITestOutputHelper output) in /_/src/ProjectTemplates/Shared/ProjectFactoryFixture.cs:line 55
   at Templates.Test.WebApiTemplateTest.PublishAndBuildWebApiTemplate(String languageOverride, String auth, String[] args) in /_/src/ProjectTemplates/test/WebApiTemplateTest.cs:line 72
--- End of stack trace from previous location ---

Likely culprit

The TemplatePackageInstaller is rather different in our 'release/5.0' and 'main' branches. Here, the relevant bits are

aspnetcore/src/ProjectTemplates/Shared/TemplatePackageInstaller.cs

Lines 119 to 138 in 60819f7

    
           var proc = await RunDotNetNew(output, "--uninstall --help"); 
        
           var lines = proc.Output.Split(Environment.NewLine); 
        
           // Remove any previous or prebundled version of the template packages 
        
           foreach (var packageName in _templatePackages) 
        
           { 
        
               // Depending on the ordering, there may be multiple matches: 
        
               // Microsoft.DotNet.Web.Spa.ProjectTemplates.3.0.3.0.0-preview7.*.nupkg 
        
               // Microsoft.DotNet.Web.Spa.ProjectTemplates.3.0.0-preview7.*.nupkg 
        
               // Error on the side of caution and uninstall all of them 
        
               foreach (var command in lines.Where(l => l.Contains("dotnet new") && l.Contains(packageName, StringComparison.OrdinalIgnoreCase))) 
        
               { 
        
                   var uninstallCommand = command.TrimStart(); 
        
                   Debug.Assert(uninstallCommand.StartsWith("dotnet new")); 
        
                   uninstallCommand = uninstallCommand.Substring("dotnet new".Length); 
        
                   await RunDotNetNew(output, uninstallCommand); 
        
               } 
        
           } 
        
           await VerifyCannotFindTemplateAsync(output, "web");

aspnetcore/src/ProjectTemplates/Shared/TemplatePackageInstaller.cs

Line 148 in 60819f7

var result = await RunDotNetNew(output, $"--install \"{packagePath}\"");

aspnetcore/src/ProjectTemplates/Shared/TemplatePackageInstaller.cs

Lines 71 to 82 in 60819f7

    
           public static async Task<ProcessEx> RunDotNetNew(ITestOutputHelper output, string arguments) 
        
           { 
        
               var proc = ProcessEx.Run( 
        
                   output, 
        
                   AppContext.BaseDirectory, 
        
                   DotNetMuxer.MuxerPathOrDefault(), 
        
                   $"new {arguments} --debug:custom-hive \"{CustomHivePath}\""); 
        
               await proc.Exited; 
        
               return proc; 
        
           }

In 'main', we use --debug:disable-sdk-templates --debug:custom-hive \"{CustomHivePath}\", not just --debug:custom-hive \"{CustomHivePath}\" and delete the custom hive instead of uninstalling anything.

Note

The addition of --debug:disable-sdk-templates broke us a while ago. I'm surprised something similar is happening w/ a 5.0.x SDK.

Another cc

@pranavkm

vlada-shubina · 2021-09-26T07:54:42Z

We haven't done any changes to 5.0 in a while. You definitely don't need --debug:disable-sdk-templates in 5.0

dougbu · 2021-09-26T18:03:34Z

We haven't done any changes to 5.0 in a while. You definitely don't need --debug:disable-sdk-templates in 5.0

Could you please help us understand why the template tests are failing then❔ I'll put up a draft PR in a few minutes moving 'release/5.0' to the latest SDK but not changing anything about Arcade to confirm the v5.0.207 -> v5.0.401 change is what's breaking us.

dougbu · 2021-09-26T18:06:54Z

https://dev.azure.com/dnceng/public/_build/results?buildId=1387068&view=results is the build to watch

dougbu · 2021-09-26T19:59:18Z

Errors like the following failed numerous template tests due solely to the switch to use the 5.0.401 SDK. E.g. from https://dev.azure.com/dnceng/public/_build/results?buildId=1387068&view=ms.vss-test-web.build-test-results-tab&runId=40385670&resultId=100002&paneView=debug:

System.InvalidOperationException : Failed to uninstall previous templates. The template 'web' could still be found.

As you can see in #36989, I changed only the SDK. There is a breaking change between 5.0.207 and 5.0.401. Is this a bug that should be addressed in 5.0.402❔ If not, what is the workaround❔

dougbu · 2021-09-26T20:11:15Z

/fyi every Blazor and project template test is failing with the minor exception of skipped tests and a couple of project template tests that don't attempt to install templates (they just check byte order marks).

vlada-shubina · 2021-09-27T15:59:40Z

I added the fix in #37011, CI is in progress now; however templates job passed.

Basically it modifies VerifyCannotFindTemplateAsync same as main branch now:
https://github.com/dotnet/aspnetcore/blob/main/src/ProjectTemplates/Shared/TemplatePackageInstaller.cs#L142-L161

We removed search for the templates on missing template in .NET 5.0.3xx with introducing --search option instead.

DavidKarlas · 2021-09-27T16:22:34Z

I pushed @vlada-shubina commit to this branch to get green build asap...

dougbu · 2021-09-27T19:26:07Z

Reran failed jobs in 20210927.16 because I don't want to wait for any quarantining (and we tend not to do much quarantining in servicing branches anyhow).

wtgodbe · 2021-09-27T20:28:39Z

Darn it. Cancelling builds now

dougbu · 2021-09-27T20:33:41Z

Thanks for catching things before builds got too far @wtgodbe ☹️

Update dependencies from https://github.com/dotnet/arcade build 20210…

798c1af

…921.4 Microsoft.DotNet.Build.Tasks.Installers , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Helix.Sdk From Version 5.0.0-beta.21427.7 -> To Version 5.0.0-beta.21471.4

dotnet-maestro bot requested review from dougbu, Pilchie and a team as code owners September 22, 2021 14:24

ghost added area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework Type: Dependency Update 🔼 labels Sep 22, 2021

ghost approved these changes Sep 22, 2021

View reviewed changes

vlada-shubina mentioned this pull request Sep 27, 2021

fix template tests for SDK 5.0.401 #37011

Closed

fix tests for 5.0.401

dc6f3a0

dotnet-maestro bot merged commit 3628abc into release/5.0 Sep 27, 2021

dotnet-maestro bot deleted the darc-release/5.0-23feed3d-94a1-403b-9cc0-9608bb4f7f75 branch September 27, 2021 20:26

dougbu added this to the 5.0.12 milestone Oct 5, 2021

[release/5.0] Update dependencies from dotnet/arcade #36838

[release/5.0] Update dependencies from dotnet/arcade #36838

Uh oh!

Conversation

dotnet-maestro bot commented Sep 22, 2021

From https://github.com/dotnet/arcade

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

mmitche commented Sep 22, 2021

Uh oh!

wtgodbe commented Sep 22, 2021

Uh oh!

wtgodbe commented Sep 22, 2021

Uh oh!

dougbu commented Sep 22, 2021

Uh oh!

mmitche commented Sep 22, 2021

Uh oh!

dougbu commented Sep 24, 2021

Uh oh!

dsplaisted commented Sep 24, 2021

Uh oh!

dougbu commented Sep 24, 2021

Error message

Likely culprit

Note

Another cc

Uh oh!

vlada-shubina commented Sep 26, 2021

Uh oh!

dougbu commented Sep 26, 2021

Uh oh!

dougbu commented Sep 26, 2021

Uh oh!

dougbu commented Sep 26, 2021

Uh oh!

dougbu commented Sep 26, 2021

Uh oh!

vlada-shubina commented Sep 27, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DavidKarlas commented Sep 27, 2021

Uh oh!

dougbu commented Sep 27, 2021

Uh oh!

wtgodbe commented Sep 27, 2021

Uh oh!

dougbu commented Sep 27, 2021

Uh oh!

Uh oh!

vlada-shubina commented Sep 27, 2021 •

edited

Loading