ASP.NET Core RestoreNpmPackages logic is failing in offline scenario

[MichaelSimons]

The Fedora39_Offline_CurrentSourceBuiltSdk_x64 CI leg is failing with the following error:

/vmr/repo-projects/Directory.Build.targets(209,5): error MSB3073: The command "/vmr/src/aspnetcore/eng/build.sh  --ci --configuration Release --restore --build --pack  -bl /p:ArcadeBuildFromSource=true /p:CopyWipIntoInnerSourceBuildRepo=true /p:CopySrcInsteadOfClone=true /p:DotNetPackageVersionPropsPath="/vmr/artifacts/obj/x64/Release/PackageVersions.aspnetcore.props" /p:AdditionalSourceBuiltNupkgCacheDir="/vmr/artifacts/obj/x64/Release/blob-feed/packages/" /p:ReferencePackageNupkgCacheDir="/vmr/prereqs/packages/reference/" /p:PreviouslySourceBuiltNupkgCacheDir="/vmr/prereqs/packages/previously-source-built/" /p:SourceBuildUseMonoRuntime= /p:DotNetBuildVertical=false /p:CrossBuild= --arch x64 --no-build-repo-tasks --no-build-nodejs /p:PortableBuild=false /p:TargetRuntimeIdentifier=fedora.39-x64 /p:MicrosoftNetFrameworkReferenceAssembliesVersion=1.0.0 >> /vmr/artifacts/logs/aspnetcore.log 2>&1" exited with code 1. [/vmr/repo-projects/aspnetcore.proj]
  warning: Some managed projects depend on NodeJS projects. Building NodeJS is disabled so the managed projects will fallback to using the output from previous builds. The output may not be correct or up to date.
    Checking node version...
    v20.10.0
    Checking npm version...
    10.2.3
    npm notice 
    npm notice New minor version of npm available! 10.2.3 -> 10.3.0
    npm notice Changelog: <[https://github.com/npm/cli/releases/tag/v10.3.0>](https://github.com/npm/cli/releases/tag/v10.3.0%3E)
    npm notice Run `npm install -g [email protected]` to update!
    npm notice 
    npm ERR! code ETIMEDOUT
    npm ERR! errno ETIMEDOUT
    npm ERR! network request to https://sqdvsblobprodcus333.vsblob.vsassets.io/b-b55de4ed4b5a4215a8e40a0a5f71e7d8/DF06192086E589B85280558D379B08B0BADE1E447E679F26AC709F372FA06D1F00.blob?sv=2019-07-07&sr=b&si=1&sig=ZZDrfGyfARx8ePDcLdFIa9rxonYH1QciI8H81cjIce8%3D&spr=https&se=2024-01-11T15%3A00%3A20Z&rscl=x-e2eid-f3b083fe-fed9489f-88064ab7-5dff1f0a-session-f3b083fe-fed9489f-88064ab7-5dff1f0a&rscd=attachment%3B%20filename%3D%22enhanced-resolve-4.5.0.tgz%22&P1=1704982854&P2=1&P3=2&P4=hWPUEN8dOl0JS%2fj2so2SZgBWhnwBQaycRG%2fewM%2fxz5I%3d failed, reason: 
    npm ERR! network This is a problem related to network connectivity.
    npm ERR! network In most cases you are behind a proxy or have bad network settings.
    npm ERR! network 
    npm ERR! network If you are behind a proxy, please make sure that the
    npm ERR! network 'proxy' config is set properly.  See: 'npm help config'
    
    npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2024-01-11T10_18_26_133Z-debug-0.log
  /vmr/src/aspnetcore/eng/SourceBuild.props(99,5): error MSB3073: The command "npm ci" exited with code 1. [/vmr/artifacts/source-built-sdks/Microsoft.DotNet.Arcade.Sdk/tools/Build.proj]
##[error]/vmr/src/aspnetcore/eng/SourceBuild.props(99,5): error MSB3073: The command "npm ci" exited with code 1.

Failing build (Microsoft internal link)

It is odd that other offline legs are not failing - especially on the same distro - e.g. Fedora39_Offline_MsftSdk_x64. It would be nice to know why that is.

This appears related to dotnet/aspnetcore#53154.

[MichaelSimons]

@wtgodbe - can you take a look?

[MichaelSimons]

@mthalman pointed out to me that it looks like they have an option to remove the version check (assuming this is the source of the online check):
npm/npm#17453 (comment)

[mthalman]

This is still failing, at least in the Alpine319_Online_MsftSdk_x64 build leg (internal link). I can't check the state of the original build leg that this was logged for because it's failing on an issue earlier in the build.

[MichaelSimons]

The skip version check was not added for the Alpine image. Could that be the problem?

[mthalman]

The skip version check was not added for the Alpine image. Could that be the problem?

Yes, that was it. Confirmed with a test build. Submitted a fix: dotnet/dotnet-buildtools-prereqs-docker#962

[MichaelSimons]

This is still failing in the CentOS Stream8 legs - https://dev.azure.com/dnceng/internal/_build/results?buildId=2356798&view=logs&j=3d0d030d-16a0-525f-3599-c6948332c359&t=57c3b827-ae8f-5728-95a9-9ba020b689a4 (Microsoft internal link)

[mthalman]

This error is still popping up: example build 1, example build 2 (internal links). This was working in previous builds. It would seem that this is a sporadic error and that the environment variable that was added doesn't really work?

[omajid]

Since this is an offline build, how do we get the npm packages into the build system? Or is this "offline" mode limited to .NET only?

[mthalman]

Since this is an offline build, how do we get the npm packages into the build system? Or is this "offline" mode limited to .NET only?

We need to revisit the whole NPM package strategy. Source build is already configured to not build with Node but it's not being honored correctly. That will be fixed by dotnet/aspnetcore#53489.