Always install node & build node components in CI #53154

wtgodbe · 2024-01-04T20:58:49Z

Needed as part of the work to move to NPM

javiercn · 2024-01-05T11:59:19Z

/azp run

azure-pipelines · 2024-01-05T11:59:35Z

Azure Pipelines successfully started running 3 pipeline(s).

javiercn · 2024-01-05T12:32:53Z

.azure/pipelines/ci.yml

@@ -536,20 +526,18 @@ stages:
        jobName: Linux_musl_x64_build
        jobDisplayName: "Build: Linux Musl x64"
        agentOs: Linux
-        container: mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-WithNode


This thing is pinning the nodejs version and it's using a very old version.

I've switched to 3.17 and let the Install Nodejs task do the right thing.

Is there a reason why we don't have an image that it is based on alpine:latest (3.19) so that we can alternatively install nodejs in the image? (In case we need it). For example, latest comes with nodejs 20.x, which is ideally what we want.

We just haven't set one up yet. The ownership of creating docker images has always been somewhat obtuse to me, but we're allowed to add images to https://github.com/dotnet/dotnet-buildtools-prereqs-docker if we want. I'd probably just add a new node image to the https://github.com/dotnet/dotnet-buildtools-prereqs-docker/tree/main/src/alpine/3.18 set. What errors were you seeing with the other images you tried out in this PR?

See https://learn.microsoft.com/en-us/azure/devops/pipelines/process/container-phases?view=azure-devops#linux-based-containers

See https://learn.microsoft.com/en-us/azure/devops/pipelines/process/container-phases?view=azure-devops#bring-your-own-nodejs

We should start with something like https://learn.microsoft.com/en-us/azure/devops/pipelines/process/container-phases?view=azure-devops#full-example-of-a-dockerfile but obviously using a modern alpine image 3.19 contains node and can be installed via apk update && apk add nodejs npm which brings in node 20.10.

Ideally what we want is make sure that the node we use in the build is the one that we install, as that "frees us" from having to install it through a separate mechanism.

.azure/pipelines/ci.yml

wtgodbe · 2024-01-05T21:19:07Z

@javiercn looks like linux-musl is working & building node components now

Enable building nodejs

Try with 3.15-local

Remove local suffix

Go back to 3.17

javiercn · 2024-01-08T08:05:36Z

@wtgodbe looks great so far!

I've pushed a small extra "experimental" commit to delete the checked-in files and see if it breaks anywhere and there's something that we missed.

If that mostly works (the only thing I expect to break at this point is source build), then, we can do either 2 things:

Revert my commit and get this merged.
Fix source build in this PR (maybe we could add an image like the one you have in the tag here) and merge all of it.
- We can then have a small follow up issue to update the image tags to stable versions.

javiercn

Try removing build condition for node projects

* [Blazor] Remove checked-in JS files * Update .gitignore and fix paths * Update images to Centos8 * Update ubuntu images * Make sure NPM scripts build inside sourcebuild * Disable running Firefox on Mac OS

wtgodbe · 2024-01-09T01:49:02Z

#53228

MichaelSimons · 2024-01-09T14:25:24Z

eng/SourceBuild.props

@@ -83,4 +83,35 @@
    </ItemGroup>
  </Target>

+  <Target Name="RestoreNpmPackages"


What are the prereqs for this? I am wondering if changes are going to be needed to the containers used in SB CI. This should be flushed out to prevent blocking the dependency flow into installer.

The image needs node 20 (the LTS) version to be part of the image.

Only one of the container images used in SB CI has node - ubuntu 22.04 and it is extremely old. The list is specified here - https://github.com/dotnet/installer/blob/main/eng/pipelines/templates/stages/vmr-build.yml#L19 Can someone from aspnet add node prior to this flowing into installer?

Regarding the version - has there been any discussion with @dotnet/distro-maintainers to ensure this is the minimum version supported across the distros that will source-build .NET 9.0? @leecow this is something to raise in the next source-build partner sync.

Thanks, @MichaelSimons ! Since this becomes a build-time dependency with minimum version, I agree on calling this out with distros and partners. After all, this impacts whether/where we can build .NET 9.

Hopefully, it shouldn't be an issue this time around.

https://github.com/nodejs/release#release-schedule is a good resource for Node.js versioning. The last version before 20 (18) goes EOL early 2025 - so just a few months after .NET 9 GA. I don't think using that is really an option for .NET 9, unless a rebase is on the table during a servicing release just a few months post GA. Even Node.js 20 goes EOL in mid-2026, (almost) aligning with the expected .NET 9 EOL.

@MichaelSimons we've already added Node to centos-stream8, as well as new alpine image - alpine-3.19-WithNode. Does every image in that list need to have node added?

We need to test on each of these distros. We can update to a newer version per what .NET 9 will support when it releases (e.g. updating to the new Alpine 3.19 image)

Gotcha - so to make sure I understand:

alpine-3.17: can be updated to alpine-3.19-WithNode

centos-stream8: already has Node 20

centos-stream9, fedora-39, ubuntu-22.04. ubuntu-22.04-arm64 : need to add Node 20

Does that sound right? If so I can do that today.

Does that sound right? If so I can do that today.

Yes that sounds correct.

dotnet/dotnet-buildtools-prereqs-docker#949

wtgodbe requested a review from javiercn January 4, 2024 20:58

wtgodbe requested a review from a team as a code owner January 4, 2024 20:58

ghost added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label Jan 4, 2024

javiercn reviewed Jan 5, 2024

View reviewed changes

.azure/pipelines/ci.yml Outdated Show resolved Hide resolved

javiercn reviewed Jan 5, 2024

View reviewed changes

.azure/pipelines/ci.yml Outdated Show resolved Hide resolved

javiercn reviewed Jan 5, 2024

View reviewed changes

.azure/pipelines/ci.yml Outdated Show resolved Hide resolved

wtgodbe changed the title ~~Always install node in CI~~ Always install node & build node components in CI Jan 5, 2024

wtgodbe requested a review from a team as a code owner January 8, 2024 07:56

wtgodbe and others added 7 commits January 8, 2024 08:58

Always install node in CI

23f2848

Update ci.yml

4a78ef4

Enable building nodejs

Update to Alpine 3.17

c1dcff0

Update .azure/pipelines/ci.yml

1fdc3e2

Try with 3.15-local

Update .azure/pipelines/ci.yml

49e9895

Remove local suffix

Update .azure/pipelines/ci.yml

d635847

Go back to 3.17

Update ci.yml

dc11a2a

javiercn force-pushed the wtgodbe/InstallNode branch from 471f9f9 to 75fa7ad Compare January 8, 2024 07:59

javiercn reviewed Jan 8, 2024

View reviewed changes

javiercn force-pushed the wtgodbe/InstallNode branch from 39f827d to dc11a2a Compare January 8, 2024 09:13

javiercn approved these changes Jan 8, 2024

View reviewed changes

[Blazor] Remove checked-in JS files (#53209)

aca017a

* [Blazor] Remove checked-in JS files * Update .gitignore and fix paths * Update images to Centos8 * Update ubuntu images * Make sure NPM scripts build inside sourcebuild * Disable running Firefox on Mac OS

wtgodbe requested review from BrennanConroy, halter73 and a team as code owners January 8, 2024 20:28

wtgodbe merged commit 0bc8a2f into main Jan 9, 2024

wtgodbe deleted the wtgodbe/InstallNode branch January 9, 2024 01:49

ghost added this to the 9.0-preview1 milestone Jan 9, 2024

wtgodbe mentioned this pull request Jan 9, 2024

Build blazor.*.js in CI; don't commit the artifacts #11592

Closed

javiercn mentioned this pull request Jan 9, 2024

Use an image w/ node for SourceBuild #52969

Closed

MichaelSimons reviewed Jan 9, 2024

View reviewed changes

wtgodbe mentioned this pull request Jan 9, 2024

Add Node to images used in SourceBuild dotnet/dotnet-buildtools-prereqs-docker#949

Merged

This was referenced Jan 11, 2024

ASP.NET Core RestoreNpmPackages logic is failing in offline scenario dotnet/source-build#3925

Closed

ASP.NET Core node error in CI on Ubuntu2204 legs dotnet/source-build#3927

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Always install node & build node components in CI #53154

Always install node & build node components in CI #53154

wtgodbe commented Jan 4, 2024

javiercn commented Jan 5, 2024

azure-pipelines bot commented Jan 5, 2024

javiercn Jan 5, 2024

wtgodbe Jan 5, 2024

javiercn Jan 5, 2024

wtgodbe commented Jan 5, 2024 •

edited

Loading

javiercn commented Jan 8, 2024

javiercn left a comment

wtgodbe commented Jan 9, 2024

MichaelSimons Jan 9, 2024

javiercn Jan 9, 2024

MichaelSimons Jan 9, 2024

omajid Jan 9, 2024

wtgodbe Jan 9, 2024

MichaelSimons Jan 9, 2024

wtgodbe Jan 9, 2024

MichaelSimons Jan 9, 2024

wtgodbe Jan 9, 2024

Always install node & build node components in CI #53154

Always install node & build node components in CI #53154

Conversation

wtgodbe commented Jan 4, 2024

javiercn commented Jan 5, 2024

azure-pipelines bot commented Jan 5, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wtgodbe commented Jan 5, 2024 • edited Loading

javiercn commented Jan 8, 2024

javiercn left a comment

Choose a reason for hiding this comment

wtgodbe commented Jan 9, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wtgodbe commented Jan 5, 2024 •

edited

Loading