Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
168 commits
Select commit Hold shift + click to select a range
4be4d43
Added basic pipeline to do workload set publishing. Added initial scr…
MiYanni Jul 22, 2024
cf016b5
Realized I don't want to make a new AzDO pipeline yet. So, reusing th…
MiYanni Jul 22, 2024
756cfa3
Fix calling the darc command over the version details.
MiYanni Jul 22, 2024
46cfed9
Fixed tools path. Clean up commented script.
MiYanni Jul 22, 2024
465efcd
Added logic to move the workloads into a drops folder.
MiYanni Jul 22, 2024
d7a3a59
Clean up commented out scripting. Prep prior to drop creation work.
MiYanni Jul 23, 2024
be61d02
Passing in PAT information into the gather-drop script.
MiYanni Jul 23, 2024
e90c923
Forgot to comment out parameters node.
MiYanni Jul 23, 2024
e3729ca
Add asset-filter to gather-drop.
MiYanni Jul 23, 2024
464304c
Convert the script parameters to secure strings.
MiYanni Jul 24, 2024
2ad818a
Forgot the additional arguments for ConvertTo-SecureString. Added Con…
MiYanni Jul 24, 2024
1d8bdb9
Consolidate ConvertFrom-SecureString. Fixed --output-dir.
MiYanni Jul 24, 2024
67ce7f3
Extract the workload packages into the drops directory.
MiYanni Jul 24, 2024
4f7cf15
In-progress determining how to use the Vsman SDK to build the vsman f…
MiYanni Aug 26, 2024
78d6343
Trying to update the SwixBuild version so that the Vsman stuff will b…
MiYanni Aug 28, 2024
24040fe
Got the build to work from build.cmd via the xcopy-msbuild tool versi…
MiYanni Aug 29, 2024
6784044
Progress on making a multi-build vsman target for each drop folder.
MiYanni Sep 25, 2024
de67bea
Made each workload use the drop folder as their output for creating t…
MiYanni Sep 30, 2024
f4a3333
Trying parameter use for dynamic variable access.
MiYanni Sep 30, 2024
2a9a9fd
Temporarily comment out other feeds because those feeds potentially c…
MiYanni Sep 30, 2024
33526f9
In-progress having the drop acquisition process being able to work lo…
MiYanni Oct 1, 2024
0718e95
Continued in-progress of enabling local drop download processing.
MiYanni Oct 1, 2024
7fe5688
Got the local build process for VSMAN files working.
MiYanni Oct 2, 2024
de46dfa
Merge branch 'release/8.0.4xx' into feature/workload-publish
MiYanni Oct 2, 2024
11d28fc
In-progress of getting the workload info spit out to the pipeline for…
MiYanni Oct 2, 2024
5c32157
Moved all the logic for doing the build to the download-workloads.ps1…
MiYanni Oct 3, 2024
1d2d21d
Using the 1ES task for creating a drop.
MiYanni Oct 3, 2024
463689d
Missed a slash in file path. Updated VSDrop task display name.
MiYanni Oct 3, 2024
d722f71
Using plain pats for testing the built.
MiYanni Oct 3, 2024
e1d92ae
Trying a different image pool for CI.
MiYanni Oct 3, 2024
24e059a
Split workload download and drop creation into 2 scripts. That way, P…
MiYanni Oct 4, 2024
efbc4ae
Used incorrect PAT name.
MiYanni Oct 4, 2024
03686a0
Adjusted the download script to provide the darc arguments as a string.
MiYanni Oct 4, 2024
ec8d8a1
Moved the string arguments into the loop so the loop variable is valid.
MiYanni Oct 4, 2024
81dbb7b
Trying to do the darc arguments as a single string line.
MiYanni Oct 4, 2024
c78ca25
Using lists as the argument mechanism to pass to darc.
MiYanni Oct 4, 2024
b3c0d5f
Updated the package version for Vsman because that is what is availab…
MiYanni Oct 4, 2024
2ee6bf4
Cleaned up comments in scripts. Added back downloading drops locally.
MiYanni Oct 4, 2024
a590186
Trying default settings for VSTS drop task.
MiYanni Oct 6, 2024
f4dbfae
Added proper access to the DevDiv drop service.
MiYanni Oct 6, 2024
b60ab43
Clean up defaulted value parameters.
MiYanni Oct 6, 2024
9824c11
Added process for creating componentJsonValues. This allowed me to ad…
MiYanni Oct 6, 2024
98ddfbb
Trying to see if putting the name of the PAT as the PatAuthEnvVar works.
MiYanni Oct 7, 2024
a324f33
Removing accessToken, just to see what happens.
MiYanni Oct 7, 2024
a246961
Back to using the token again. It is definitely the right token.
MiYanni Oct 7, 2024
ed160cf
Increase the timeout time for the build stage.
MiYanni Oct 7, 2024
0ae3c68
Trying an older version of the insertion task.
MiYanni Oct 11, 2024
026dd83
Used the existing DevDiv PAT for the insertion task.
MiYanni Oct 11, 2024
6cc0a31
Removing aspire from Version.Details.xml. Removing downloading pre.co…
MiYanni Oct 29, 2024
ff795fd
Added logic for handling multitarget. Added logic for mutually exclus…
MiYanni Nov 1, 2024
1d933dd
Added logic for handling primary and secondary insertion concepts.
MiYanni Nov 1, 2024
958fc7a
Merge branch 'release/8.0.4xx' into feature/workload-publish
MiYanni Nov 1, 2024
f0bb97c
Added missing dollar sign for boolean parameter.
MiYanni Nov 2, 2024
31dcfab
Moved the preview components checkbox up.
MiYanni Nov 4, 2024
a6944e5
Added ability to read the drop version number out of the .metadata fi…
MiYanni Nov 8, 2024
1aedc11
Added algorithm for hashing a drop folder.
MiYanni Nov 12, 2024
28540a3
Added the file hash logic to create-workload-drops.ps1.
MiYanni Nov 16, 2024
96dc833
Merge branch 'release/8.0.4xx' into feature/workload-publish
MiYanni Nov 16, 2024
0535537
Merge branch 'main' into feature/workload-publish-main
MiYanni Nov 16, 2024
6cd9f46
Merge branch 'main' into feature/workload-publish-main
MiYanni Nov 16, 2024
20be1ae
Trying to use a newer version of the Android workload to see if it wo…
MiYanni Nov 25, 2024
35b0780
Added android to the workloads list. Updated android workload version.
MiYanni Nov 26, 2024
b32a8ee
Merge branch 'main' into feature/workload-publish-main
MiYanni Dec 6, 2024
3eac272
Changed build pool to windows.vs2022.amd64.
MiYanni Dec 9, 2024
0addbb2
Changed build pool to 1es-windows-2022.
MiYanni Dec 9, 2024
27a3de8
Added publishing the binlog.
MiYanni Dec 9, 2024
3b218b7
Publish all artifacts.
MiYanni Dec 9, 2024
9ca2af5
Try changing the dotnet version.
MiYanni Dec 9, 2024
ea974f5
Add workaround for broken path searching for Arcade .NET Framework lo…
MiYanni Dec 9, 2024
384551e
Fix broken tools files.
MiYanni Dec 9, 2024
0ac4ac9
Add publishing workload drops. Update default branches for insertions.
MiYanni Dec 10, 2024
ae5e362
Added the ability to check if the VS Drop is already published or not.
MiYanni Dec 10, 2024
6df0491
Using a different means to check if the VS drop exists.
MiYanni Dec 11, 2024
f9ea4fc
Consolidating the workload upload check.
MiYanni Dec 11, 2024
0504657
Changed tokens for checking VS Drop and insertion PR creation.
MiYanni Dec 11, 2024
26072b6
Trying the DotNetRelease-PME service connection for creating the VS i…
MiYanni Dec 16, 2024
1dd7339
Merge branch 'feature/workload-publish-main' of https://dnceng.visual…
MiYanni Dec 16, 2024
e7e3e3b
Trying infra tokens from the fsharp insertion process.
MiYanni Dec 16, 2024
d0a23d4
Try using the MicroBuildTemplate.
MiYanni Dec 16, 2024
ac06805
Changed the base drop url to https://vsdrop.corp.microsoft.com/file/v1
MiYanni Dec 16, 2024
62abb59
Set the version information inside the component parameter expression…
MiYanni Dec 17, 2024
64bc345
Added purple circle to stages, jobs, and steps defined in this repo.
MiYanni Dec 17, 2024
e66a4f6
Show environment variables.
MiYanni Dec 17, 2024
d6db026
Added dropexe acquisition. Added InsertionTopicBranch logic.
MiYanni Dec 17, 2024
0e39ded
Trying different expression for pipeline.startTime.
MiYanni Dec 17, 2024
3216f11
Trying different expression for pipeline.startTime. Forgot the other …
MiYanni Dec 17, 2024
3e2d9ae
Trying to figure out why I couldn't get the drop dir.
MiYanni Dec 17, 2024
aad1ee2
Added ability to set VS topic branch manually. More debugging for dro…
MiYanni Dec 17, 2024
3b6d882
Incorrect indentation.
MiYanni Dec 17, 2024
e51a568
Minor cleanup.
MiYanni Dec 17, 2024
0f8c194
Temporarily use different pool.
MiYanni Dec 17, 2024
9997775
Moved the drop exe stuff to preSteps. Added general logic for using d…
MiYanni Dec 17, 2024
34315f4
Fix powershell script to be multiline via vertical bar instead of arrow.
MiYanni Dec 17, 2024
0d5a1ce
Still trying to get drop exe info.
MiYanni Dec 17, 2024
2553bc0
The folders are hidden. Added -Force so hidden things are included.
MiYanni Dec 17, 2024
41fd732
Try against the _tasks folder.
MiYanni Dec 18, 2024
d736a8f
Show task hierarchy.
MiYanni Dec 18, 2024
a13b911
Added -Force to every Get-ChildItem call.
MiYanni Dec 18, 2024
beb81eb
Publish the version folder for drop stuff.
MiYanni Dec 18, 2024
0cd3b0c
Added drop check logic.
MiYanni Dec 18, 2024
4f52351
Testing how failed list calls react.
MiYanni Dec 18, 2024
92f754b
Cleaned up testing VS drop logic. Added ability to change it in pipel…
MiYanni Dec 18, 2024
8b4e504
Remove unused parameter.
MiYanni Dec 18, 2024
3101c76
Fix task folder lookup.
MiYanni Dec 18, 2024
49f3c15
Cleaned up drop checking script. Should work once I'm able to run a f…
MiYanni Dec 18, 2024
723b854
Removed drop test parameter.
MiYanni Dec 19, 2024
c458e17
Trying empty string as default topic branch id.
MiYanni Dec 19, 2024
ba5606d
Made a sentinel value of |default|.
MiYanni Dec 19, 2024
28edda3
See if the publish task works properly.
MiYanni Jan 6, 2025
55d90c6
Added missing BuildConfiguration, which is required by Arcade.
MiYanni Jan 7, 2025
b285ec7
Added missing build args.
MiYanni Jan 7, 2025
879fed1
Change to _BuildConfig.
MiYanni Jan 7, 2025
5fb2edc
Remove version workaround. Add in logic to replace plugin.ps1 with lo…
MiYanni Jan 7, 2025
fafacfd
Trying UTF8 for access token instead of ASCII.
MiYanni Jan 8, 2025
de72121
Trying Bearer auth. Added |temp| value for topic branch name.
MiYanni Jan 8, 2025
4d74bd4
Added missing build flags for CI building.
MiYanni Jan 8, 2025
901396b
Fix broken |temp| topic branch name.
MiYanni Jan 8, 2025
8bf488b
Shorten |temp| topic branch name. Attempt to use UseDefaultCredential…
MiYanni Jan 8, 2025
e5d9896
Trying to set Credentials via NetworkCredential.
MiYanni Jan 8, 2025
5959f2c
Trying to set Credentials via PSCredential.
MiYanni Jan 8, 2025
5a661c5
Trying empty string for PSCredential username.
MiYanni Jan 8, 2025
0063c12
Made username PAT.
MiYanni Jan 8, 2025
7355e31
Explicitly set system.accesstoken environment variable. Trying differ…
MiYanni Jan 9, 2025
7b5ddca
Trying different PAT.
MiYanni Jan 9, 2025
c1f0c45
Cleanup of commented code in official.yml.
MiYanni Jan 10, 2025
54b267a
Remove custom logic for checking if drop exists remotely. Trying to r…
MiYanni Jan 10, 2025
f94bb60
Changed the way the build is started.
MiYanni Jan 10, 2025
21bc89b
Trying different insertion PAT for now.
MiYanni Jan 10, 2025
81aae99
Forgot to set msbuildEngine to vs.
MiYanni Jan 10, 2025
0fcbe4a
Trying only release pipeline PATs. Minor cleanup of other extraneous …
MiYanni Jan 10, 2025
4d2157b
Pass the workload drop list to the download-workloads.ps1 script.
MiYanni Jan 10, 2025
92e88e5
Fix broken workload list script. Use MicroBuildInsertionVS service co…
MiYanni Jan 10, 2025
b37d40b
Hope the json passes properly now.
MiYanni Jan 10, 2025
fb26558
Trying to add back PostBuildSign to see if signing continues to have …
MiYanni Jan 10, 2025
a895b5d
Adding the log folder to always be published to the artifacts.
MiYanni Jan 10, 2025
bbad4ad
Commenting out ItemsToSign.
MiYanni Jan 10, 2025
7471c03
Switched from the jobs.yml template to the job.yml template.
MiYanni Jan 10, 2025
92f9ec9
Using prior token for VS drop access.
MiYanni Jan 10, 2025
f40e5d6
Continuing cleanup.
MiYanni Jan 10, 2025
6607cf5
Go back to jobs.yml so the Publish Assets job exists again.
MiYanni Jan 10, 2025
44c2a3b
Provided the access token that should (hopefully) get used for clonin…
MiYanni Jan 10, 2025
9208473
Added workload insertion template. Stopped hotswapping plugin.ps1 files.
MiYanni Jan 11, 2025
b317935
Trying to pass in the service name.
MiYanni Jan 11, 2025
0bc02d9
Not passing in service name. Added the env access token back.
MiYanni Jan 11, 2025
31b26e6
Moved workload publish check into its own script file.
MiYanni Jan 11, 2025
86e2076
Cleanup after knowing the system accesstoken is required.
MiYanni Jan 11, 2025
0799e1f
More cleanup. Fixed wrong powershell version and _BuildConfig.
MiYanni Jan 11, 2025
ad2789c
More cleanup of variable and parameter names.
MiYanni Jan 11, 2025
a56dda2
Remove NerdbankGitVersioningTasksPath workaround. Update Microsoft.Vi…
MiYanni Jan 15, 2025
3b3e492
Added usable public.yml pipeline (for PRs).
MiYanni Jan 15, 2025
416509c
Merge branch 'main' into feature/workload-publish-main
MiYanni Jan 17, 2025
ccbb539
My merge tool had an issue with the 🟣 emoji.
MiYanni Jan 17, 2025
b521be2
Move the build into a template so that workloadDropTypes isn't exposed.
MiYanni Jan 17, 2025
e4429d5
Changed vsTopicBranchId to vsTopicBranch. This allows setting the ent…
MiYanni Jan 17, 2025
e80b8dc
Trying this connection for downloading the workloads.
MiYanni Jan 17, 2025
1db531b
Back to the other subscription.
MiYanni Jan 17, 2025
0fc7204
Updated based on PR creation review.
MiYanni Jan 18, 2025
6a4cd67
Removed unused testing plugin.ps1 replacements. Using the new skipUpl…
MiYanni Jan 27, 2025
3932dcc
Trying string 'True' for the moment on skipUploadIfExists.
MiYanni Jan 28, 2025
704b647
Debugging skipUploadIfExists issues.
MiYanni Jan 28, 2025
6f880bb
Added more debugging output.
MiYanni Jan 28, 2025
3387e00
Update filtering to not use ForEach. This should let return exit the …
MiYanni Jan 28, 2025
d58d5ff
Missed commenting out a curly brace.
MiYanni Jan 28, 2025
aba7d75
See if the new MicroBuild task works now.
MiYanni Jan 29, 2025
a00e952
Removed debugging commented code and logic. Enabled version injection…
MiYanni Jan 29, 2025
573de13
Merge branch 'main' into feature/workload-publish-main2
MiYanni Jan 29, 2025
64a48c0
Update src/Microsoft.NET.Workloads.Vsman/Microsoft.NET.Workloads.Vsma…
MiYanni Feb 4, 2025
a4b87ac
Addressing a bunch of PR comments. Testing if xcopy is needed. Change…
MiYanni Feb 5, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 0 additions & 108 deletions eng/VstsDropFolder/plugin.ps1

This file was deleted.

27 changes: 0 additions & 27 deletions eng/check-drop-publish.ps1

This file was deleted.

3 changes: 1 addition & 2 deletions eng/create-workload-drops.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,7 @@ Get-ChildItem -Path $workloadDropPath -Directory | ForEach-Object {
Write-Host "##vso[task.setvariable variable=$($shortName)_$($dropType)_url]$dropUrl"

# Each vsman file is comma-separated. First .vsman is destination and the second is source.
# $vsComponentValue = "$assemblyName.vsman{$workloadVersion}=$dropUrl,"
$vsComponentValue = "$assemblyName.vsman=$dropUrl,"
$vsComponentValue = "$assemblyName.vsman{$workloadVersion}=$dropUrl,"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: Tested logic for VS Drop access

# All VS components are added to the primary VS component JSON string.
$primaryVSComponentJsonValues += $vsComponentValue

Expand Down
73 changes: 1 addition & 72 deletions eng/pipelines/templates/jobs/workload-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,45 +48,6 @@ jobs:
-workloadListJson '${{ convertToJson(parameters.workloadDropNames) }}'
-usePreComponents:$${{ parameters.usePreComponentsForVSInsertion }}

# Sets the VstsDropFolder variable for the path to the MicroBuildUploadVstsDropFolder task.
# Used during VS insertion to verify if the drop has been uploaded already or not.
# Example: D:\a\_work\_tasks\MicroBuildUploadVstsDropFolder_4305a8de-ba66-4d8b-b2d1-0dc4ecbbf5e8\2.2.16\
# - pwsh: |
# $tasksFolder = '$(Agent.RootDirectory)\_tasks'
# # The folders in _tasks are hidden. The -Force parameter allows them to be listed.
# $taskRootFolder = Get-ChildItem $tasksFolder -Directory -Filter 'MicroBuildUploadVstsDropFolder*' -Force | Select-Object -First 1
# # Get the first version folder since the agent will only install a single version.
# $vstsDropFolder = ($taskRootFolder | Get-ChildItem -Directory -Force | Select-Object -First 1).FullName
# Write-Host "VstsDropFolder: $vstsDropFolder"
# Write-Host "##vso[task.setvariable variable=VstsDropFolder]$vstsDropFolder"
# displayName: 🟣 Set VstsDropFolder variable

# - pwsh: |
# $localPluginPath = '$(Build.SourcesDirectory)/eng/VstsDropFolder/plugin.ps1'
# Write-Host "LocalPluginPath: $localPluginPath"
# $taskPluginPath = '$(VstsDropFolder)/plugin.ps1'
# Write-Host "TaskPluginPath: $taskPluginPath"
# Copy-Item -Path $localPluginPath -Destination $taskPluginPath -Force
# displayName: 🟣 Replace VstsDropFolder plugin.ps1

# - pwsh: |
# $tasksFolder = '$(Agent.RootDirectory)\_tasks'
# # The folders in _tasks are hidden. The -Force parameter allows them to be listed.
# $taskRootFolder = Get-ChildItem $tasksFolder -Directory -Filter 'MicroBuildInsertVsPayload*' -Force | Select-Object -First 1
# # Get the first version folder since the agent will only install a single version.
# $insertVsPayload = ($taskRootFolder | Get-ChildItem -Directory -Force | Select-Object -First 1).FullName
# Write-Host "InsertVsPayload: $insertVsPayload"
# Write-Host "##vso[task.setvariable variable=InsertVsPayload]$insertVsPayload"
# displayName: 🟣 Set InsertVsPayload variable

# - pwsh: |
# $localPluginPath = '$(Build.SourcesDirectory)/eng/InsertVsPayload/plugin.ps1'
# Write-Host "LocalPluginPath: $localPluginPath"
# $taskPluginPath = '$(InsertVsPayload)/plugin.ps1'
# Write-Host "TaskPluginPath: $taskPluginPath"
# Copy-Item -Path $localPluginPath -Destination $taskPluginPath -Force
# displayName: 🟣 Replace InsertVsPayload plugin.ps1

# https://github.com/dotnet/arcade/blob/ccae251ef033746eb0213329953f5e3c1687693b/Documentation/CorePackages/Publishing.md#basic-onboarding-scenario-for-new-repositories-to-the-current-publishing-version-v3
- powershell: >-
eng/common/build.ps1
Expand Down Expand Up @@ -117,40 +78,8 @@ jobs:
# The variables comprised of workloadShortName and workloadType are set during create-workload-drops.ps1 in Microsoft.NET.Workloads.Vsman.csproj.
- ${{ each workloadShortName in parameters.workloadDropNames }}:
- ${{ each workloadType in parameters.workloadDropTypes }}:
# Only create the drop if the workload drop directory exists, or if the drop wasn't published already.
# Only create the drop if the workload drop directory exists.
# Sets the PublishWorkloadDrop variable. See: https://stackoverflow.com/a/60630739/294804
# - pwsh: >-
# eng/check-drop-publish.ps1
# -workloadId '${{ workloadShortName }}_${{ workloadType }}'
# -workloadPath '$(${{ workloadShortName }}_${{ workloadType }}_dir)'
# -workloadName '$(${{ workloadShortName }}_${{ workloadType }}_name)'
# -token (ConvertTo-SecureString -String '$(dn-bot-devdiv-drop-rw-code-rw)' -AsPlainText -Force)
# -vstsDropFolder '$(VstsDropFolder)'
# displayName: 🟣 Check if ${{ workloadShortName }}_${{ workloadType }} needs published

# - pwsh: |
# # If the drop folder doesn't exist (not downloaded via DARC), the drop cannot be published.
# if (-not (Test-Path -Path '$(${{ workloadShortName }}_${{ workloadType }}_dir)')) {
# Write-Host "##vso[task.setvariable variable=PublishWorkloadDrop]False"
# Write-Host 'Drop "${{ workloadShortName }}_${{ workloadType }}" was not downloaded via DARC. Skipping VS drop publish...'
# return
# }
# Import-Module '$(VstsDropFolder)/Engineering.PowerShell.Vsts.Drop.psd1'
# # Default value within: https://devdiv.visualstudio.com/Engineering/_git/MicroBuild?path=/src/Tasks/UploadDrop/task.json
# $serviceUri = 'https://devdiv.artifacts.visualstudio.com'
# $token = '$(dn-bot-devdiv-drop-rw-code-rw)'
# $server = Get-VstsDropServer -Url $serviceUri -TraceToHost ActivityTracing -PatAuth $token
# # Returns a JSON string. If the drop exists, an array with a single object is returned. If the drop does not exist, an empty array is returned.
# $dropJson = $server.Client.List('$(${{ workloadShortName }}_${{ workloadType }}_name)') | ConvertFrom-Json
# if ($dropJson.Count -ne 0) {
# Write-Host "##vso[task.setvariable variable=PublishWorkloadDrop]False"
# Write-Host 'Drop "${{ workloadShortName }}_${{ workloadType }}" has already been published. Skipping VS drop publish...'
# return
# }
# Write-Host "PublishWorkloadDrop: True"
# Write-Host "##vso[task.setvariable variable=PublishWorkloadDrop]True"
# displayName: 🟣 Check if ${{ workloadShortName }}_${{ workloadType }} needs published

- pwsh: |
# If the drop folder doesn't exist (not downloaded via DARC), the drop cannot be published.
if (-not (Test-Path -Path '$(${{ workloadShortName }}_${{ workloadType }}_dir)')) {
Expand Down