build: move library release to internal #272
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| resources: | ||
| repositories: | ||
| - repository: 1es | ||
| type: git | ||
| name: 1ESPipelineTemplates/1ESPipelineTemplates | ||
| ref: refs/tags/release | ||
|
|
||
| extends: | ||
| template: v1/1ES.Official.PipelineTemplate.yml@1es | ||
| parameters: | ||
| pool: | ||
| name: 1es-pool-azfunc-public | ||
| image: 1es-windows-2022 | ||
| os: windows | ||
| sdl: | ||
| codeql: | ||
| compiled: | ||
| enabled: true # still only runs for default branch | ||
| runSourceLanguagesInSourceAnalysis: true | ||
| settings: | ||
| skipBuildTagsForGitHubPullRequests: ${{ variables['System.PullRequest.IsFork'] }} | ||
|
Comment on lines
+15
to
+21
There was a problem hiding this comment. this may not be required for release pipeline build tags were being skipped for public build since we needed to run public build from fork PRs. There was a problem hiding this comment. For CodeQL, the pipeline is not building anything, but it is downloading a build artifact from another pipeline. Can CodeQL be skipped in this case? There was a problem hiding this comment. yeah, we don't need to add it. Also, official templates take care of adding itself if required. |
||
|
|
||
| stages: | ||
| - stage: Release | ||
| jobs: | ||
| - template: /eng/templates/official/jobs/publish-release.yml@self | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,262 @@ | ||
| jobs: | ||
|
|
||
| - job: "CreateReleaseBranch" | ||
| displayName: 'Create Release Branch' | ||
| pool: | ||
| name: 1es-pool-azfunc | ||
| image: 1es-ubuntu-22.04 | ||
| os: linux | ||
| steps: | ||
| - powershell: | | ||
| $githubToken = "$(GithubPat)" | ||
| $newLibraryVersion = "$(NewLibraryVersion)" | ||
|
|
||
| if($newLibraryVersion -match '(\d)+.(\d)+.(\d)+') { | ||
| # Create GitHub credential | ||
| git config --global user.name "AzureFunctionsPython" | ||
| git config --global user.email "[email protected]" | ||
|
|
||
| # Heading to Artifact Repository | ||
| Write-Host "Operating based on $stagingDirectory/azure-functions-python-library" | ||
| git checkout -b "release/$newLibraryVersion" | ||
|
|
||
| # Change __init__.py version | ||
| Write-Host "Change version number in azure/functions/__init__.py to $newLibraryVersion" | ||
| ((Get-Content azure/functions/__init__.py) -replace "__version__ = '(\d)+.(\d)+.*'", "__version__ = '$newLibraryVersion'" -join "`n") + "`n" | Set-Content -NoNewline azure/functions/__init__.py | ||
| git add azure/functions/__init__.py | ||
| git commit -m "build: update Python Library Version to $newLibraryVersion" | ||
|
|
||
| # Create release branch release/X.Y.Z | ||
| Write-Host "Creating release branch release/$newLibraryVersion" | ||
| git push --repo="https://[email protected]/Azure/azure-functions-python-library.git" | ||
| } else { | ||
| Write-Host "NewLibraryVersion $newLibraryVersion is malformed (example: 1.5.0)" | ||
| exit -1 | ||
| } | ||
| displayName: 'Push release/x.y.z' | ||
|
|
||
| - job: "CheckReleaseBranch" | ||
| dependsOn: ['CreateReleaseBranch'] | ||
| displayName: '(Manual) Check Release Branch' | ||
| pool: server | ||
| steps: | ||
| - task: ManualValidation@1 | ||
| displayName: '(Optional) Modify release/x.y.z branch' | ||
| inputs: | ||
| notifyUsers: '' # No email notifications sent | ||
| instructions: | | ||
| 1. Check if the https://github.com/Azure/azure-functions-python-library/tree/release/$(NewLibraryVersion) build succeeds and passes all unit tests. | ||
| 2. If not, modify the release/$(NewLibraryVersion) branch. | ||
| 3. Ensure release/$(NewLibraryVersion) branch contains all necessary changes. | ||
|
|
||
| - job: "CreateReleaseTag" | ||
| dependsOn: ['CheckReleaseBranch'] | ||
| displayName: 'Create Release Tag' | ||
| steps: | ||
| - powershell: | | ||
| $githubToken = "$(GithubPat)" | ||
| $newLibraryVersion = "$(NewLibraryVersion)" | ||
|
|
||
| if($newLibraryVersion -match '(\d)+.(\d)+.(\d)+') { | ||
| # Create GitHub credential | ||
| git config --global user.name "AzureFunctionsPython" | ||
| git config --global user.email "[email protected]" | ||
|
|
||
| # Clone Repository | ||
| git clone https://[email protected]/Azure/azure-functions-python-library | ||
| Write-Host "Cloned azure-functions-python-library into local" | ||
| Set-Location "azure-functions-python-library" | ||
| git checkout "origin/release/$newLibraryVersion" | ||
|
|
||
| # Create release tag X.Y.Z | ||
| Write-Host "Creating release tag $newLibraryVersion" | ||
| git tag -a "$newLibraryVersion" -m "$newLibraryVersion" | ||
|
|
||
| # Push tag to remote | ||
| git push origin $newLibraryVersion | ||
| } else { | ||
| Write-Host "NewLibraryVersion $newLibraryVersion is malformed (example: 1.5.0)" | ||
| exit -1 | ||
| } | ||
| displayName: 'Tag and push x.y.z' | ||
| - powershell: | | ||
| $githubUser = "$(GithubUser)" | ||
| $githubToken = "$(GithubPat)" | ||
| $newLibraryVersion = "$(NewLibraryVersion)" | ||
|
|
||
| if($newLibraryVersion -match '(\d)+.(\d)+.(\d)+') { | ||
| # Create GitHub credential | ||
| $credential = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes("${githubUser}:${githubToken}")) | ||
|
|
||
| # Create Release Note | ||
| Write-Host "Creating release note in GitHub" | ||
| $body = (@{tag_name="$newLibraryVersion";name="Release $newLibraryVersion";body="- Fill in Release Note Here";draft=$true} | ConvertTo-Json -Compress) | ||
| $response = Invoke-WebRequest -Headers @{"Cache-Control"="no-cache";"Content-Type"="application/json";"Authorization"="Basic $credential"} -Method Post -Body "$body" -Uri "https://api.github.com/repos/Azure/azure-functions-python-library/releases" | ||
|
|
||
| # Return Value | ||
| if ($response.StatusCode -ne 201) { | ||
| Write-Host "Failed to create release note in GitHub" | ||
| exit -1 | ||
| } | ||
|
|
||
| $draftUrl = $response | ConvertFrom-Json | Select -expand url | ||
| Write-Host "Release draft created in $draftUrl" | ||
| } else { | ||
| Write-Host "NewLibraryVersion $newLibraryVersion is malformed (example: 1.1.8)" | ||
| exit -1 | ||
| } | ||
| displayName: 'Create GitHub release draft' | ||
|
|
||
| - job: "CheckGitHubRelease" | ||
| dependsOn: ['CreateReleaseTag'] | ||
| displayName: '(Manual) Check GitHub release note' | ||
| pool: server | ||
| steps: | ||
| - task: ManualValidation@1 | ||
| displayName: 'Write GitHub release note' | ||
| inputs: | ||
| notifyUsers: '' | ||
| instructions: 'Please head to https://github.com/Azure/azure-functions-python-library/releases to finish the release note' | ||
|
|
||
| - job: "TestWithWorker" | ||
| dependsOn: ['CheckGitHubRelease'] | ||
| displayName: 'Test with Worker' | ||
| steps: | ||
| - powershell: | | ||
| $githubUser = "$(GithubUser)" | ||
| $githubToken = "$(GithubPat)" | ||
| $newLibraryVersion = "$(NewLibraryVersion)" | ||
| $newBranch = "sdk/$newLibraryVersion" | ||
|
|
||
| if($newLibraryVersion -match '(\d)+.(\d)+.(\d)+') { | ||
| # Create GitHub credential | ||
| git config --global user.name "AzureFunctionsPython" | ||
| git config --global user.email "[email protected]" | ||
|
|
||
| # Create GitHub credential | ||
| $credential = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes("${githubUser}:${githubToken}")) | ||
|
|
||
| # Clone Repository | ||
| git clone https://[email protected]/Azure/azure-functions-python-worker | ||
| Write-Host "Cloned azure-functions-python-worker into local and checkout $newBranch branch" | ||
| Set-Location "azure-functions-python-worker" | ||
| git checkout -b $newBranch "origin/dev" | ||
|
|
||
| # Modify SDK Version in pyproject.toml | ||
| Write-Host "Replacing SDK version in worker's pyproject.toml" | ||
| ((Get-Content pyproject.toml) -replace '"azure-functions==(\d)+.(\d)+.*"','"azure-functions==$(NewLibraryVersion)"' -join "`n") + "`n" | Set-Content -NoNewline pyproject.toml | ||
|
|
||
| # Commit Python Version | ||
| Write-Host "Pushing $newBranch to azure-functions-python-worker repo" | ||
| git add pyproject.toml | ||
| git commit -m "Update Python SDK Version to $newLibraryVersion" | ||
| git push origin $newBranch | ||
|
|
||
| # Create PR | ||
| Write-Host "Creating PR draft in GitHub" | ||
| $body = (@{head="$newBranch";base="dev";body="Python SDK Version [$newLibraryVersion](https://github.com/Azure/azure-functions-python-library/releases/tag/$newLibraryVersion)";draft=$true;maintainer_can_modify=$true;title="build: update Python SDK Version to $newLibraryVersion"} | ConvertTo-Json -Compress) | ||
| $response = Invoke-WebRequest -Headers @{"Cache-Control"="no-cache";"Content-Type"="application/json";"Authorization"="Basic $credential";"Accept"="application/vnd.github.v3+json"} -Method Post -Body "$body" -Uri "https://api.github.com/repos/Azure/azure-functions-python-worker/pulls" | ||
|
|
||
| # Return Value | ||
| if ($response.StatusCode -ne 201) { | ||
| Write-Host "Failed to create PR in Azure Functions Python Worker" | ||
| exit -1 | ||
| } | ||
|
|
||
| $draftUrl = $response | ConvertFrom-Json | Select -expand url | ||
| Write-Host "PR draft created in $draftUrl" | ||
| } else { | ||
| Write-Host "NewLibraryVersion $newLibraryVersion is malformed (example: 1.1.8)" | ||
| exit -1 | ||
| } | ||
| displayName: 'Create PR in Worker Repo' | ||
|
|
||
| - job: "WaitForPythonWorkerPR" | ||
| dependsOn: ['TestWithWorker'] | ||
| displayName: '(Manual) Check Python Worker PR' | ||
| pool: server | ||
| steps: | ||
| - task: ManualValidation@1 | ||
| displayName: 'Check Python Worker PR' | ||
| inputs: | ||
| notifyUsers: '' | ||
| instructions: | | ||
| 1. Please wait and check if all goes green in the https://github.com/Azure/azure-functions-python-worker/pulls | ||
| 2. Merge the PR into worker dev branch | ||
|
|
||
| - job: "PyPIPackage" | ||
| dependsOn: ['WaitForPythonWorkerPR'] | ||
| displayName: 'PyPI Package' | ||
| steps: | ||
| - task: DownloadPipelineArtifact@2 | ||
| displayName: 'Download Python SDK release/x.y.z Artifact' | ||
| inputs: | ||
| buildType: specific | ||
| project: '3f99e810-c336-441f-8892-84983093ad7f' | ||
| definition: 679 | ||
| specificBuildWithTriggering: true | ||
| buildVersionToDownload: latestFromBranch | ||
| branchName: refs/heads/dev | ||
| targetPath: PythonSdkArtifact | ||
| - task: UsePythonVersion@0 | ||
| displayName: 'Use Python 3.11' | ||
| inputs: | ||
| versionSpec: 3.11 | ||
| - powershell: | | ||
| $newLibraryVersion = "$(NewLibraryVersion)" | ||
| $pypiToken = "$(PypiToken)" | ||
|
|
||
| # Setup local Python environment | ||
| Write-Host "Setup local Python environment" | ||
| python -m pip install -U pip | ||
| pip install twine | ||
|
|
||
| # Publish artifacts to PyPi | ||
| twine upload --repository-url https://upload.pypi.org/legacy/ --username "__token__" --password "$pypiToken" PythonSdkArtifact/azure-functions/dist/* | ||
| Start-Sleep -Seconds 3 | ||
|
|
||
| # Checking if the new version is uploaded | ||
| Write-Host "Check if new version is uploaded" | ||
| $response = Invoke-WebRequest -Headers @{"Cache-Control"="no-cache"} -Method Get -Uri "https://pypi.org/project/azure-functions/$newLibraryVersion/" | ||
|
|
||
| # Return Value | ||
| if ($response.StatusCode -ne 200) { | ||
| Write-Host "Failed to verify https://pypi.org/project/azure-functions/$newLibraryVersion/" | ||
| exit -1 | ||
| } | ||
| displayName: 'Publish package to pypi.org' | ||
|
|
||
| - job: "MergeBack" | ||
| dependsOn: ['PyPIPackage'] | ||
| displayName: 'Merge Back' | ||
| steps: | ||
| - powershell: | | ||
| $githubToken = "$(GithubPat)" | ||
| $newLibraryVersion = "$(newLibraryVersion)" | ||
|
|
||
| if($newLibraryVersion -match '(\d)+.(\d)+.(\d)+') { | ||
| # Create GitHub credential | ||
| git config --global user.name "AzureFunctionsPython" | ||
| git config --global user.email "[email protected]" | ||
|
|
||
| # Clone Repository | ||
| git clone https://[email protected]/Azure/azure-functions-python-library | ||
| Write-Host "Cloned azure-functions-python-library into local" | ||
| Set-Location "azure-functions-python-library" | ||
|
|
||
| # Merge back to master | ||
| Write-Host "Merging release/$newLibraryVersion back to master" | ||
| git checkout master | ||
| git merge "origin/release/$newLibraryVersion" | ||
| git push origin master | ||
|
|
||
| # Merge back to dev | ||
| Write-Host "Merging release/$newLibraryVersion back to dev" | ||
| git checkout dev | ||
| git merge "origin/release/$newLibraryVersion" | ||
| git push origin dev | ||
| } else { | ||
| Write-Host "newLibraryVersion $newLibraryVersion is malformed (example: 1.1.8)" | ||
| exit -1 | ||
| } | ||
| displayName: 'Merge release/x.y.z back to master & dev' |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pipelines in internal repo using official templates should use pool -
1es-pool-azfuncto separate internal and public workload