Backend/azure/update to latest sdks by magodo · Pull Request #36258 · hashicorp/terraform

magodo · 2025-01-03T07:06:50Z

This PR updates the azure backend authentication to match the terraform-provider-azurermprovider authentication, in several ways:

github.com/hashicorp/go-azure-helpers: v0.43.0 -> v0.71.0 （The latest one so far, used by azurerm provider v4.14.0)
github.com/hashicorp/go-azure-sdk/[resource-manager/sdk]: v0.20241212.1154051. This is the new hashicorp Azure SDK, which replaces the deprecated Azure Track1 SDK used before.
github.com/tombuildsstuff/giovanni: v0.15.1 -> v0.27.0. Meanwhile, updating the azure storage API version from 2018-11-09 to 2023-11-03.

The backend configuration logic is updated to match the provider logic. As a result, some new properties are added:

use_cli
use_aks_workload_identity
client_id_file_path
client_certificate
client_id_file_path
client_secret_file_path

One implementation detail is that the using the same Azure storage dataplane SDK, the storage client requires a base URI of the storage account, which is derived by sending a GET to the storage account. This is skipped in case the storage shared access key or sas token is specified, which is to behave identically as the current version.

Also, this PR improves the acctests in following ways:

Removing the spaghettitized code that impacts the production code merely to make the test works. Now the test code and prod code is splitted clearly
All tests run in the following patter: test client build authorizer with parameters set via env vars-> test client create test resources -> clean up these env vars -> merely using the hcl config for testing out the backend/remote client -> test client cleans up test resources. This pattern works fine in all facts, except that when you want to run tests in parallel, you'd ensure the parallelism set in go test is big enough to avoid env vars clean for the single process (launched by go test) won't interfere the paused tests.

Fixes #34322

Target Release

1.11.0

Draft CHANGELOG entry

ENHANCEMENTS

Test

# Run all the tests, except 3 of them are skipped
❯ TF_ACC=1 go test -timeout=20h -parallel=20 ./...
ok      github.com/hashicorp/terraform/internal/backend/remote-state/azure      217.326s

# Run 2 MI related tests on Azure VM
magodo@acctest-magodo-backend:~/terraform/internal/backend/remote-state/azure$ TF_RUNNING_IN_AZURE=1 TF_ACC=1 go test -parallel=2 -run='TestAccBackendManagedS
erviceIdentityBasic|TestRemoteClientManagedServiceIdentityBasic'
PASS
ok  github.com/hashicorp/terraform/internal/backend/remote-state/azure 117.385s

# Run OIDC test from GitHub action
# Following is the GitHub action log output
Run cd internal/backend/remote-state/azure
  cd internal/backend/remote-state/azure
  TF_RUNNING_IN_GITHUB_ACTIONS=1 \
  TF_ACC=1 \
  ARM_SUBSCRIPTION_ID=*** \
  ARM_TENANT_ID=*** \
  ARM_CLIENT_ID=*** \
  ARM_TEST_LOCATION=westus2 \
  go test -run="TestAccBackendGithubOIDCBasic" .
  shell: /usr/bin/bash -e {0}
...
go: download...
...
ok  	github.com/hashicorp/terraform/internal/backend/remote-state/azure	106.039s

…e-to-latest-sdks

…-11-09 to 2023-11-03

magodo · 2025-01-03T10:01:53Z

The failed "Unit Tests" is not related to this PR.

crw · 2025-01-08T21:55:56Z

Thanks for this submission, I let the HashiCorp Azure team know about this PR.

magodo · 2025-01-29T06:12:47Z

@mbfrahry Thanks for the new round of review! I've updated the code according to all the feedbacks you've provided.

Test passed:

backend/remote-state/azure on εéá backend/azure/update-to-latest-sdks via Go v1.23.3
$ TF_ACC=1 go test -timeout=20h -parallel=20 ./...
ok      github.com/hashicorp/terraform/internal/backend/remote-state/azure      264.298s

mbfrahry

Thanks for taking care of all the review comments @Magado. These changes look good and looks like we're about ready to merge once we get the all the github actions passing!

magodo · 2025-01-30T00:28:08Z

@mbfrahry CI passed now.

mbfrahry

Hey @magodo, it's nearly there but we should make that changelog entry a little more descriptive

mbfrahry · 2025-01-30T20:46:41Z

.changes/unreleased/ENHANCEMENTS-20250130-105700.yaml

@@ -0,0 +1,5 @@
+kind: ENHANCEMENTS
+body: Update the `azure` backend authentication mechanisms


We should make this more descriptive with some of the changes coming into this PR like the new fields and any bug fixes that might be fixed through these changes

trujillo-adam

I reviewed the docs and suggested some changes to improve the grammar and consistency with our style guidelines. I am uncertain about the accuracy of the suggestions because the existing description are vague. Please make any necessary corrections following the sentence patterns I added.

trujillo-adam · 2025-01-31T00:14:46Z