SOPS (AWS KMS) in Gitlab CI pipeline: Failed to get the data key required to decrypt the SOPS file.

[ccabsolute]

I have the following set up to use SOPS AWS KMS decryption:

~/.aws/credentials:
[default]
aws_access_key_id = *******
aws_secret_access_key = ********

~/.aws/config:
[profile secrets-editor]
region = us-west-2
output = json
role_arn = arn:aws:iam::222244446666:role/KmsEncryptDecryptRole
source_profile = default
role_session_name = cc

.sops.yaml:
- kms: 'arn:aws:kms:us-west-2:222244446666:key/1a2b34c5-d678-9e01-2345-fg67hi8j9012'
  path_regex: path/to/secrets/.*

When I ran the following command locally it works:

export AWS_SDK_LOAD_CONFIG=1
export AWS_PROFILE=secrets-editor
sops --verbose -i -d path/to/secrets/DB_CREDS.enc.txt

But when I set it up in a GitLab CI pipeline job below:

sops_test:
  stage: test
  image: docker:20.10.7
  script:
    - apk add curl bash jsonnet jq git gpgme
    - mkdir -p /root/.aws
    - echo "$AWS_CREDENTIALS" >> /root/.aws/credentials
    - echo "$AWS_CONFIG" > /root/.aws/config
    - curl -Lo ./sops "https://github.com/mozilla/sops/releases/download/v3.6.1/sops-v3.6.1.linux"
    - chmod +x ./sops
    - mv ./sops /usr/local/bin/
    - export AWS_SDK_LOAD_CONFIG=1
    - export AWS_PROFILE=secrets-editor
    - sops --verbose -i -d path/to/secrets/DB_CREDS.enc.txt

It failed with the following error:

[AWSKMS]	 time="2021-10-19T00:50:05Z" level=info msg="Decryption failed" arn="arn:aws:kms:us-west-2:222244446666:key/1a2b34c5-d678-9e01-2345-fg67hi8j9012"
Failed to get the data key required to decrypt the SOPS file.
Group 0: FAILED
  arn:aws:kms:us-west-2:222244446666:key/1a2b34c5-d678-9e01-2345-fg67hi8j9012: FAILED
    - | Error decrypting key: AccessDeniedException: The ciphertext
      | refers to a customer master key that does not exist, does
      | not exist in this region, or you are not allowed to access.
      | 	status code: 400, request id:
      | e9271c89-c1a7-4d56-b28a-bd2084763d2d
Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

I'm using the same ~/.aws/* files inside the CI pipeline, so there shouldn't be any permission issue. If I install AWS CLI in the pipeline container, and use the same profile to list and describe the specified KMS key, then it also works. I'm not sure what else from SOPS can cause the above error. Please help shed some light. Thanks!

[gnommer]

Similar issue on my side as I have used kms in similar fashion. No solution yet. Would ask the maintainer to resolve this soon

[ilyagorban-codefresh]

I have the same issue, same manner of assuming roles and getting the same error message. No solution...

[markussiebert]

Struggling with the same problem. Profile should be nothing written to the sops file, as it's highly individual and i suspect there is any use case where you have the same profile in ci and local environment.