Skip to content

Apply command not reading credentials from AWS Secrets manager without placeholder in cluster config #205

@srini-hv

Description

@srini-hv

Looks like topicctl apply is not trying to read SASL credentials from AWS Secrets manager while topicctl check does.

$ echo $TOPICCTL_SASL_SECRETS_MANAGER_ARN
arn:aws:secretsmanager:<region>:<account-id>:secret:<secret-id>

$ topicctl --version
topicctl version v1.18.0 (ref:dev)

Steps to reproduce:

$ cat /tmp/topic.yaml
---
meta:
  name: test_topic                     
  cluster: test_cluster       
  environment: dev
  region: some-region           
spec:                  
  partitions: 12
  replicationFactor: 3
  placement:          
    strategy: any


$ cat /tmp/cluster.yaml 
---
meta:
  name: test_cluster
  environment: dev
  region: some-region
spec:
  bootstrapAddrs:
    - <b1-endpoint>:9096
    - <b2-endpoint>:9096
    - <b3-endpoint>:9096
  tls:
    enabled: true
  sasl:
    enabled: true
    mechanism: SCRAM-SHA-512

$ topicctl apply /tmp/topic.yaml --skip-confirm --cluster-config /tmp/cluster.yaml --expand-env --debug

{{test_topic test_cluster some-region dev  map[] []} {12 3 0 map[] {any  [] []} <nil>}}
[2024-06-24 13:04:08] DEBUG No ZK addresses provided, using broker admin client
[2024-06-24 13:04:08] DEBUG Connecting to cluster on address <b1-endpoint>:9096 with TLS enabled=true, SASL enabled=true
[2024-06-24 13:04:08] DEBUG Getting supported API versions
[2024-06-24 13:04:09] ERROR 58: SASL Authentication Failed: SASL Authentication failed: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512

Output from topicctl check:

$ topicctl get topics --cluster-config /tmp/cluster.yaml --expand-env --debug

time="2024-06-24 12:59:50" level=debug msg="No ZK addresses provided, using broker admin client"
time="2024-06-24 12:59:50" level=debug msg="Setting SASL SecretsManagerArn from override value"
time="2024-06-24 12:59:50" level=debug msg="Fetching credentials from Secrets Manager for secret: arn:aws:secretsmanager:<region>:<account-id>:secret:<secret-id>"
time="2024-06-24 12:59:50" level=debug msg="Fetching secret value for secret name: <secret-id>"
time="2024-06-24 12:59:50" level=debug msg="Connecting to cluster on address <b1-endpoint>:9096 with TLS enabled=true, SASL enabled=true"
time="2024-06-24 12:59:50" level=debug msg="Getting supported API versions"

You can clearly see that topicctl check tries to read secret from Secrets manager.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions