EKS cluster authentication KO on TerraformCloud

[pimousse1099]

Hi !

We are trying to deploy some applications on a given EKS cluster using your helmfile provider through Terraform Cloud 👍

At this time, we did not succeed ...

---

Here is the tf code :

resource "local_file" "kubeconfig" {
  filename          = "${path.module}/.kube/config"
  file_permission   = "600"
  # kubeconfig that comes from another tf workspace
  sensitive_content = data.terraform_remote_state.back_infra.outputs.eks_cluster_kubeconfig
}

# deploy all backend apps
resource "helmfile_release_set" "example_app" {
  version           = "0.142.0"
  helm_version      = "3.7.1"
  helm_diff_version = "v3.1.3"

  # load helmfile where helm releases are defined
  content                   = file("helmfile/helmfile.yaml")
  working_directory  = "${path.module}/helmfile"
  kubeconfig             = local_file.kubeconfig.filename
  # ask helmfile to deploy the app
  selector              = {
    appName = "exampleApp" # corresponds to -l appName=exampleApp
  }
}

Here is the kubeconfig we are passing :

apiVersion: v1
preferences: {}
kind: Config

clusters:
- cluster:
  server: <hidden>
  certificate-authority-data: <hidden>
  name: <hidden>

contexts:
- context:
  cluster: eks_example
  user: eks_example
  name: eks_example

current-context: eks_example

users:
- name: eks_example
  user:
  exec:
  apiVersion: client.authentication.k8s.io/v1alpha1
  command: aws-iam-authenticator
  args:
  - \"token\"
  - \"-i\"
  - \"example\"
  "

---

Terraform Cloud returns an error saying that aws-iam-authenticator binary is missing

We tried to add aws-iam-authenticator using a null-resource like this without any success :

resource "null_resource" "install_aws_iam_authenticator" {
  # always recreate the config on the remote machine
  triggers = {
    always_run = timestamp()
  }

  provisioner "local-exec" {
    command = <<-INSTALL_AWS_IAM_AUTH
      curl -o aws-iam-authenticator https://amazon-eks.s3.us-west-2.amazonaws.com/1.21.2/2021-07-05/bin/linux/amd64/aws-iam-authenticator
      chmod +x ./aws-iam-authenticator
      export PATH=$PATH:${path.module} # that one does not actually add the module path to the PATH ...
      echo $PATH
      mv aws-iam-authenticator /usr/local/bin # that one fails ...
      aws-iam-authenticator help
    INSTALL_AWS_IAM_AUTH
  }
}

---

We also tried to generate a kubeconfig using aws eks update-kubeconfig so that the kubeconfig uses aws CLI to perform authentication ... but the helmfile_release_set ressource keep returning an error saying that the aws profile (xxxx) is not present in the config file ... doing a cat on it show that the profile is present 😬 🤦

---

Do you have any idea on how to perform AWS EKS authentication through TerraformCloud ?

To me, the whole issue resides in the fact that helmfile provider does not asks for any kubernetes conf as the helm provider does 🤷‍♂️

provider "helm" {
  kubernetes {

  }
}

Thanks a lot for your help !
Let me know if I can help on anything 👍