Skip to content

Conversation

aknysh
Copy link
Member

@aknysh aknysh commented Jun 23, 2025

what

  • Add Terraform/OpenTofu Multi-Component Commands (Filtered/Bulk Operations)
    • atmos terraform plan/apply/deploy --all
    • atmos terraform plan/apply/deploy --all --stack <stack>
    • atmos terraform plan/apply/deploy --affected
    • atmos terraform plan/apply/deploy --affected --stack <stack>
    • atmos terraform plan/apply/deploy --affected --include-dependents
    • atmos terraform plan/apply/deploy --affected --stack <stack> --include-dependents
    • atmos terraform plan/apply/deploy --components <component1>,<component2>
    • atmos terraform plan/apply/deploy --components <component1>,<component2> --stack <stack>
    • atmos terraform plan/apply/deploy --query <yq-expression>
    • atmos terraform plan/apply/deploy --query <yq-expression> --stack <stack>
  • Add unit tests
  • Update docs

why

  • Allow Terraform/Opentofu operations (plan, apply, deploy) on a set of components in all stacks or a specific stack depending on stack, set of components, query, or filters

description

Atmos Terraform/OpenTofu commands fall into two categories:

  • Single-Component: Run Terraform for one component at a time

  • Multi-Component (Filtered/Bulk): Run Terraform across multiple components using stack names, selectors, or change detection

Single-Component Commands Usage

# Execute `terraform <command>` on a `component` in a `stack`
atmos terraform <command> <component> -s <stack> [options]
atmos terraform <command> <component> --stack <stack> [options]

Multi-Component Commands (Bulk Operations) Usage

# Execute `terraform <command>` on all components in the stack `prod`
atmos terraform <command> --stack prod

# Execute `terraform <command>` on components `component-1` and `component-2` in all stacks
atmos terraform <command> --components component-1,component-2

# Execute `terraform <command>` on components `component-1` and `component-2` in the stack `prod`
atmos terraform <command> --stack prod --components component-1,component-2

# Execute `terraform <command>` on all components in all stacks
atmos terraform <command> --all

# Execute `terraform <command>` on all components in the stack `prod`
atmos terraform <command> --all --stack prod

# Execute `terraform <command>` on all the directly affected components in all stacks in dependency order
# (if component dependencies are configured)
atmos terraform <command> --affected

# Execute `terraform <command>` on all the directly affected components in the `prod` stack in dependency order
# (if component dependencies are configured)
atmos terraform <command> --affected --stack prod

# Execute `terraform <command>` on all the directly affected components in all stacks in dependency order.
# For each directly affected component, detect the dependent components and process them in dependency order, recursively.
# Dependents are components that are indirectly affected, meaning that nothing in the current branch modifies their code
# or configs, but they are configured as dependencies of the components that are modified
atmos terraform <command> --affected --include-dependents

# Execute `terraform <command>` on all the directly affected components in the `prod` stack in dependency order.
# For each directly affected component, detect the dependent components and process them in dependency order, recursively.
atmos terraform <command> --affected --include-dependents --stack prod

# Execute `terraform <command>` on all components that have `vars.tags.team == "data"`, in all stacks
atmos terraform <command> --query '.vars.tags.team == "data"'

# Execute `terraform <command>` on all components that have `vars.tags.team == "eks"`, in the stack `prod`
atmos terraform <command> --query '.vars.tags.team == "eks"' --stack prod

# Execute `terraform <command>` on all components that have `settings.context.account_id == 12345`, in all stacks
atmos terraform <command> --query '.settings.context.account_id == 12345'

Multi-Component Commands (Bulk Operations) Examples

Let's assume that we have the following Atmos stack manifests in the prod and nonprod stacks,
with dependencies between the components:

components:
  terraform:
    vpc:
      vars:
        tags:
          # Team `network` manages the `vpc` component
          team: network
    eks/cluster:
      vars:
        tags:
          # Team `eks` manages the `eks/cluster` component
          team: eks
      settings:
        depends_on:
          # `eks/cluster` depends on the `vpc` component
          1:
            component: vpc
    eks/external-dns:
      vars:
        tags:
          # Team `eks` manages the `eks/external-dns` component
          team: eks
      settings:
        depends_on:
          # `eks/external-dns` depends on the `eks/cluster` component
          1:
            component: eks/cluster
    eks/karpenter:
      vars:
        tags:
          # Team `eks` manages the `eks/karpenter` component
          team: eks
      settings:
        depends_on:
          # `eks/karpenter` depends on the `eks/cluster` component
          1:
            component: eks/cluster
    eks/karpenter-node-pool:
      vars:
        tags:
          # Team `eks` manages the `eks/karpenter-node-pool` component
          team: eks
      settings:
        # `eks/karpenter-node-pool` depends on the `eks/cluster` and `eks/karpenter` components
        depends_on:
          1:
            component: eks/cluster
          2:
            component: eks/karpenter
    eks/istio/base:
      vars:
        tags:
          # Team `istio` manages the `eks/istio/base` component
          team: istio
      settings:
        # `eks/istio/base` depends on the `eks/cluster` component
        depends_on:
          1:
            component: eks/cluster
    eks/istio/istiod:
      vars:
        tags:
          # Team `istio` manages the `eks/istio/istiod` component
          team: istio
      settings:
        # `eks/istio/istiod` depends on the `eks/cluster` and `eks/istio/base` components
        depends_on:
          1:
            component: eks/cluster
          2:
            component: eks/istio/base
    eks/istio/test-app:
      vars:
        tags:
          # Team `istio` manages the `eks/istio/test-app` component
          team: istio
      settings:
        # `eks/istio/test-app` depends on the `eks/cluster`, `eks/istio/istiod` and `eks/istio/base` components
        depends_on:
          1:
            component: eks/cluster
          2:
            component: eks/istio/istiod
          3:
            component: eks/istio/base

Let's run the following Multi-Component commands in dry-run mode and review the output to understand what each command executes:

# Execute the `terraform apply` command on all components in all stacks

> atmos terraform apply --all --dry-run

Executing command="atmos terraform apply vpc -s nonprod"
Executing command="atmos terraform apply eks/cluster -s nonprod"
Executing command="atmos terraform apply eks/external-dns -s nonprod"
Executing command="atmos terraform apply eks/istio/base -s nonprod"
Executing command="atmos terraform apply eks/istio/istiod -s nonprod"
Executing command="atmos terraform apply eks/istio/test-app -s nonprod"
Executing command="atmos terraform apply eks/karpenter -s nonprod"
Executing command="atmos terraform apply eks/karpenter-node-pool -s nonprod"

Executing command="atmos terraform apply vpc -s prod"
Executing command="atmos terraform apply eks/cluster -s prod"
Executing command="atmos terraform apply eks/external-dns -s prod"
Executing command="atmos terraform apply eks/istio/base -s prod"
Executing command="atmos terraform apply eks/istio/istiod -s prod"
Executing command="atmos terraform apply eks/istio/test-app -s prod"
Executing command="atmos terraform apply eks/karpenter -s prod"
Executing command="atmos terraform apply eks/karpenter-node-pool -s prod"
# Execute the `terraform apply` command on all components in the `prod` stack

> atmos terraform apply --all --stack prod --dry-run

Executing command="atmos terraform apply vpc -s prod"
Executing command="atmos terraform apply eks/cluster -s prod"
Executing command="atmos terraform apply eks/external-dns -s prod"
Executing command="atmos terraform apply eks/istio/base -s prod"
Executing command="atmos terraform apply eks/istio/istiod -s prod"
Executing command="atmos terraform apply eks/istio/test-app -s prod"
Executing command="atmos terraform apply eks/karpenter -s prod"
Executing command="atmos terraform apply eks/karpenter-node-pool -s prod"
# Execute the `terraform apply` command on all components in the `prod` stack

> atmos terraform apply --stack prod --dry-run

Executing command="atmos terraform apply vpc -s prod"
Executing command="atmos terraform apply eks/cluster -s prod"
Executing command="atmos terraform apply eks/external-dns -s prod"
Executing command="atmos terraform apply eks/istio/base -s prod"
Executing command="atmos terraform apply eks/istio/istiod -s prod"
Executing command="atmos terraform apply eks/istio/test-app -s prod"
Executing command="atmos terraform apply eks/karpenter -s prod"
Executing command="atmos terraform apply eks/karpenter-node-pool -s prod"
# Execute the `terraform apply` command on the `vpc` and `eks/cluster` components
# in all stacks.

> atmos terraform apply --components vpc,eks/cluster --dry-run

Executing command="atmos terraform apply vpc -s nonprod"
Executing command="atmos terraform apply eks/cluster -s nonprod"

Executing command="atmos terraform apply vpc -s prod"
Executing command="atmos terraform apply eks/cluster -s prod"
# Execute the `terraform apply` command on the `vpc` and `eks/cluster` components
# in the `prod` stack.

> atmos terraform apply --stack prod --components vpc,eks/cluster --dry-run

Executing command="atmos terraform apply vpc -s prod"
Executing command="atmos terraform apply eks/cluster -s prod"
# Execute the `terraform apply` command on the components filtered by the query expression,
# in all stacks.

> atmos terraform apply --query '.vars.tags.team == "eks"' --dry-run

Skipping the component because the query criteria not satisfied command="atmos terraform apply vpc -s nonprod" query=".vars.tags.team == \"eks\""
Executing command="atmos terraform apply eks/cluster -s nonprod"
Executing command="atmos terraform apply eks/external-dns -s nonprod"
Skipping the component because the query criteria not satisfied command="atmos terraform apply eks/istio/base -s nonprod" query=".vars.tags.team == \"eks\""
Skipping the component because the query criteria not satisfied command="atmos terraform apply eks/istio/istiod -s nonprod" query=".vars.tags.team == \"eks\""
Skipping the component because the query criteria not satisfied command="atmos terraform apply eks/istio/test-app -s nonprod" query=".vars.tags.team == \"eks\""
Executing command="atmos terraform apply eks/karpenter -s nonprod"
Executing command="atmos terraform apply eks/karpenter-node-pool -s nonprod"

Skipping the component because the query criteria not satisfied command="atmos terraform apply vpc -s prod" query=".vars.tags.team == \"eks\""
Executing command="atmos terraform apply eks/cluster -s prod"
Executing command="atmos terraform apply eks/external-dns -s prod"
Skipping the component because the query criteria not satisfied command="atmos terraform apply eks/istio/base -s prod" query=".vars.tags.team == \"eks\""
Skipping the component because the query criteria not satisfied command="atmos terraform apply eks/istio/istiod -s prod" query=".vars.tags.team == \"eks\""
Skipping the component because the query criteria not satisfied command="atmos terraform apply eks/istio/test-app -s prod" query=".vars.tags.team == \"eks\""
Executing command="atmos terraform apply eks/karpenter -s prod"
Executing command="atmos terraform apply eks/karpenter-node-pool -s prod"
# Execute the `terraform apply` command on the components filtered by the query expression,
# in the `prod` stack.

> atmos terraform apply --query '.vars.tags.team == "eks"' --stack prod --dry-run

Skipping the component because the query criteria not satisfied command="atmos terraform apply vpc -s prod" query=".vars.tags.team == \"eks\""
Executing command="atmos terraform apply eks/cluster -s prod"
Executing command="atmos terraform apply eks/external-dns -s prod"
Skipping the component because the query criteria not satisfied command="atmos terraform apply eks/istio/base -s prod" query=".vars.tags.team == \"eks\""
Skipping the component because the query criteria not satisfied command="atmos terraform apply eks/istio/istiod -s prod" query=".vars.tags.team == \"eks\""
Skipping the component because the query criteria not satisfied command="atmos terraform apply eks/istio/test-app -s prod" query=".vars.tags.team == \"eks\""
Executing command="atmos terraform apply eks/karpenter -s prod"
Executing command="atmos terraform apply eks/karpenter-node-pool -s prod"
# Execute the `terraform apply` command on all components affected by the changes
# in the current branch, in all stacks, in dependency order.
# Assume that the components `vpc` and `eks/cluster` in all stacks are affected (e.g. just added).

> atmos terraform apply --affected --dry-run

Executing command="atmos terraform apply vpc -s nonprod"
Executing command="atmos terraform apply eks/cluster -s nonprod"

Executing command="atmos terraform apply vpc -s prod"
Executing command="atmos terraform apply eks/cluster -s prod"
# Execute the `terraform apply` command on all components affected by the changes
# in the current branch, in the `prod` stack, in dependency order.
# Assume that the components `vpc` and `eks/cluster` in the `prod` stack are affected (e.g. just added).

> atmos terraform apply --affected --stack prod --dry-run

Executing command="atmos terraform apply vpc -s prod"
Executing command="atmos terraform apply eks/cluster -s prod"
# Execute the `terraform apply` command on all the components affected by the changes
# in the current branch, in all stacks.
# For each directly affected component, detect the dependent components and process
# them in dependency order, recursively.
# Dependents are components that are indirectly affected, meaning that nothing in the
# current branch modifies their code or configs, but they are configured as
# dependencies of the components that are modified.

> atmos terraform apply --affected --include-dependents --dry-run

Executing command="atmos terraform apply vpc -s nonprod"
Executing command="atmos terraform apply eks/cluster -s nonprod" dependency of component=vpc in stack=nonprod
Executing command="atmos terraform apply eks/karpenter -s nonprod" dependency of component=eks/cluster in stack=nonprod
Executing command="atmos terraform apply eks/karpenter-node-pool -s nonprod" dependency of component=eks/karpenter in stack=nonprod
Executing command="atmos terraform apply eks/external-dns -s nonprod" dependency of component=eks/cluster in stack=nonprod
Executing command="atmos terraform apply eks/istio/base -s nonprod" dependency of component=eks/cluster in stack=nonprod
Executing command="atmos terraform apply eks/istio/istiod -s nonprod" dependency of component=eks/istio/base in stack=nonprod
Executing command="atmos terraform apply eks/istio/test-app -s nonprod" dependency of component=eks/istio/istiod in stack=nonprod

Executing command="atmos terraform apply vpc -s prod"
Executing command="atmos terraform apply eks/cluster -s prod" dependency of component=vpc in stack=prod
Executing command="atmos terraform apply eks/external-dns -s prod" dependency of component=eks/cluster in stack=prod
Executing command="atmos terraform apply eks/istio/base -s prod" dependency of component=eks/cluster in stack=prod
Executing command="atmos terraform apply eks/istio/istiod -s prod" dependency of component=eks/istio/base in stack=prod
Executing command="atmos terraform apply eks/istio/test-app -s prod" dependency of component=eks/istio/istiod in stack=prod
Executing command="atmos terraform apply eks/karpenter -s prod" dependency of component=eks/cluster in stack=prod
Executing command="atmos terraform apply eks/karpenter-node-pool -s prod" dependency of component=eks/karpenter in stack=prod
# Execute the `terraform apply` command on all the components affected by the changes
# in the current branch, in the `prod` stack.
# For each directly affected component, detect the dependent components and process
# them in dependency order, recursively.
# Dependents are components that are indirectly affected, meaning that nothing in the
# current branch modifies their code or configs, but they are configured as
# dependencies of the components that are modified.

> atmos terraform apply --affected --stack prod --include-dependents --dry-run

Executing command="atmos terraform apply vpc -s prod"
Executing command="atmos terraform apply eks/cluster -s prod" dependency of component=vpc in stack=prod
Executing command="atmos terraform apply eks/external-dns -s prod" dependency of component=eks/cluster in stack=prod
Executing command="atmos terraform apply eks/istio/base -s prod" dependency of component=eks/cluster in stack=prod
Executing command="atmos terraform apply eks/istio/istiod -s prod" dependency of component=eks/istio/base in stack=prod
Executing command="atmos terraform apply eks/istio/test-app -s prod" dependency of component=eks/istio/istiod in stack=prod
Executing command="atmos terraform apply eks/karpenter -s prod" dependency of component=eks/cluster in stack=prod
Executing command="atmos terraform apply eks/karpenter-node-pool -s prod" dependency of component=eks/karpenter in stack=prod

Summary by CodeRabbit

Summary by CodeRabbit

  • New Features

    • Added --affected and --all flags to Terraform commands for selective or full-stack operations.
    • Introduced filtering flags: --components, --query, and --include-dependents for precise targeting of components and their dependencies.
    • Added Git-related flags (--repo-path, --ref, --sha, --clone-target-ref, --ssh-key, --ssh-key-password) to identify affected components based on code changes.
    • Implemented --dry-run mode to simulate Terraform commands without applying changes.
    • Enhanced Terraform execution to process affected components in dependency order, including recursive handling of dependents.
    • Added detailed CLI argument parsing and validation for the describe affected command with deprecation handling for --verbose.
    • Extended CLI commands with new persistent flags supporting advanced Git workflows and component filtering.
    • Introduced a new shell command for native Terraform usage within component/stack context.
    • Added new generate subcommands for backend config and varfile generation.
    • Enhanced clean, workspace, import, plan, deploy, and apply commands with new flags and behaviors.
    • Added utility functions to determine affected components by comparing Git commits, supporting cloning, checkout, and local repo path modes.
    • Added recursive dependency processing to include dependents of affected components automatically.
    • Introduced query-based filtering for multi-component Terraform commands.
    • Added support for --verbose flag deprecation with warning and log level adjustment.
  • Bug Fixes

    • Centralized and improved error handling for CLI argument parsing and flag validation.
    • Fixed error handling during flag processing to ensure immediate termination on errors.
    • Resolved concurrency issues on Windows by explicit temporary directory removal after Git operations.
  • Documentation

    • Expanded CLI help and usage documentation with comprehensive examples for single and multi-component Terraform operations.
    • Updated examples to demonstrate bulk operations, dependency-aware execution, and filtering by queries or affected components.
    • Clarified usage of Terraform/OpenTofu commands and flags within Atmos.
    • Updated CLI help texts to include new flags and detailed descriptions.
    • Corrected documentation text for clarity and consistency.
  • Chores

    • Upgraded multiple dependencies and updated default Atmos version in Dockerfile and CI workflows.
  • Tests

    • Added new tests covering affected components execution with dependents and query-based component filtering.
    • Introduced fixtures modeling complex component dependency graphs for multi-component Terraform workflows.
    • Improved test readability and adjusted mocks to align with updated method signatures.

aknysh added 30 commits May 28, 2025 10:56
…affected-all

# Conflicts:
#	internal/exec/describe_affected.go
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c51f42c and 8906417.

📒 Files selected for processing (3)
  • cmd/errors_test.go (1 hunks)
  • internal/exec/atlantis_generate_repo_config.go (0 hunks)
  • internal/exec/atlantis_generate_repo_config_test.go (1 hunks)
💤 Files with no reviewable changes (1)
  • internal/exec/atlantis_generate_repo_config.go
🧰 Additional context used
📓 Path-based instructions (1)
`cmd/*.go`: Implement each Cobra command in a separate file under the cmd/ directory.

cmd/*.go: Implement each Cobra command in a separate file under the cmd/ directory.

  • cmd/errors_test.go
⏰ Context from checks skipped due to timeout of 90000ms (6)
  • GitHub Check: website-deploy-preview
  • GitHub Check: Lint (golangci)
  • GitHub Check: Analyze (go)
  • GitHub Check: Build (ubuntu-latest, linux)
  • GitHub Check: Build (windows-latest, windows)
  • GitHub Check: Summary

coderabbitai[bot]
coderabbitai bot previously approved these changes Jun 24, 2025
coderabbitai[bot]
coderabbitai bot previously approved these changes Jun 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
minor New features that do not break anything size/xl Extra large size PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants