feat: Edit dag definitions before retry

[yottahmd]

Problem

When a DAG run fails due to incorrect step parameters, users must retry with the same broken parameters or create a new DAG run from scratch.

Solution

Allow users to export DAG configuration as JSON, edit it, and import it back before retry.

New Commands

dagu run export

Export DAG configuration from a specific run.

dagu export <dag name> <dag-run-id>                    # Output to stdout
dagu export <dag name> <dag-run-id> > edited.json      # Redirect to file
dagu export <dag name> <dag-run-id> | jq '.steps[0]'  # Pipe to tools

Behavior:

• Exports current DAG configuration (edited version if exists, otherwise original)
• Output is formatted JSON
• Exit code 1 if run ID not found

dagu run import

Import edited DAG configuration.

dagu import <dag name> <dag-run-id> edited.json        # From file
dagu import <dag name> <dag-run-id> -                  # From stdin
cat edited.json | dagu import <dag-run-id>  # From pipe

Behavior:

• Validates DAG structure before importing
• Backs up current dag.json to dag.edits/.json
• Overwrites dag.json with new content
• Creates dag.edits/ directory on first import
• Exit code 1 if validation fails

dagu run show-edits

Show edit history for a DAG run.

dagu run show-edits <dag name> <dag-run-id>                # List all edits
dagu run show-edits <dag name> <dag-run-id> --diff         # Show diffs

Output example:

Edit history for run "abc123":
  1. 2024-01-15 14:30:22 (original)
  2. 2024-01-15 14:55:12 
  3. 2024-01-15 15:01:33 (current)

dagu run diff

Compare current configuration with original or specific version.

dagu run diff <dag name> <dag-run-id>                      # Current vs original
dagu run diff <dag name> <dag-run-id> --version 2          # Current vs version 2
dagu run diff <dag name> <dag-run-id> --from 1 --to 3      # Version 1 vs 3

Output:

• Unified diff format showing changes
• Highlights modified steps, commands, environment variables

Storage Structure

attempt_<timestamp>_<attemptID>/
├── dag.json              # Current DAG (modified by imports)
└── dag.edits/            # Backup history (created on first import)
    ├── 20240115_143022.json  # First backup = original
    ├── 20240115_145512.json
    └── 20240115_150133.json

Example Workflow

# 1. DAG fails
$ dagu status my-pipeline
Step "fetch-data" failed: API endpoint not found

# 2. Export configuration
$ dagu run export dag-1 failed-run-123 > fix.json

# 3. Edit (fix API endpoint)
$ jq '.steps[] | select(.name=="fetch-data") | .command = "curl https://api.v2.example.com"' fix.json > fixed.json

# 4. Import fixed configuration
$ dagu run import dag-1 failed-run-123 fixed.json
Configuration imported successfully. Backup saved to dag.edits/20240115_143022.json

# 5. Verify changes
$ dagu diff dag-1 failed-run-123
--- original
+++ current
@@ -10,7 +10,7 @@
   steps:
     - name: fetch-data
-      command: curl https://api.example.com
+      command: curl https://api.v2.example.com

# 6. Retry with fixed configuration
$ dagu retry dag-1 failed-run-123

Implementation Notes

• Retry command needs no changes - it already uses dag.json
• Import validates DAG structure to prevent corruption
• Backups are timestamped for easy tracking
• Original DAG is never modified (first backup)