Skip to content

Commit 49675db

Browse files
committed
Add CPU_SIZE and MEMORY_SIZE support to Squirro workflow
- Updated build-and-push-to-ecr.yml.template to handle CPU/Memory variables - Added support for GitHub Variables override (same pattern as main workflow) - Updated documentation in .squirro/README.md - Updated setup-for-squirro.sh.template instructions This ensures Squirro deployments can also benefit from configurable resource allocation without hardcoding values.
1 parent 48aa1ff commit 49675db

File tree

4 files changed

+26
-8
lines changed

4 files changed

+26
-8
lines changed

.squirro/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,8 @@ These are **optional overrides** for config.env values:
7979
- `AWS_ACCOUNT_ID` - Override the AWS account ID from config.env
8080
- `AWS_REGION` - Override the AWS region from config.env
8181
- `ECR_REPOSITORY` - Override the ECR repository name from config.env
82+
- `CPU_SIZE` - Override CPU allocation (e.g., "1 vCPU", "4 vCPU")
83+
- `MEMORY_SIZE` - Override memory allocation (e.g., "2 GB", "8 GB")
8284

8385
### GitHub Secrets (Repository Settings → Secrets and variables → Secrets)
8486
These are **sensitive** values that must be kept secret:

.squirro/setup-for-squirro.sh.template

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,8 @@ echo "2. Configure GitHub Variables (optional overrides):"
7878
echo " - AWS_ACCOUNT_ID (override config.env value)"
7979
echo " - AWS_REGION (override config.env value)"
8080
echo " - ECR_REPOSITORY (override config.env value)"
81+
echo " - CPU_SIZE (override config.env value, e.g., '4 vCPU')"
82+
echo " - MEMORY_SIZE (override config.env value, e.g., '8 GB')"
8183
echo ""
8284
echo "3. Configure GitHub Secrets (required):"
8385
echo " - AWS_ACCESS_KEY_ID (for ECR push access)"

.squirro/workflows/build-and-push-to-ecr.yml.template

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,8 @@ jobs:
4141
[ -n "${{ vars.AWS_REGION }}" ] && AWS_REGION="${{ vars.AWS_REGION }}"
4242
[ -n "${{ vars.AWS_ACCOUNT_ID }}" ] && AWS_ACCOUNT_ID="${{ vars.AWS_ACCOUNT_ID }}"
4343
[ -n "${{ vars.ECR_REPOSITORY }}" ] && ECR_REPOSITORY="${{ vars.ECR_REPOSITORY }}"
44+
[ -n "${{ vars.CPU_SIZE }}" ] && CPU_SIZE="${{ vars.CPU_SIZE }}"
45+
[ -n "${{ vars.MEMORY_SIZE }}" ] && MEMORY_SIZE="${{ vars.MEMORY_SIZE }}"
4446

4547
# Validate required values exist (from either source)
4648
if [ -z "$AWS_ACCOUNT_ID" ]; then
@@ -107,17 +109,29 @@ jobs:
107109
export ECR_REPOSITORY="${{ vars.ECR_REPOSITORY }}"
108110
echo "📝 ECR_REPOSITORY overridden by GitHub Variable: $ECR_REPOSITORY"
109111
fi
112+
if [ -n "${{ vars.CPU_SIZE }}" ]; then
113+
export CPU_SIZE="${{ vars.CPU_SIZE }}"
114+
echo "📝 CPU_SIZE overridden by GitHub Variable: $CPU_SIZE"
115+
fi
116+
if [ -n "${{ vars.MEMORY_SIZE }}" ]; then
117+
export MEMORY_SIZE="${{ vars.MEMORY_SIZE }}"
118+
echo "📝 MEMORY_SIZE overridden by GitHub Variable: $MEMORY_SIZE"
119+
fi
110120

111121
# Export to GITHUB_ENV for subsequent steps
112122
echo "AWS_REGION=$AWS_REGION" >> $GITHUB_ENV
113123
echo "AWS_ACCOUNT_ID=$AWS_ACCOUNT_ID" >> $GITHUB_ENV
114124
echo "ECR_REPOSITORY=$ECR_REPOSITORY" >> $GITHUB_ENV
115125
echo "ECR_REGISTRY=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com" >> $GITHUB_ENV
126+
echo "CPU_SIZE=$CPU_SIZE" >> $GITHUB_ENV
127+
echo "MEMORY_SIZE=$MEMORY_SIZE" >> $GITHUB_ENV
116128

117129
echo "📊 Final configuration:"
118130
echo " AWS_REGION=$AWS_REGION"
119131
echo " AWS_ACCOUNT_ID=$AWS_ACCOUNT_ID"
120132
echo " ECR_REPOSITORY=$ECR_REPOSITORY"
133+
echo " CPU_SIZE=$CPU_SIZE"
134+
echo " MEMORY_SIZE=$MEMORY_SIZE"
121135

122136
- name: Configure AWS credentials
123137
uses: aws-actions/configure-aws-credentials@v4

README.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ graph TB
6565
ValidateConfig["just validate-config<br/>📝 YAML validation"]
6666
CiTests["just ci-tests-with-data<br/>🔍 CI tests + data"]
6767
FullPipeline["just full-pipeline<br/>🏗️ Complete dev pipeline"]
68-
TestIntegration["just test-integration<br/>🔧 Integration tests"]
68+
TestTools["just test-tools<br/>🔧 Tool tests"]
6969
PrepareBuild["just prepare-build<br/>📦 Data preparation"]
7070
TestData["just test-data<br/>🧪 Level 1: dbt tests"]
7171
TestEvals["just test-evals<br/>🤖 Level 3: LLM evals"]
@@ -90,7 +90,7 @@ graph TB
9090
%% Workflow Relationships
9191
Deploy -->|"1. Validation"| CiTests
9292
Deploy -->|"Fallback"| ValidateConfig
93-
Deploy -->|"2. Post-deployment"| TestIntegration
93+
Deploy -->|"2. Post-deployment"| TestTools
9494
9595
Test -->|"PR Testing"| FullPipeline
9696
Test -->|"Fallback"| ValidateConfig
@@ -102,9 +102,9 @@ graph TB
102102
103103
FullPipeline --> PrepareBuild
104104
FullPipeline --> TestData
105-
FullPipeline --> TestIntegration
105+
FullPipeline --> TestTools
106106
107-
TestIntegration -->|"Uses"| Tools
107+
TestTools -->|"Uses"| Tools
108108
TestData -->|"Tests"| Models
109109
110110
%% Docker Build Process
@@ -120,12 +120,12 @@ graph TB
120120
%% 3-Tiered Testing
121121
subgraph "🎯 3-Tiered Testing"
122122
Level1["Level 1: Data Quality<br/>🧪 dbt schema tests<br/>💰 Free"]
123-
Level2["Level 2: Integration<br/>🔧 MXCP tools tests<br/>💰 Free"]
123+
Level2["Level 2: Integration<br/>🔧 MXCP tools & API tests<br/>💰 Free"]
124124
Level3["Level 3: LLM Evaluation<br/>🤖 AI behavior tests<br/>💰 Costs Apply"]
125125
end
126126
127127
TestData -.->|"Implements"| Level1
128-
TestIntegration -.->|"Implements"| Level2
128+
TestTools -.->|"Implements"| Level2
129129
TestEvals -.->|"Implements"| Level3
130130
131131
%% Styling
@@ -136,7 +136,7 @@ graph TB
136136
classDef testing fill:#fce4ec,stroke:#880e4f,stroke-width:2px
137137
138138
class Deploy,Test,Release workflow
139-
class ValidateConfig,CiTests,FullPipeline,TestIntegration,PrepareBuild,TestData,TestEvals justfile
139+
class ValidateConfig,CiTests,FullPipeline,TestTools,PrepareBuild,TestData,TestEvals justfile
140140
class Dockerfile,ConfigEnv,MxcpSite,UserConfig,StartSh deployment
141141
class Scripts,Tools,Models project
142142
class Level1,Level2,Level3 testing
@@ -688,7 +688,7 @@ just dev-full # Download data + build + test all 3 levels
688688
689689
# Individual testing levels
690690
just test-data # Level 1: dbt schema tests
691-
just test-integration # Level 2: MXCP tools tests
691+
just test-tools # Level 2: MXCP tools tests
692692
just test-evals # Level 3: LLM evaluation tests (requires OPENAI_API_KEY)
693693
694694
# CI/CD pipeline

0 commit comments

Comments
 (0)