Support Platform CLI (#240)

Switch to the Platform CLI[1] for install/update. 

- Add `copier.yml` config file
- Move template files into `template/`
- And tag the app services in Compose files with `{{app_name}}`
- Move `.hadolint.yaml` and `.dockleconfig` from project root to
  `{{app_name}}/`. Upstream scanning CI has been updated to apply app-specific
  configs when running for those apps.
- Remove old install/update scripts
- Update from unsupported GitHub actions
- Update template CD to use platform-cli

[1] https://github.com/navapbc/platform-cli

https://github.com/navapbc/template-application-nextjs

Author: doshitan

.github/workflows/template-only-cd.yml renamed to .github/cd.yml

@@ -7,34 +7,43 @@ on:
   workflow_dispatch:
 
 # Only allow one workflow at a time to prevent race conditions when pushing changes to the project repo
-concurrency: template-only-cd
+concurrency: cd
 
 jobs:
   deploy:
     name: Deploy
     runs-on: ubuntu-latest
     steps:
       - name: Checkout template repo
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         with:
           path: template-application-flask
+
       - name: Checkout project repo
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         with:
           path: project-repo
           repository: navapbc/platform-test-flask
           token: ${{ secrets.PLATFORM_BOT_GITHUB_TOKEN }}
 
-      - name: Update application template
-        working-directory: project-repo
-        run: ../template-application-flask/template-only-bin/update-template.sh
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.13'
 
-      - name: Push changes to project repo
+      - name: Install nava-platform CLI
+        run: pipx install --python "$(which python)" git+https://github.com/navapbc/platform-cli
+
+      - name: Configure git
         working-directory: project-repo
         run: |
           git config user.name nava-platform-bot
           git config user.email [email protected]
-          git add --all
-          # Commit changes (if no changes then no-op)
-          git diff-index --quiet HEAD || git commit -m "Template application deploy #${{ github.run_id }}"
-          git push
+
+      - name: Update application template
+        working-directory: project-repo
+        run: nava-platform app update --template-uri ${{ github.server_url }}/${{ github.repository }} --version HEAD . app
+
+      - name: Push changes to project repo
+        working-directory: project-repo
+        run: git push

README.md

@@ -19,23 +19,20 @@ The template application is intended to work with the infrastructure from [templ
 
 To get started using the template application on your project:
 
-1. Run the [download and install script](./template-only-bin/download-and-install-template.sh) in your project's root directory.
-
-    ```bash
-    curl https://raw.githubusercontent.com/navapbc/template-application-flask/main/template-only-bin/download-and-install-template.sh | bash -s
+1. [Install the nava-platform tool](https://github.com/navapbc/platform-cli).
+2. Install template by running in your project's root:
+    ```sh
+    nava-platform app install --template-uri https://github.com/navapbc/template-application-flask . <APP_NAME>
     ```
-
-    This script will:
-
-    1. Clone the template repository
-    2. Copy the template files into your project directory
-    3. Remove any files specific to the template repository.
-2. Optional, if using the Platform infra template: [Follow the steps in the `template-infra` README](https://github.com/navapbc/template-infra#installation) to set up the various pieces of your infrastructure.
+3. Follow the steps in `/docs/<APP_NAME>/getting-started.md` to set up the application locally.
+4. Optional, if using the Platform infrastructure template: [Follow the steps in the `template-infra` README](https://github.com/navapbc/template-infra#installation) to set up the various pieces of your infrastructure.
 
 ## Note on memory usage
 
-If you are using [template-infra](https://github.com/navapbc/template-infra), you may want to increase the [default memory](https://github.com/navapbc/template-infra/blob/main/infra/modules/service/variables.tf#L33) allocated to the ECS service to 2048 Mb (2 Gb) to avoid the gunicorn workers running out of memory. This is because the application is currently [configured to create multiple workers](https://github.com/navapbc/template-application-flask/blob/main/app/gunicorn.conf.py#L24) based on the number of virtual CPUs available, which can take up more memory.
-
-## Getting started
-
-Now you're ready to [get started](/docs/app/getting-started.md).
+If you are using [template-infra](https://github.com/navapbc/template-infra),
+you may want to increase the [default
+memory](https://github.com/navapbc/template-infra/blob/main/infra/modules/service/variables.tf#L33)
+allocated to the ECS service to 2048 Mb (2 Gb) to avoid the gunicorn workers
+running out of memory. This is because the application is currently configured
+to create multiple workers based on the number of virtual CPUs available, which
+can take up more memory (see `/<APP_NAME>/gunicorn.conf.py`).

copier.yml

@@ -0,0 +1,29 @@
+#
+# App vars
+#
+app_name:
+  type: str
+  help: The name of the app
+  validator: >-
+    {% if not (app_name | regex_search('^[a-z0-9\-_]+$')) %}
+    The app name can not be empty and should only contain lower case letters, digits, dashes, and underscores.
+    {% endif %}
+
+app_local_port:
+  type: int
+  help: "The port to be used in local development of '{{ app_name }}'"
+  default: 3000
+
+_envops:
+  trim_blocks: true
+  lstrip_blocks: true
+
+# ideally we could just:
+#
+# _answers_file: .template-application-flask/{{app_name}}.yml
+#
+# but alas, no:
+#
+# https://github.com/copier-org/copier/issues/1868
+
+_subdirectory: template

docs/app/api-details.md

@@ -4,9 +4,9 @@ name: Update OpenAPI Docs
 on:
   pull_request:
     paths:
-      - app/**
+      - {{app_name}}/**
       - Makefile
-      - .github/workflows/ci-openapi.yml
+      - .github/workflows/ci-{{app_name}}-openapi.yml
 
 defaults:
   run:
@@ -16,18 +16,18 @@ defaults:
 # If new commits are pushed to the branch, cancel in progress runs and start
 # a new one.
 concurrency:
-  group: ${{ github.head_ref }}
+  group: ${{'{{'}} github.head_ref {{'}}'}}
   cancel-in-progress: true
 
 
 jobs:
   update-openapi-docs:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
         with:
           # Checkout the feature branch associated with the pull request
-          ref: ${{ github.head_ref }}
+          ref: ${{'{{'}} github.head_ref {{'}}'}}
 
       - name: Update OpenAPI spec
         run: make openapi-spec

template-only-bin/download-and-install-template.sh

@@ -1,20 +1,20 @@
-name: CI - App
+name: CI - {{app_name}}
 
 on:
   push:
     branches:
       - main
     paths:
-      - app/**
-      - .github/workflows/ci-app.yml
+      - {{app_name}}/**
+      - .github/workflows/ci-{{app_name}}.yml
   pull_request:
     paths:
-      - app/**
-      - .github/workflows/ci-app.yml
+      - {{app_name}}/**
+      - .github/workflows/ci-{{app_name}}.yml
 
 defaults:
   run:
-    working-directory: ./app
+    working-directory: ./{{app_name}}
 
 jobs:
   # As an enhancement, it is possible to share the built docker image and share
@@ -24,7 +24,7 @@ jobs:
     name: Lint
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Run format check
         run: make format-check
@@ -35,15 +35,15 @@ jobs:
     name: Security scan
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Run security linting
         run: make lint-security
   test:
     name: Test
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Start tests
         run: |

template-only-bin/install-template.sh

@@ -0,0 +1,2 @@
+# Changes here will be overwritten by Copier
+{{ _copier_answers|to_nice_yaml -}}

template-only-bin/update-template.sh

@@ -18,6 +18,6 @@ With the switch from Connexion to APIFlask (see [Connection replacement ADR](./0
 
 We chose to keep the openapi.yml file in source control because we want changes to the API to be called out explicitly so that developers do not accidentally make backwards-incompatible changes to the API as part of a code change. This is particularly important since the API spec is now implicit as the OpenAPI specification is automatically generated from the code.
 
-We chose to keep the openapi.yml file in sync with the API application automatically using a [CI workflow that generates the OpenAPI and pushes and changes to the PR branch](../../.github/workflows/ci-openapi.yml). This reduces the amount of manual work required by the engineer compared to a CI check that only checks for diffs but does not make the change. That said, we don't feel strongly about this decision so are open to changes in the future.
+We chose to keep the openapi.yml file in sync with the API application automatically using a CI workflow that generates the OpenAPI and pushes and changes to the PR branch (`.github/workflows/ci-<APP_NAME>-openapi.yml`). This reduces the amount of manual work required by the engineer compared to a CI check that only checks for diffs but does not make the change. That said, we don't feel strongly about this decision so are open to changes in the future.
 
 To minimize developer confusion, we chose to rename the `openapi.yml` file to `openapi.generated.yml` to clearly indicate that it is a generated file and not something that the developer should manually adjust.

.github/workflows/ci-openapi.yml renamed to template/.github/workflows/ci-{{app_name}}-openapi.yml.jinja

@@ -46,7 +46,7 @@ root
 
 `make clean-volumes` will spin down the docker containers + delete the volumes. This can be useful to reset your DB, or fix any bad states your local environment may have gotten into.
 
-See the [Makefile](/app/Makefile) for a full list of commands you can run.
+See the [Makefile](/{{ app_name }}/Makefile) for a full list of commands you can run.
 
 ## Docker and Native Development
 
@@ -88,18 +88,18 @@ export DB_HOST=localhost
 And then running `direnv allow .` in the /app folder. You should see something like:
 ```shell
 ➜  template-application-flask git:(main) ✗ cd app
-direnv: loading ~/workspace/template-application-flask/app/.envrc
+direnv: loading ~/workspace/template-application-flask/{{ app_name }}/.envrc
 direnv: export +API_AUTH_TOKEN +AWS_ACCESS_KEY_ID +AWS_DEFAULT_REGION +AWS_SECRET_ACCESS_KEY +DB_HOST +DB_NAME +DB_PASSWORD +DB_SCHEMA +DB_SSL_MODE +DB_USER +ENVIRONMENT +FLASK_APP +HIDE_SQL_PARAMETER_LOGS +LOG_ENABLE_AUDIT +LOG_FORMAT +PORT +PYTHONPATH
 ```
 
 ## Environment Variables
 
 Most configuration options are managed by environment variables.
 
-Environment variables for local development are stored in the [local.env](/app/local.env) file. This file is automatically loaded when running. If running within Docker, this file is specified as an `env_file` in the [docker-compose](/app/docker-compose.yml) file, and loaded [by a script](/app/src/util/local.py) automatically when running unit tests (see running natively above for other cases).
+Environment variables for local development are stored in the [local.env](/{{ app_name }}/local.env) file. This file is automatically loaded when running. If running within Docker, this file is specified as an `env_file` in the [docker-compose](/{{ app_name }}/docker-compose.yml) file, and loaded [by a script](/{{ app_name }}/src/util/local.py) automatically when running unit tests (see running natively above for other cases).
 To override environment variables for locally, create an `override.env` file in the same directory as local.env. The docker-compose file [optionally checks this file](https://docs.docker.com/reference/compose-file/services/#required) as well.
 
-Any environment variables specified directly in the [docker-compose](/app/docker-compose.yml) file will take precedent over those specified in the [local.env](/app/local.env) file.
+Any environment variables specified directly in the [docker-compose](/{{ app_name }}/docker-compose.yml) file will take precedent over those specified in the [local.env](/{{ app_name }}/local.env) file.
 
 ## Authentication
 
@@ -117,7 +117,7 @@ The API can be run in debug mode that allows for remote attach debugging (curren
 - First create a file `./vscode/launch.json` - as shown below. (Default name of `Python: Remote Attach`)
 
 - Start the server in debug mode via `make start-debug` or `make start-debug run-logs`.
-    - This will start the `main-app` service with port 5678 exposed.
+    - This will start the `{{ app_name }}` service with port 5678 exposed.
 
 - The server will start in waiting mode, waiting for you to attach the debugger (see `/src/app.py`) before continuing to run.
 
@@ -169,14 +169,14 @@ for any breaking changes of features you may use
 
 ### Upgrade Steps
 To upgrade the Python version, make changes in the following places:
-1. Local Python version (see more about managing local Python versions in [getting started](/docs/app/getting-started.md))
-2. [Dockerfile](/app/Dockerfile)
+1. Local Python version (see more about managing local Python versions in [getting started](/docs/{{ app_name }}/getting-started.md))
+2. [Dockerfile](/{{ app_name }}/Dockerfile)
     search for the line `FROM python:3.12-slim as base` - supported versions can be found on [Dockerhub](https://hub.docker.com/_/python)
-3. [pyproject.toml](/app/pyproject.toml)
+3. [pyproject.toml](/{{ app_name }}/pyproject.toml)
     search for the line
     ```toml
     [tool.poetry.dependencies]
     python = "~3.12"
     ```
-   Then run `poetry lock --no-update` to update the [poetry.lock](/app/poetry.lock) file.
-4. [.python-version](/app/.python-version) which is used by tools like pyenv
+   Then run `poetry lock --no-update` to update the [poetry.lock](/{{ app_name }}/poetry.lock) file.
+4. [.python-version](/{{ app_name }}/.python-version) which is used by tools like pyenv