Update template-only-bin scripts

[rocketnova]

Ticket

• Related to Add infra integration test and CD #7

Changes

What was added, updated, or removed in this PR.

• Updates install-template script
• Updates rename-template-app script
• Adds update-template script
• Modifies all template-only-bin scripts to remove .sh extension

Context for reviewers

Background context, more in-depth details of the implementation, and anything else you'd like to call out or ask reviewers.

The install-template script now:

• is scoped to more tightly update only files added by the template
• handles path changes more accurately when doing local development on scripts

The rename-template-app script now:

• more accurately handles find-and-replace strings using word boundaries (e.g. trying to replace app no longer will replace the substring in happens)
• changes file and directory names
• no longer enforces app naming conventions; instead, relies on the user to pick a name that won't cause them problems later

This PR adds the update-template script:

• Previously, the recommendation for this application template repo was to use the install script, which would overwrite configuration changes.
• Now, the update-template script diffs between the installed version of the application template (stored in .template-application-rails-version) and the target version. This is the pattern established on the other Platform templates.
• ⚠️ Known side effects of this approach:
  • This will overwrite configuration changes in the project repo if the diff modified any default configuration settings that the project repo overrode.
  • This will provide unapply-able diff patches if the project repo removed or renamed any paths that the diff affects.

Shell script notes

Note that sed does not behave consistently cross-platform (i.e. GNU versions installed by default on linux and BSD-ish versions installed by default on macOS). Checking the result of running sed --version will reliably distinguish between the two versions. The two main ways this impacts this PR are:

1. In-place file editing: This is handled by using sed -i on linux and sed -i "" on macOS.
2. Word boundaries: This is handled by using \< and \> on linux and [[:<:]] and [[:>:]] on macOS.

Security notes

In general, running curl <something> | bash -s is not good security practice. Users should always read the contents of scripts that are piped into bash. However, most users do not. We should switch away from this practice as soon as it is feasible. It opens our repos to becoming a vector for running or installing bad scripts.

This PR temporarily hardcodes the scripts to use scripts stored on this feature branch. For example:

curl "https://raw.githubusercontent.com/navapbc/${template_name}/rocket/update-template-only-bin-scripts/template-only-bin/rename-template-app" | bash -s -- "${template_short_name}" "${app_name}"

After this PR has been tested and approved by reviewers, I will update the curl commands to use main before merging to main.

Testing

Provide testing instructions and evidence that the code works as expected. Include screenshots, GIF demos, shell commands or output to help show the changes working as expected. ProTip: you can drag and drop or paste images into this textbox.

Test installing and updating a project that uses the default application name

1. Create a test project directory and change into it: mkdir test-template-rails && cd test-template-rails
2. Install the template:

   curl -H 'Cache-Control: no-cache, no-store' https://raw.githubusercontent.com/navapbc/template-application-rails/rocket/update-template-only-bin-scripts/template-only-bin/download-and-install-template | bash -s -- a18946b606c258ce2e9a0f303741f1cca618c85e

   • This command prevents curl from using previously cached versions of the script, uses
     the installer in this branch, and installs a previous commit from the main branch.
   • This will install the application template with the rails app named app-rails.
   • Running ls -la should return this:
   • 
   • Running cat .template-application-rails-version should return a18946b606c258ce2e9a0f303741f1cca618c85e.
3. Create a git repo and commit so you can see git diff results: git init && git add -A . && git commit -m "Initial commit"
4. Update the template:

   curl -H 'Cache-Control: no-cache, no-store' https://raw.githubusercontent.com/navapbc/template-application-rails/rocket/update-template-only-bin-scripts/template-only-bin/update-template | bash -s

   • Running git diff should return this:
   • 
5. You can delete this test project directory

Test installing and updating a project that uses a custom application name

1. Create a fresh test project directory and change into it: mkdir test-template-rails && cd test-template-rails
2. Install the template:

   curl -H 'Cache-Control: no-cache, no-store' https://raw.githubusercontent.com/navapbc/template-application-rails/rocket/update-template-only-bin-scripts/template-only-bin/download-and-install-template | bash -s -- a18946b606c258ce2e9a0f303741f1cca618c85e my-rails

   • This will install the application template with the rails app named my-rails.
   • Running ls -la should return this:
   • 
   • Running ls -la .github/workflows should return this:
   • 
   • Running ls -la docs should return this:
   • 
   • Running cat .template-application-rails-version should return a18946b606c258ce2e9a0f303741f1cca618c85e.
3. Create a git repo and commit so you can see git diff results: git init && git add -A . && git commit -m "Initial commit"
4. Update the template:

   curl -H 'Cache-Control: no-cache, no-store' https://raw.githubusercontent.com/navapbc/template-application-rails/rocket/update-template-only-bin-scripts/template-only-bin/update-template | bash -s -- main my-rails

   • Running git diff should return this:
   • 
   • Running git status should return this:
   • 

[ellery-nava]

@rocketnova - ran through all the testing instructions and everything worked as expected!

[lorenyu]

looks great, have some questions about the rename logic and some of the workflows that i couldn't seem to find in this PR

[lorenyu]

this parameters of this script makes it seem like it supports renaming from any current name to any new name, but i imagine the find/replace logic won't work for some edge cases, like if the current name is something too common.

i know we have a goal of switching to copier soon, where effectively "current name" would become something like {{app_name}}. have we considered the following options:

• option a: going ahead and naming the current name {{app_name}}, and in all template-only github actions workflows, we first run the rename script to an actual app name (e.g. app) so that CI can run properly (i haven't thought through all of the details to see if this is feasible, but just floating the high level idea first)
• option b: reducing the amount of flexibility (and therefore complexity around edge cases) in this script by sticking with an app name like template-only-app rather than an app name that differs per application template? then current_name can be hardcoded to template-only-app, and we have more consistency across our application templates (and can further simplify some of the scripts here).

[rocketnova]

this parameters of this script makes it seem like it supports renaming from any current name to any new name, but i imagine the find/replace logic won't work for some edge cases, like if the current name is something too common.

It currently supports renaming from anything to anything. The regex is restricted to word boundaries, so if you try to rename app to my-app, it won't replace happen with hmy-appen. But yes, if the user were to name it from app-rails to class or else... they would have some bad times when trying to rename it back to something less common.

i know we have a goal of switching to copier soon,

🤩

option a: going ahead and naming the current name {{app_name}}, and in all template-only github actions workflows, we first run the rename script to an actual app name (e.g. app) so that CI can run properly (i haven't thought through all of the details to see if this is feasible, but just floating the high level idea first)

I have a separate script I'm finishing up that effectively "installs an app" for the infra template, allowing the user to provide the app name as an argument. That's in a forthcoming PR.

option b: reducing the amount of flexibility (and therefore complexity around edge cases) in this script by sticking with an app name like template-only-app rather than an app name that differs per application template? then current_name can be hardcoded to template-only-app, and we have more consistency across our application templates (and can further simplify some of the scripts here).

Per the interest and arguments in navapbc/platform#21, I was sticking with app-*.

Is the idea behind option b that all the application templates will use template-only-app and in the installation instructions, the user specifies the name they want to use?

[lorenyu]

Yeah, the idea behind option b is that all application templates will use template-only-app and force the project team to decide on an app name (which if they really don't want to think about it they can rename it back to something generic like app). The benefit is to simplify the code path in the install/update paths to limit the amount of dynamic code (the logic for generating template_short_name for example and guaranteeing the "current_name" is the same string in the rename script). I admit it's not a huge simplification.

[rocketnova]

I'm going to postpone digging into this question until after I've had a chance to make a proof-of-concept of how things might work with copier.

[lorenyu]

clarification question: is the template version file being stored in the application folder? or the project repo root?

[rocketnova]

Currently, in the project root dir. Not the application dir. Same as template-application-nextjs and template-application-flask.

[lorenyu]

Thanks for clarifying. I think that poses a problem if a repo has two application using the same application template. Should we consider moving the template file into the app folder?

[rocketnova]

That's a good point. I agree that we should move the .template-*-version files to within the app folders. I'll start a 🔒 slack discussion thread about this.