Devcontainer update

.devcontainer/Readme.md

@@ -1,31 +1,46 @@
-Build CircuitPython in a Github-Devcontainer
-============================================
+Build CircuitPython in a Github-Codespace
+=========================================
 
-To build CircuitPython within a Github-Devcontainer, you need to perform
+To build CircuitPython within a Github codespace, you need to perform
 the following steps.
 
-  1. checkout the code to a devcontainer
-
-    - click on the green "<> Code"-button
-    - select the Codespaces-tab
-    - choose "+ new with options..." from the "..."-menu
-    - in the following screen select the branch and then
-    - select ".devcontainer/cortex-m/devcontainer.json" instead
-      of "Default Codespaces configuration"
-    - update region as necessary
-    - finally, click on the green "Create codespace" button
-
-  2. Your codespace is created. Cloning the images is quite fast, but
-     preparing it for CircuitPython-development takes about 10 minutes.
-     Note that this is a one-time task.
+  1. checkout the code to a codespace
+
+  - click on the green "<> Code"-button
+  - select the Codespaces-tab
+  - choose "+ new with options..." from the "..."-menu
+  - in the following screen select the branch and then
+  - select the port instead of "Default project configuration"
+    (unsupported: ports not using cortex-m or esp-idf)\
+    ![](./codespace_options.png)
+  - update region as necessary
+  - finally, click on the green "Create codespace" button
+
+  2. Your codespace is created. Cloning the image and the repo is quite fast,
+     but preparing it for CircuitPython-development takes about 10 minutes.
+     But this is a one-time task: once created, your codespace exists
+     until you explicitly delete it or until it times out (default: 30 days).\
+     (Technical note: due to a bug in codespace creation, the setup is
+     triggered from `$HOME/.bashrc` and runs in the background).
 
   3. During creation, you can run the command
-     `tail -f /workspaces/.codespaces/.persistedshare/creation.log`
-     to see what is going on.
+     `tail -f /workspaces/install_build_env.log.active`
+     to see what is going on. Once finished the log file is available
+     as `/workspaces/install_build_env.log`.
 
-  4. To actually build CircuitPython, run
+  4. To actually build CircuitPython, open a new terminal and run e.g.
 
          cd ports/raspberrypi
          make -j $(nproc) BOARD=whatever TRANSLATION=xx_XX
 
-     This takes about 2m40s.
+     This takes about 2m40s. The new terminal is necessary since the
+     setup of the build environment also changes `$HOME/.bashrc` and
+     sets important environment variables in that file.
+
+As a normal user, you have 120 CPU-hours and 15GB per month free. Since
+the smallest machine has two CPUs, you effectively have 60 hours active
+time available.
+
+All scripts are in `circuitpython/.devcontainer` and can also be executed
+manually which should usually not be necessary. With small changes, they
+should also work on a Linux-PC or laptop.

.devcontainer/cortex-m/devcontainer.json → .devcontainer/atmel-samd/devcontainer.json

@@ -1,10 +1,11 @@
 // For format details, see https://aka.ms/devcontainer.json. For config options, see the
 // README at: https://github.com/devcontainers/templates/tree/main/src/universal
 {
-	"name": "CircuitPython Cortex-M Build-Environment (base: Default Linux Universal)",
-	"image": "mcr.microsoft.com/devcontainers/universal:2-linux",
-	"postCreateCommand": ".devcontainer/cortex-m/on-create.sh",
-	"remoteEnv": { "PATH": "/workspaces/gcc-arm-none-eabi/bin:${containerEnv:PATH}" }
+	"name": "atmel-samd",
+	"image": "mcr.microsoft.com/devcontainers/universal:2",
+	"postCreateCommand": ".devcontainer/post_create.sh",
+	"remoteEnv": { "CP_TOOLCHAIN": "cortex-m",
+                      "CP_PORT": "atmel-samd" }
 
 	// Features to add to the dev container. More info: https://containers.dev/features.
 	// "features": {},

.devcontainer/common_tools.sh

@@ -0,0 +1,43 @@
+#!/bin/bash
+# -----------------------------------------------------------------------------
+# common_tools.sh: install tools and requirements for CircuitPython
+#
+# This script installs tools common to all builds.
+#
+# Author: Bernhard Bablok
+#
+# -----------------------------------------------------------------------------
+
+REPO_ROOT="/workspaces/circuitpython"
+
+echo -e "[common_tools.sh] starting install"
+cd "$REPO_ROOT"
+
+# --- repositories and tools   ------------------------------------------------
+
+echo -e "[common_tools.sh] adding pybricks/ppa"
+sudo add-apt-repository -y ppa:pybricks/ppa
+echo -e "[common_tools.sh] installing uncrustify and mtools"
+sudo apt-get -y install uncrustify mtools
+
+# dosfstools >= 4.2 needed, standard repo only has 4.1
+echo -e "[common_tools.sh] downloading and installing dosfstools"
+wget https://github.com/dosfstools/dosfstools/releases/download/v4.2/dosfstools-4.2.tar.gz
+tar -xzf dosfstools-4.2.tar.gz
+(cd dosfstools-4.2/
+ ./configure
+ make -j $(nproc)
+ sudo make install
+)
+rm -fr dosfstools-4.2 dosfstools-4.2.tar.gz
+
+# --- circuitpython setup   --------------------------------------------------
+
+# additional python requirements
+echo -e "[common_tools.sh] pip-installing requirements"
+pip install --upgrade -r requirements-dev.txt
+pip install --upgrade -r requirements-doc.txt
+
+# add pre-commit
+echo -e "[common_tools.sh] installing pre-commit"
+pre-commit install

.devcontainer/cortex-m-toolchain.sh

@@ -0,0 +1,24 @@
+#!/bin/bash
+# -----------------------------------------------------------------------------
+# cortex-m-toolchain.sh: install toolchain for CircuitPython
+#
+# Author: Bernhard Bablok
+#
+# -----------------------------------------------------------------------------
+
+echo -e "[cortex-m-toolchain.sh] starting install"
+
+# --- tooling   --------------------------------------------------------------
+
+echo -e "[cortex-m-toolchain.sh] downloading and installing gcc-arm-non-eabi toolchain"
+cd /workspaces
+
+wget -qO gcc-arm-none-eabi.tar.xz \
+  https://developer.arm.com/-/media/Files/downloads/gnu/13.2.rel1/binrel/arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz
+
+tar -xJf gcc-arm-none-eabi.tar.xz
+ln -s arm-gnu-toolchain-13.2.Rel1-x86_64-arm-none-eabi gcc-arm-none-eabi
+rm -f gcc-arm-none-eabi.tar.xz
+
+echo -e "[cortex-m-toolchain.sh] update PATH in environment"
+echo -e "\nexport PATH=/workspaces/gcc-arm-none-eabi/bin:$PATH"  >> $HOME/.bashrc

.devcontainer/cxd56/devcontainer.json

@@ -0,0 +1,24 @@
+// For format details, see https://aka.ms/devcontainer.json. For config options, see the
+// README at: https://github.com/devcontainers/templates/tree/main/src/universal
+{
+	"name": "cxd56",
+	"image": "mcr.microsoft.com/devcontainers/universal:2",
+	"postCreateCommand": ".devcontainer/post_create.sh",
+	"remoteEnv": { "CP_TOOLCHAIN": "cortex-m",
+                      "CP_PORT": "cxd56" }
+
+	// Features to add to the dev container. More info: https://containers.dev/features.
+	// "features": {},
+
+	// Use 'forwardPorts' to make a list of ports inside the container available locally.
+	// "forwardPorts": [],
+
+	// Use 'postCreateCommand' to run commands after the container is created.
+	// "postCreateCommand": "uname -a",
+
+	// Configure tool-specific properties.
+	// "customizations": {},
+
+	// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
+	// "remoteUser": "root"
+}

.devcontainer/esp-idf-toolchain.sh

@@ -0,0 +1,42 @@
+#!/bin/bash
+# -----------------------------------------------------------------------------
+# esp-idf-toolchain.sh: install toolchain for CircuitPython
+#
+# Author: Bernhard Bablok
+#
+# -----------------------------------------------------------------------------
+REPO_ROOT="/workspaces/circuitpython"
+
+echo -e "[esp-idf-toolchain.sh] starting install"
+
+# --- tooling   --------------------------------------------------------------
+
+echo -e "[esp-idf-toolchain.sh] fetch packages"
+sudo apt-get update
+sudo apt-get -y install ninja-build cmake libusb-1.0-0
+
+# --- esp-idf   --------------------------------------------------------------
+
+echo -e "[esp-idf-toolchain.sh] installing esp-idf"
+cd "$REPO_ROOT/ports/espressif"
+esp-idf/install.sh
+source esp-idf/export.sh
+
+# --- re-install our packages in venv created by export.sh   -----------------
+
+echo -e "[esp-idf-toolchain.sh] updating python-packages"
+cd "$REPO_ROOT"
+pip3 install --upgrade -r requirements-dev.txt
+pip3 install --upgrade -r requirements-doc.txt
+
+# --- and again install esp-idf (needs other versions)   ----------------------
+
+echo -e "[esp-idf-toolchain.sh] installing esp-idf (2nd iteration)"
+cd "$REPO_ROOT/ports/espressif"
+esp-idf/install.sh
+
+# --- update $HOME/.bashrc   --------------------------------------------------
+
+echo -e "[esp-idf-toolchain.sh] update environment in .bashrc"
+
+echo -e "\nsource $REPO_ROOT/ports/espressif/esp-idf/export.sh &> /dev/null\n" >> "$HOME"/.bashrc

.devcontainer/espressif/devcontainer.json

@@ -0,0 +1,24 @@
+// For format details, see https://aka.ms/devcontainer.json. For config options, see the
+// README at: https://github.com/devcontainers/templates/tree/main/src/universal
+{
+	"name": "espressif",
+	"image": "mcr.microsoft.com/devcontainers/universal:2",
+	"postCreateCommand": ".devcontainer/post_create.sh",
+	"remoteEnv": { "CP_TOOLCHAIN": "esp-idf",
+                       "CP_PORT": "espressif" }
+
+	// Features to add to the dev container. More info: https://containers.dev/features.
+	// "features": {},
+
+	// Use 'forwardPorts' to make a list of ports inside the container available locally.
+	// "forwardPorts": [],
+
+	// Use 'postCreateCommand' to run commands after the container is created.
+	// "postCreateCommand": "uname -a",
+
+	// Configure tool-specific properties.
+	// "customizations": {},
+
+	// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
+	// "remoteUser": "root"
+}

.devcontainer/fetch-port-submodules.sh

@@ -0,0 +1,19 @@
+#!/bin/bash
+# -----------------------------------------------------------------------------
+# fetch-port-submodules.sh: fetch port specific submodules
+#
+# Author: Bernhard Bablok
+#
+# -----------------------------------------------------------------------------
+
+REPO_ROOT="/workspaces/circuitpython"
+cd "$REPO_ROOT"
+
+if [ -z "$CP_PORT" ]; then
+  echo -e "[fetch-port-submodules.sh] CP_PORT not set. Cannot fetch submodules!"
+  exit 3
+fi
+
+cd "ports/$CP_PORT"
+echo -e "[fetch-port-submodules.sh] fetching necessary submodules"
+make fetch-port-submodules

.devcontainer/install_build_env.sh

@@ -0,0 +1,64 @@
+#!/bin/bash
+# -----------------------------------------------------------------------------
+# install_build_env.sh: install build-environment for CircuitPython
+#
+# Normally, this should run directly as postCreateCommand during container
+# creation. Due to an unresolved bug on how Github-codespaces creates a clone,
+# this script is started from $HOME/.bashrc instead.
+#
+# The script delegates parts to other scripts for reuse across toolchains.
+# This has the added benefit that they can be called independently later again
+# if necessary.
+#
+# The scripts expect the environment-variables CP_TOOLCHAIN and CP_PORT to be set
+# to valid values. This is normally done from within
+#   .devcontainer/<port>/devcontainer.json
+#
+# Author: Bernhard Bablok
+#
+# -----------------------------------------------------------------------------
+REPO_ROOT="/workspaces/circuitpython"
+
+# --- install exit-handler for cleanup   --------------------------------------
+
+on_exit() {
+  rc=$?
+  if [ -f /workspaces/install_build_env.log.active ]; then
+    mv /workspaces/install_build_env.log.active /workspaces/install_build_env.log
+  fi
+  rm -rf /tmp/install_build_env
+  exit $rc
+}
+
+# --- test prerequisites for installation   ------------------------------------
+
+while ! test -f /workspaces/post_create.finished; do
+  echo -e "[install_build_env.sh] waiting for /workspaces/post_create.finished ..."
+  sleep 1
+done
+
+if [ -f /workspaces/install_build_env.log ]; then
+  echo -e "[install_build_env.sh] installation already done"
+  exit 0
+elif ! mkdir /tmp/install_build_env 2>/dev/null; then
+  # mkdir is atomic, so we know we are already running
+  echo -e "[install_build_env.sh] install already running with PID $(cat /tmp/install_build_env/pid.txt)"
+  exit 0
+else
+  echo -e "$$" > /tmp/install_build_env/pid.txt
+  trap 'on_exit' EXIT
+fi
+
+echo -e "[install_build_env.sh] starting install"
+
+# --- delegate install steps to other scripts   -------------------------------
+(
+"$REPO_ROOT/.devcontainer/fetch-port-submodules.sh" || exit 3
+"$REPO_ROOT/.devcontainer/common_tools.sh" || exit 3
+"$REPO_ROOT/.devcontainer/$CP_TOOLCHAIN-toolchain.sh" || exit 3
+"$REPO_ROOT/.devcontainer/make-mpy-cross.sh" || exit 3
+echo -e "Setup complete!\nStart a new terminal and build CircuitPython!\n"
+) |& tee /workspaces/install_build_env.log.active
+
+echo -e "[install_build_env.sh] Setup complete!"
+exit 0