Fix Flux and Mac OS CI steps

The Flux CI step often fails because the `attrs` module cannot be found
on the system. This commit fixes this behaviour by explicitly installing
the module through `pip` in the CI workflow.

As per issue lima-vm/lima#1742, the QEMU 8.1.0 Homebrew package is
broken on Intel architectures. This commit adds an additional step to
compile QEMU from source, which fixes the issue.

Author: GlassOfWhiskey

.github/workflows/ci-tests.yaml

@@ -70,6 +70,10 @@ jobs:
       - uses: actions/setup-node@v3
         with:
           node-version: "15"
+      - name: "Install QEMU from binary"
+        run: |
+          brew install --build-from-source qemu
+        if: ${{ startsWith(matrix.on, 'macos-') }}
       - name: "Install Docker (MacOS X)"
         uses: douglascamata/[email protected]
         if: ${{ startsWith(matrix.on, 'macos-') }}
@@ -153,7 +157,7 @@ jobs:
       - name: Install Streamflow
         run: |
          pip install . 
-         pip install lockfile
+         pip install attrs lockfile
          chown -R fluxuser .
       - name: Start Flux and Test Workflow
         run: |
@@ -209,6 +213,10 @@ jobs:
       - uses: actions/setup-node@v3
         with:
           node-version: "15"
+      - name: "Install QEMU from binary"
+        run: |
+          brew install --build-from-source qemu
+        if: ${{ startsWith(matrix.on, 'macos-') }}
       - name: "Install Docker (MacOs X)"
         uses: douglascamata/[email protected]
         if: ${{ startsWith(matrix.on, 'macos-') }}

examples/flux/Dockerfile

@@ -14,7 +14,7 @@ WORKDIR /code
 
 # Install in development mode in case container used for development
 RUN pip install develop . \
- && pip install \
+ && pip install --upgrade \
         attrs \
         lockfile \
  && chown -R fluxuser /code

streamflow/deployment/deployment_manager.py

@@ -196,7 +196,7 @@ async def undeploy(self, deployment_name: str):
                         logger.info(f"COMPLETED Undeployment of {deployment_name}")
                 self.events_map[deployment_name].set()
             # Remove the current environment from all the other dependency graphs
-            for name, deps in (
+            for name, deps in list(
                 (k, v) for k, v in self.dependency_graph.items() if k != deployment_name
             ):
                 deps.discard(deployment_name)