Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .actrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
-P ubuntu-latest=catthehacker/ubuntu:act-24.04
12 changes: 12 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,15 @@ __pycache__/

trace_collection/post_execution/traces

# uv
.venv/
uv.lock

# protoc output of the C++ feeder build
*.pb.cc
*.pb.h

# act reads these files. They can hold secrets: never commit them.
.secrets
.env
.vars
2 changes: 2 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,5 @@ Generally we encourage people to become a MLCommons member if they wish to contr
Regardless of if you are a member, your organization needs to sign the MLCommons CLA. Please fill out this [CLA sign up form](https://forms.gle/Ew1KkBVpyeJDuRw67) form to get started.

MLCommons project work is tracked with issue trackers and pull requests. Modify the project in your own fork and issue a pull request once you want other developers to take a look at what you have done and discuss the proposed changes. Ensure that cla-bot and other checks pass for your Pull requests.

To set up a development environment and run the checks, see the [development guide](doc/DEVELOPMENT.md). Run `just gate` before you open a pull request.
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,17 @@ url={https://arxiv.org/abs/2605.11333}

Check out [`USER_GUIDE`](USER_GUIDE.md) for details.

## Development

Set up with [uv](https://docs.astral.sh/uv/) and [just](https://just.systems/):

```bash
just setup # Python 3.11, development tools, PARAM
just gate # lint, types, tests and more
```

The pip setup still works. The [development guide](doc/DEVELOPMENT.md) explains both setups, the daily steps, and how to run the CI workflows on your computer.

## License

Chakra is released under the Apache License 2.0. Please see the [`LICENSE.md`](LICENSE.md) file for more information.
Expand Down
12 changes: 12 additions & 0 deletions USER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,18 @@ $ pip uninstall mlc-chakra
# If you installed a pre-rename build, also: pip uninstall chakra
```

### Alternative: install with uv
[uv](https://docs.astral.sh/uv/) can replace `venv` and `pip`. The steps are the same as above.

```bash
$ uv venv --python 3.11
$ source .venv/bin/activate
$ uv pip install mlc-chakra
$ uv pip install "git+https://github.com/facebookresearch/param.git@7b19f586dd8b267333114992833a0d7e0d601630#subdirectory=et_replay"
```

To uninstall Chakra, run `uv pip uninstall mlc-chakra`.

## Tools Overview
### Execution Trace Link (chakra_trace_link)
Merge Chakra host execution trace and Chakra device execution trace to encode GPU operators into the output execution trace.
Expand Down
132 changes: 132 additions & 0 deletions doc/DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
# Development guide

This guide is for people who change the Chakra code. It shows how to set up your computer, check a change, and run the CI workflows there.
If you only want to use Chakra, read the [User Guide](../USER_GUIDE.md).

## Terms

| Term | Meaning |
|---|---|
| gate | The full set of local checks. `just gate` runs it. It has the Python checks and the C++ format check of CI, and more. It does not build or test the C++ feeder. |
| recipe | One task in the `justfile`, for example `just lint`. |
| virtual environment | The Python 3.11 install in `.venv/` that `uv` makes. |
| workflow | One file in `.github/workflows/`. GitHub Actions runs it. |
| act | A tool that runs a workflow in Docker on your computer. |

## What you need

| Tool | Needed for | Install on macOS |
|---|---|---|
| [uv](https://docs.astral.sh/uv/) | Python, the virtual environment, the dependencies | `brew install uv` |
| [just](https://just.systems/) | The recipes | `brew install just` |
| [Docker](https://docs.docker.com/get-started/get-docker/) | `just act` and `just ci-github-lint` | Docker Desktop or OrbStack |
| [act](https://nektosact.com/) | `just act` only | `brew install act` |

On Linux, see the install page of each tool.

## Windows

The recipes and the scripts in `scripts/` need bash. On Windows, use WSL2. Then follow the steps for Linux in this guide.

Native Windows is not supported. We tested these steps on macOS and in a WSL instance of Debian 13 (trixie). If a step fails on Windows, tell us which step and the first error.

## Set up

Run these commands in the repository folder.

```bash
just setup
just gate
```

`just setup` installs Python 3.11, the development tools, and PARAM. `just gate` must pass before you start to change code.

## Daily steps

```mermaid
flowchart LR
E[Edit code] --> F[just fmt]
F --> G[just gate]
G -->|pass| P[Open a pull request]
G -->|fail| E
```

## Recipes

Run `just` to see the list.

| Recipe | What it does | CI workflow it matches |
|---|---|---|
| `just setup` | Installs the virtual environment. | - |
| `just fmt` | Formats the code. Fixes the lint errors that ruff can fix. | - |
| `just lint` | Checks the format, runs ruff, and runs vulture. | `python_lint.yml` |
| `just lint-scripts` | Lints the shell scripts in `scripts/` with shellcheck. | - |
| `just types` | Type checks with pyright. | `python_lint.yml` |
| `just test` | Runs the unit tests. Add pytest options after the name, for example `just test -k trace_link`. | `python_tests.yml` |
| `just test-slow` | Runs the slow tests. They need a network. They build the package as an editable install, and as a wheel that has a file left out. | - |
| `just proto-smoke` | Imports the generated protobuf modules with the newest and the oldest allowed protobuf. | `python_tests.yml` |
| `just wheel-check` | Builds the wheel from a clean copy and installs it (not editable) in a new temporary virtual environment. Checks the wheel files, imports the protobuf modules with two protobuf versions, and starts the commands. The other checks use the editable install. | `python_tests.yml`, `end_to_end_tests.yml` |
| `just cpp-fmt-check` | Checks the C++ format with clang-format 16. | `cpp_lint.yml` |
| `just gate` | Runs all the checks above. | - |
| `just act` | Runs one workflow in Docker. Add a file name from `.github/workflows/`. The default is `python_lint.yml`. | - |
| `just ci-github-lint` | Lints the workflow files with actionlint in Docker. It fails today, because the workflows use old actions (`checkout@v2`, `setup-python@v2`). It is not part of the gate. | - |

## Common tasks

| Task | How |
|---|---|
| Change a `.proto` file | `just test` rebuilds the generated code. With pip, run `pip install .` again. |
| Add a runtime dependency | Add it to `dependencies` in `pyproject.toml`. |
| Add or change a development tool | Change `[dependency-groups]` in `pyproject.toml` and `requirements-dev.txt`. `just test` fails when they differ. |

## Run the CI workflows on your computer

Use `just act <file>`, for example `just act python_tests.yml`. The file is a name in `.github/workflows/`. It runs the workflow in a Docker container. After the first run, which downloads a large image, a run takes about 15 to 40 seconds.

Use it when you change a workflow file. For a code change, `just gate` is faster.

Each run starts with an empty tool cache. `scripts/act.sh` first removes the shared `act` cache volume and the `act` containers that use it. It does not touch other Docker containers. It refuses to start while another `act` run is active on your computer. It needs `pgrep` for that check.

`act` uses a copy of your working folder, with your uncommitted changes. It is not exactly GitHub. On Apple silicon, `act` runs arm64 containers, and GitHub runs x86_64. A green `act` run does not replace the check on your pull request.

| Workflow | Result with `just act` |
|---|---|
| `python_lint.yml` | Works. |
| `python_tests.yml` | Works. |
| `cpp_lint.yml` | Works. |
| `end_to_end_tests.yml` | Works. |
| `feeder_tests.yml` | Fails. `apt install` asks for a confirmation, and `act` has no terminal. The GitHub runner image sets apt to answer yes. The `act` image does not. |
| `codeql.yml`, `cla.yml` | Not run. They need GitHub services. |
| Any other workflow | Not tested. |

Tested with OrbStack on an Apple silicon Mac (November 2026). Docker Desktop is not tested. If `just act` fails there with a "mkdir ... operation not supported" error, open an issue.

## Shell scripts

Shell scripts live in `scripts/`. `just lint-scripts` checks them with shellcheck. The tests in `tests/test_scripts.py` run each script with fake `docker`, `act`, and `uv` programs. They need neither Docker nor a network, except one slow test.

## The classic setup

The `pip` setup still works. Use Python 3.11, the version that CI uses.

```bash
python3.11 -m venv .venv
source .venv/bin/activate
pip install .
pip install "git+https://github.com/facebookresearch/param.git@7b19f586dd8b267333114992833a0d7e0d601630#subdirectory=et_replay"
pip install -r requirements-dev.txt
python -m pytest tests
ruff format --check . && ruff check . && pyright && vulture src/ tests/
```

The second `pip install` line installs PARAM, which Chakra needs. `pip install .` is not an editable install. Run it again after you change code.

## Python version

`just` runs `uv` with Python 3.11, the version that CI uses. `uv` installs it for you. If you run `uv` yourself, add `--python 3.11`.

We do not ship a `.python-version` file. Your pyenv or mise setup stays as it is.

## Secrets

`act` can read the files `.secrets`, `.env`, and `.vars`. Git ignores them. Never commit them.
68 changes: 68 additions & 0 deletions justfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Developer tasks for chakra. Run `just` to list them.

# uv uses this Python, the version that CI uses. A .python-version file would also reach pyenv, mise and asdf users.
export UV_PYTHON := "3.11"

default:
@just --list

# Install the environment: Python 3.11, the dev tools and PARAM (uv sync)
setup:
uv sync

# Format the code and fix the lint errors that ruff can fix
fmt:
uv run ruff format .
uv run ruff check --fix .

# Lint: format check, ruff and vulture (the CI "Python Lint" job, plus the format check that CI lacks)
lint:
uv run ruff format --check .
uv run ruff check .
uv run vulture src/ tests/

# Lint the shell scripts in scripts/ with shellcheck
lint-scripts:
uvx --from shellcheck-py==0.11.0.1 shellcheck scripts/*.sh

# Only the dev tools are installed, as in CI. Every Python file that git does not ignore is checked, new files too.
# Pyright must not scan .venv. With the project installed, pyright reports more issues than CI does.
# Type check with pyright, as CI does
types:
scripts/run_on_git_files.sh '*.py' -- uv run --isolated --only-group dev pyright

# Unit tests. Use `python -m pytest`, not `pytest`: 3 test files import `src.*`, which needs the repo root on sys.path
[positional-arguments]
test *args:
CHAKRA_TOOLING_TESTS=1 uv run python -m pytest -vv tests "$@"

# The slow tests build the package: as an editable install, and as a wheel that has a file left out. They need a network.
test-slow:
CHAKRA_TEST_SLOW=1 uv run python -m pytest tests/test_tooling.py tests/test_scripts.py -k "test_editable_install_builds_and_imports or test_wheel_check_catches_a_file_that_is_missing_from_the_wheel"

# It imports the generated modules with the newest protobuf, then with the lowest one that pyproject.toml allows.
# The CI "Protobuf Gencode/Runtime Smoke Test"
proto-smoke:
scripts/proto_smoke.sh

# The other checks use the editable install that `uv sync` makes. It can hide a file that the wheel does not hold.
# Build the wheel from a clean copy, install it (not editable) into a new environment, and test it there, as CI does
wheel-check:
scripts/wheel_check.sh

# Check the C++ format with clang-format 16 (the CI "C++ Lint" job)
cpp-fmt-check:
scripts/run_on_git_files.sh '*.cc' '*.cpp' '*.h' '*.hh' -- uvx --from 'clang-format==16.*' clang-format --dry-run --Werror

# The full check. Run it before you open a pull request.
gate: lint lint-scripts types test test-slow proto-smoke wheel-check cpp-fmt-check

# It fails now: the workflows use old actions (checkout@v2, setup-python@v2). It is not part of `gate`.
# Lint the GitHub workflow files with actionlint (needs Docker)
ci-github-lint:
docker run --rm -v "$PWD":/repo -w /repo rhysd/actionlint:1.7.7 -color

# Needs Docker and act (`brew install act`). It removes the shared act cache volume and the act containers that use it.
# It does not replace the real check on GitHub. Run one workflow in Docker with act, for example `just act python_tests.yml`
act workflow="python_lint.yml":
scripts/act.sh {{quote(workflow)}}
26 changes: 26 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,29 @@ reportAttributeAccessIssue = false
[tool.vulture]
ignore_names = ["mock_process_thread"]
min_confidence = 100

# Developer environment for uv (`uv sync`). These pins equal requirements-dev.txt until that file is retired.
[dependency-groups]
dev = [
"pyright==1.1.359",
"pytest==8.1.1",
"ruff==0.3.7",
"vulture==2.11",
"build==1.6.1",
"twine==7.0.0",
]
# Temporary. PARAM (et_replay) from git. Delete this group and the source below when issue #237 is merged.
param = ["et-replay"]

[tool.uv]
default-groups = ["dev", "param"]
# Rebuild the package, and so the generated *_pb2.py files, when a .proto file changes. The default keys miss it.
cache-keys = [
{ file = "pyproject.toml" },
{ file = "setup.py" },
{ file = "setup.cfg" },
{ file = "schema/protobuf/*.proto" },
]

[tool.uv.sources]
et-replay = { git = "https://github.com/facebookresearch/param.git", rev = "7b19f586dd8b267333114992833a0d7e0d601630", subdirectory = "et_replay" }
53 changes: 53 additions & 0 deletions scripts/act.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#!/usr/bin/env bash
# Run one GitHub workflow in Docker with act, before you push.
# Usage: scripts/act.sh [WORKFLOW_FILE] (default: python_lint.yml, a file in .github/workflows/)
#
# Before each run, this script removes the shared act cache volume (act-toolcache) and the act containers that use it.
# Why: act keeps one volume for all runs. A package that one run installed is then visible to the next run. This does
# not happen on a GitHub runner. A failed act job also leaves its container running, and that container holds the volume.
set -euo pipefail

cd "$(dirname "$0")/.."

workflow="${1:-python_lint.yml}"
file=".github/workflows/$workflow"
if [ ! -f "$file" ]; then
echo "act: no workflow file $file" >&2
exit 2
fi

case "$(uname -m)" in
x86_64) arch=amd64 ;;
arm64 | aarch64) arch=arm64 ;;
*)
echo "act: unsupported CPU type $(uname -m)" >&2
exit 1
;;
esac

# The removal below would break an act run that is active on this computer, in this repository or in another one.
# A missing pgrep must not count as "no act run": check for the program first.
if ! command -v pgrep >/dev/null; then
echo "act: pgrep is not installed, so this script cannot tell if another act run is active." >&2
exit 1
fi
if pgrep -x act >/dev/null; then
echo "act: another act run is active. Wait for it, or stop it." >&2
exit 1
fi

# Docker Desktop and OrbStack use different sockets: ask the active Docker context.
host="${DOCKER_HOST:-$(docker context inspect --format '{{.Endpoints.docker.Host}}')}"
export DOCKER_HOST="$host"

stale=$(docker ps -aq --filter volume=act-toolcache)
listed="${stale//$'\n'/ }"
echo "act: removing the volume act-toolcache and these containers: ${listed:-none}" >&2
while IFS= read -r id; do
if [ -n "$id" ]; then
docker rm -f "$id" >/dev/null
fi
done <<<"$stale"
docker volume rm -f act-toolcache >/dev/null

exec act pull_request -W "$file" --container-architecture "linux/$arch" --container-daemon-socket -
12 changes: 12 additions & 0 deletions scripts/proto_smoke.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#!/usr/bin/env bash
# The CI "Protobuf Gencode/Runtime Smoke Test".
# Import the generated protobuf modules with the newest protobuf, then with the lowest one that pyproject.toml allows.
# Usage: scripts/proto_smoke.sh
set -euo pipefail

cd "$(dirname "$0")/.."

check='import chakra.schema.protobuf.et_def_pb2, chakra.schema.protobuf.storage_pb2'
uv run python -c "$check"
floor=$(uv run python -c "import re; from importlib.metadata import requires; print(next(re.search(r'>=([0-9.]+)', r).group(1) for r in requires('mlc-chakra') if r.startswith('protobuf')))")
uv run --with "protobuf==$floor" python -c "$check"
35 changes: 35 additions & 0 deletions scripts/run_on_git_files.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/usr/bin/env bash
# Run a command on the files that git knows or could add: tracked files and new files, but not ignored or deleted ones.
# Usage: scripts/run_on_git_files.sh PATTERN... -- COMMAND [ARGUMENT...]
# Example: scripts/run_on_git_files.sh '*.py' -- pyright
# Run it in the repository. It stops when no file matches, so a check cannot pass on an empty list.
set -euo pipefail

usage() {
echo "usage: $0 PATTERN... -- COMMAND [ARGUMENT...]" >&2
exit 2
}

patterns=()
while [ "$#" -gt 0 ] && [ "$1" != "--" ]; do
patterns+=("$1")
shift
done
[ "${#patterns[@]}" -gt 0 ] || usage
[ "$#" -ge 2 ] || usage
shift # the "--"

files=()
while IFS= read -r -d '' file; do
# A leading ./ keeps a file named -x.py from looking like an option. A link to a directory is not a file.
if [ -f "$file" ]; then
files+=("./$file")
fi
done < <(git ls-files -z --cached --others --exclude-standard -- "${patterns[@]}")

if [ "${#files[@]}" -eq 0 ]; then
echo "run_on_git_files: no files match ${patterns[*]} (run it in a git checkout)" >&2
exit 1
fi

exec "$@" "${files[@]}"
Loading
Loading