Heimdall: Heterogeneous Memory Benchmark Suite

Heimdall is a benchmark suite to profile the heterogeneous memory systems, including systems with CXL, NVLink-C2C, and AMD Infinity Fabric. See our paper for our profiling results of CXL, NVLink-C2C, and AMD Infinity Fabric, by using Heimdall.

Usage

To install and use Heimdall, please follow these instructions. Currently, Heimdall is supported on Ubuntu 22.04.

Note

While it is possible to run the benchmarks on other platforms, non Ubuntu/Debian-based distributions would require manually installing the dependencies (search the repo for apt to see all such dependencies).

Install dependencies

1. Clone the repo:

   $ git clone 'https://github.com/awesome-cxl/heimdall.git' --recursive

2. Install uv:

   # Refer to https://docs.astral.sh/uv/getting-started/installation/
   $ curl -LsSf https://astral.sh/uv/install.sh | sh

3. Activate the uv python environment and execute the heimdall command line.

   $ uv run heimdall --help

4. [Optional] When developing code, use pre-commit to format commits:

   $ pip install pre-commit
   $ pre-commit install

Run Benchmarks

Benchmarks are implemented under the benchmark/ sub dir. Check each benchmark's doc for example usages:

1. Basic Performance
2. Lock-Free Datastructure
3. LLM

Standalone Benchmark

If you need a portable standalone executable of heimdall (in case your experiment machine does not have internet or cannot install python packages), then:

$ make standalone # Executables are under dist/
$ cd ..
$ tar -zcvf heimdall.tar.gz heimdall/
$ scp heimdall.tar.gz <remote-machine>

Then, on the remote machine:

$ tar -zxvf heimdall.tar.gz
$ cd heimdall
$ ./dist/heimdall <sub-commands>

Note

Make sure the heimdall source repo is copied with the standalone binary, because all the C/C++ source code are needed and they are not part of the standalone binary

Citation

@misc{Heimdall,
      title={The Hitchhiker's Guide to Programming and Optimizing Cache Coherent Heterogeneous Systems: CXL, NVLink-C2C, and AMD Infinity Fabric},
      author={Zixuan Wang and Suyash Mahar and Luyi Li and Jangseon Park and Jinpyo Kim and Theodore Michailidis and Yue Pan and Mingyao Shen and Tajana Rosing and Dean Tullsen and Steven Swanson and Jishen Zhao},
      year={2024},
      eprint={2411.02814},
      archivePrefix={arXiv},
      primaryClass={cs.PF},
      url={https://arxiv.org/abs/2411.02814},
}