The Collective Mind eXtension or Common Metadata eXchange framework (CMX) was developed to support open science and facilitate collaborative, reproducible, and reusable research, development, and experimentation based on FAIR principles and the Collective Knowledge concept.
It helps users non-intrusively convert their software projects, directories, and Git(Hub) repositories into file-based repositories of portable and reusable artifacts (code, data, models, scripts) with extensible metadata, reusable automations, a unified command-line interface, and a simple Python API.
Such artifacts can be easily chained together into portable and technology-agnostic automation workflows, enabling users to rerun, reproduce, and reuse complex experimental setups across diverse and rapidly evolving models, datasets, software, and hardware.
Such workflows, in turn, can be easily integrated with CI/CD pipelines and GitHub Actions and used to create powerful, portable, modular and GUI-based applications.
For example, you can run image classification and the MLPerf inference benchmark on Linux, macOS, and Windows using a few CMX commands as follows:
pip install cmind
cmx pull repo mlcommons@ck --dir=cm4mlops/cm4mlops
cmx run script "app image-classification python torch" --quiet
cmx run script "run-mlperf inference _performance-only _short" --model=resnet50 --precision=float32 --backend=onnxruntime --scenario=Offline --device=cpu --env.CM_SUDO_USER=no --quiet
cmx show cacheCMX extends the legacy Collective Mind framework (MLCommons CM), which have been successfully validated to modularize, automate, and modernize MLPerf benchmarks.
CMX is written in Python, requires minimal dependencies, and has been tested on various flavors of Linux, macOS, Windows, and other operating systems to automate benchmarking, building, running, and optimizing AI, ML, and other emerging workloads and systems.
It encourages community collaboration to share, reuse, and improve artifacts, automations, and experimental setups through public and private Git repositories, rather than redeveloping them from scratch.
CMX is available alongside the legacy CM framework via the cmind PyPI package. Please follow this guide to install and start using it.
If you encounter any issues or have suggestions, please don't hesitate to open a GitHub ticket or contact the CMX author.
We have released CMX version 4.0.0 as a drop-in, backward-compatible replacement for the earlier Collective Mind framework (CM) and other MLCommons automations. Designed with user feedback in mind, CMX offers a simpler, more robust interface. It is available alongside Collective Mind (CM) in the Python cmind package: sources, pypi.
See online documentation.
We thank all our contributors for their invaluable feedback and support!
Copyright (c) 2024-2025 MLCommons
Grigori Fursin and the cTuning foundation donated this project to MLCommons to benefit everyone.