Skip to content

installation.md

Latest commit

 

History

History
249 lines (196 loc) · 10.2 KB

installation.md

File metadata and controls

249 lines (196 loc) · 10.2 KB
Raw

TsingMicro Installation Guide

TsingMicro TX8 uses the Kuiper SDK's TX runtime (tx_runtime.h, libhpgr). It is not a CUDA-compatible stack: there is no libcuda.so, no libtorch_cuda.so, and no CUDA-ABI vendor torch. Operator coverage therefore comes from FlagGems' TsingMicro Triton backend, with cpu_fallback for every op the routing file does not name. The one native kernel set is the host-draw RNG family in csrc/aten/backends/tsingmicro/rng.cc.

This platform is Experimental. It is measured on TX8110 hardware, but it has no CI job and no published wheel.

Prerequisites

  • CPU PyTorch 2.10.x: torch==2.10.0+cpu from the upstream CPU index. A CUDA wheel must not be installed: the build links no CUDA kernels and the FLAGOS_BUILD_BOXING path has nothing to box onto.
  • Kuiper SDK: installed at /usr/local/kuiper by default. It must provide include/tx_runtime.h and lib/libhpgr.so. A different root is selected with the CMake cache variable KUIPER_ROOT (-DKUIPER_ROOT=/path/to/kuiper).
  • FlagGems: flag_gems==5.4.0 (the version the routing tables were generated against).
  • Triton: triton>=3.5.1. TsingMicro has no FlagTree build, so setup.py declares a stock Triton requirement instead of pinning FlagTree. A vendor Triton build that carries the tsingmicro backend is the validated case.
  • Python: 3.8 or later (validated with 3.10).
  • Device access: the TX8 runtime must see at least one device (tsm_smi lists them).
  • Operating system: Linux.

Installation

1. Install CPU PyTorch

pip install torch==2.10.0 --index-url https://download.pytorch.org/whl/cpu

2. Build and install torch_fl

git clone https://github.com/flagos-ai/PyTorch-Plugin-FL.git
cd PyTorch-Plugin-FL

FLAGOS_ACCELERATOR=tsingmicro pip install --no-build-isolation -v -e .

Build flags:

  • FLAGOS_ACCELERATOR=tsingmicro: selects the Kuiper runtime and backends_tsingmicro.conf, and defines USE_TSINGMICRO.
  • FLAGOS_BUILD_VENDOR=ON (default, pinned): compiles the host-draw RNG kernels. Turning it off is rejected rather than accepted silently, because csrc/aten/register.cc claims those symbols unconditionally and the build would not link.
  • FLAGOS_BUILD_FLAGGEMS=ON (default): the operator path.
  • FLAGOS_BUILD_BOXING=OFF (default, must stay off): the wheel bundles no vendor libtorch and the TX stack exports no CUDA runtime.
  • FLAGOS_BUILD_FLAGGEMS_CPP=OFF (default): no liboperators.so exists for the Kuiper toolchain.
  • FLAGOS_BUILD_TILEOPS=OFF (default): TileOps targets SM90 NVIDIA parts.
  • --no-build-isolation: builds against the installed CPU torch.

The routing registration is generated by scripts/codegen/codegen_boxing_flaggems_register.py into csrc/aten/generated/register_flaggems_tsingmicro.inc. Re-run it with --check after changing torch_fl/configs/backends_tsingmicro.conf.

3. In-tree incremental builds

The runtime can also be rebuilt without reinstalling the Python package:

cmake -S . -B build -DFLAGOS_ACCELERATOR=tsingmicro
cmake --build build --target flagos -j"$(nproc)"
cmake --install build          # must run from the repository root

cmake --install writes into <repo>/torch_fl, the install prefix this repository uses for in-place development.

Runtime environment

The Kuiper library path is embedded in the shared library's RUNPATH, but the profiler tool and any FlagCX installation are loaded separately. Set:

export LD_LIBRARY_PATH=/usr/local/kuiper/lib:/usr/local/kuiper/tsm8-profiler/lib:<repo>/torch_fl/lib:$LD_LIBRARY_PATH

Verification

Import torch_fl before torch on this platform, then query the device:

import torch_fl
import torch

print(torch.flagos.is_available(), torch.flagos.device_count())

x = torch.randn(2, 3, device="flagos")
for name, fn in [
    ("unsqueeze", lambda: x.unsqueeze(0)),
    ("erfc", lambda: torch.erfc(x)),
    ("cat", lambda: torch.cat([x, x], dim=0)),
    ("randn", lambda: torch.randn(2, 2, device="flagos")),
    ("mean", lambda: x.mean()),
]:
    print(name, tuple(fn().shape))

Measured on a TX8110 with the default FlagGems-primary build (2026-10-21, torch 2.10.0+cpu, flag_gems 5.4.0, triton 3.6.0): is_available() is True, both devices are reported, and all five calls succeed. This is a smoke test, not a per-overload correctness survey; see docs/reference/compatibility.md for the support status and the evidence record.

Operator routing

torch_fl/configs/backends_tsingmicro.conf has 81 hand-written routes:

Route Count Meaning
flaggems / flagos_python 67 Served by FlagGems' TsingMicro Triton backend
tsingmicro 14 Metadata-only view ops (alias, detach, permute, select, slice, squeeze, t, transpose, unbind, unsqueeze, _conj, _neg_view, _unsafe_view) served by stride arithmetic, no device kernel
unlisted — cpu_fallback

The RNG family (rand, randn, uniform_, normal_, randint, randperm, and their out/_like/generator overloads) is registered directly in csrc/aten/register.cc, not through the routing file. TX8 has no random-number unit, so those kernels draw on the host from the flagos generator and copy the result to the device.

Two compatibility shims are installed at import and are specific to this platform:

  • torch.txda / txda device alias. FlagGems' TsingMicro backend and its Triton driver import torch_txda and call torch.device("txda"). torch_fl does not import the vendor package (it hard-links its own libtorch and claims the same PrivateUse1 key), and instead publishes a synthetic torch.txda module backed by the flagos device, plus a torch_txda stub.
  • Qwen-Image-2.1 rotary embedding. The 2.1 rope builds a complex64 operand, which FlagGems' Triton kernels cannot consume (cat/broadcast multiply raise KeyError: 'complex64'). The shim keeps the rotation real-valued. It is equivalent to the complex path to one fp32 ulp and is disabled with FLAGOS_DISABLE_QWENIMAGE_ROPE=1.

Profiler (optional)

Kernel timelines come from Kuiper's rocprofiler-sdk. The tracer is compiled only when ${KUIPER_ROOT}/tsm8-profiler/include/rocprofiler-sdk/buffer_tracing.h exists at configure time, and the build then also produces torch_fl/lib/libtorch_fl_profiler.so. That library is the rocprofiler tool; it links the profiler SDK and nothing else, because rocprofiler dlopens its tools while it is still initializing and must not pull the TsingMicro runtime in with them. The tracer inside libtorch_fl.so loads the tool at run time and turns its kernel dispatch records into kineto activities.

rocprofiler is inert unless the process is started with the environment below. A build without the SDK headers has neither the tracer nor the tool, and simply produces no device timeline.

Environment

Every shell that starts a profiled run needs all four variables:

export TSM_PROFILER_EN=1
export LD_PRELOAD=/usr/local/kuiper/tsm8-profiler/lib/libtsmprofiler-register.so:/usr/local/kuiper/tsm8-profiler/lib/libtsmprofiler-sdk.so
export ROCP_TOOL_LIBRARIES="$(python -c 'import importlib.util, os; s = importlib.util.find_spec("torch_fl"); print(os.path.join(os.path.dirname(s.origin), "lib", "libtorch_fl_profiler.so"))')"
export LD_LIBRARY_PATH=/usr/local/kuiper/lib:/usr/local/kuiper/tsm8-profiler/lib:<repo>/torch_fl/lib:$LD_LIBRARY_PATH
  • TSM_PROFILER_EN=1 makes hpgr intercept the tx* runtime API. Without it the runtime is never profiled and the tool receives no kernel dispatches.
  • LD_PRELOAD loads rocprofiler before the runtime so it can intercept the first tx* call. It cannot be added to an already-running process, so a run started without it cannot be profiled.
  • ROCP_TOOL_LIBRARIES names the tool rocprofiler loads. The command above prints the installed path without importing torch_fl.
  • LD_LIBRARY_PATH must contain the tool's directory so the tracer in libtorch_fl.so finds the same library. FLAGOS_TRACER_LIBRARY overrides that lookup.

Usage

import torch_fl
import torch

x = torch.randn(1024, 1024, device="flagos")
with torch.profiler.profile(
    activities=[
        torch.profiler.ProfilerActivity.CPU,
        torch.profiler.ProfilerActivity.PrivateUse1,
    ],
) as prof:
    y = torch.mm(x, x)
    y.cpu()

print(prof.key_averages().table(sort_by="self_device_time_total", row_limit=10))
prof.export_chrome_trace("tsm_trace.json")

The trace carries:

  • one CONCURRENT_KERNEL per dispatch on a per-stream row (Stream <n>), with the launch grid/block geometry;
  • one runtime slice per dispatch on the issuing CPU thread — the CPU-side launch window — and an ac2g flow arrow from it to the kernel;
  • per-operator device time as self_device_time_total, so both aten::mm and the mm_kernel behind it are attributable.

Only kernel dispatches are collected: memcpy and memset work does not appear as a device activity.

Diagnostics

export FLAGOS_TRACE=1
export FLAGOS_TRACER_LIBRARY=/path/to/libtorch_fl_profiler.so

FLAGOS_TRACE=1 prints the tool's and tracer's diagnostics to stderr, including the clock offset, the number of records drained, and how many activities were linked to CPU ops. If the tool cannot be loaded at all, the tracer prints one unconditional warning when the session starts, because an empty device timeline is otherwise indistinguishable from a workload that launched no kernels.

Known limitations

  • No CI and no published wheel. Every result in this guide is on-hardware measurement, not continuous validation.
  • No FlagTree or FlagCX pin. setup.py declares a plain triton>=3.5.1 for this platform, and FlagCX has no TsingMicro build in the version pins, so distributed collectives are not validated.
  • No complex64 on the device. The Triton backend has no complex pointer type; keep complex tensors on the CPU. The Qwen-Image-2.1 rope is the known workaround.
  • CPU fallback is expected. Any op outside the routing table runs on the host. Use FLAGOS_LOG=fallback to list them.

Testing

pytest tests/unit/test_tsingmicro_qwenimage21_rope.py -v
pytest tests/unit/test_device_tracer_selection.py -v