Given a first frame and a path for each object, TrajLoc generates a video in which every object follows its own path, even in crowded scenes where paths cross. Each object gets its own text token, and in every transformer block, that token's attention is replaced by a Gaussian at the object's target location, which locks each trajectory to the right object.
TrajLoc on CogVideoX-5B (top) and Wan2.1-14B (bottom), against the ground truth and prior methods.
| Model | Base model | Trained on | Weights |
|---|---|---|---|
| TrajLoc-CogVideoX-5B-I2V | CogVideoX-5B-I2V | SA-V, 28,500 steps | Hugging Face |
| TrajLoc-Wan2.1-I2V-14B-480P | Wan2.1-I2V-14B-480P | SA-V, 49,000 steps | Hugging Face |
git clone --recursive https://github.com/Sela-Omer/traj-loc trajloc && cd trajloc
scripts/env/install.sh && source .venv/bin/activateRequires Linux, an NVIDIA driver for CUDA 12.4, and uv
(curl -LsSf https://astral.sh/uv/install.sh | sh).
trajloc generate wan --image assets/examples/pigs/image.png \
--scene assets/examples/pigs/scene.json --run_name pigsassets/examples holds four scenes from the project page, each a first frame and its
scene.json: boats, motorbike_trick, goldfish and pigs. Use cogvideox for the other model,
and --checkpoint for one of your own.
To animate your own image, draw its scene in tools/scene_editor.html: open it
in a browser (offline, nothing to install), draw each object's path on the first frame, write the
prompt with each object's <traj_i> after its noun, and export scene.json.
scripts/download_data.sh movebench && GPUS=0,1,2,3 scripts/prepare_data.sh movebench
scripts/download_data.sh magicbench && GPUS=0,1,2,3 scripts/prepare_data.sh magicbenchMoveBench and MagicBench download from Hugging Face.
trajloc infer wan --run_name wan
trajloc infer cogvideox --run_name cogvideoxEvery MoveBench and MagicBench clip is generated with the released model into
runs/infer/<backbone>/<run_name>/. To split a run across GPUs, start one process per GPU with
--device cuda:<i> --shard <i> --num_shards <n>.
SAV_LINKS=<sa-v links file> scripts/download_data.sh sav
GPUS=0,1,2,3 scripts/prepare_data.sh savSA-V requires accepting Meta's license;
Meta then sends the links file to pass as SAV_LINKS. Its
SVG2 scene graphs download from Hugging
Face.
trajloc train cogvideox --config configs/cogvideox.yaml --run_name my_cogvideox
trajloc train wan --config configs/wan.yaml --run_name my_wanEach command trains on one GPU with the released model's config. The released models are the
checkpoints at steps 28,500 (CogVideoX) and 49,000 (Wan). Checkpoints are written to
runs/train/<backbone>/<run_name>/checkpoints/checkpoint-<step>/; pass one to trajloc infer or
trajloc generate with --checkpoint.
@inproceedings{sela2026trajloc,
title = {TrajLoc: Trajectory-Attention Localization for
Multi-Object Motion Control},
author = {Sela, Omer and Huberman-Spiegelglas, Inbar and
Rotman, Michael and Benaim, Sagie and Ben-Cohen, Avi},
booktitle = {Advances in Neural Information Processing Systems},
year = {2026},
note = {To appear},
eprint = {2607.00861},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2607.00861}
}The code and the TrajLoc weights are released under the Apache License 2.0. TrajLoc-CogVideoX-5B-I2V is also subject to the CogVideoX License, and data preparation uses CoTracker3 (CC BY-NC 4.0). Third-party assets in this repository are listed in NOTICE.
TrajLoc builds on CogVideoX, Wan2.1, DiffSynth-Studio, SAM 2, CoTracker and Qwen2.5-VL. The models are trained on SA-V, with SVG2's scene graphs, and evaluated on MoveBench and MagicBench. The example scenes are first frames and tracks from DAVIS and MoveBench.

