Code for
Diffusion-MPC in Discrete Domains: Feasibility Constraints, Horizon Effects, and Critic Alignment
Case Study: Tetris
📄 Paper: https://arxiv.org/abs/2603.02348
This repository provides a complete implementation of diffusion-based planning in discrete domains, using Tetris as a case study.
It includes:
- Offline dataset generation (teacher policy)
- MaskGIT-style diffusion policy training
- Diffusion + MPC reranking for decision-time planning
- Scalable experiment pipeline (Lightning-ready)
Clone the repository:
git clone git@github.com:KevinChunye/Diffusion-Tetris.git
cd Diffusion-TetrisRun the full pipeline:
# 1. Generate dataset
python -m experiments.pipeline dataset --config configs/dataset_gen.yaml --output_dir runs/dataset/ds_v1
# 2. Train diffusion model
python -m experiments.pipeline train --config configs/diffusion_train.yaml --output_dir runs/train/tr_v1
# 3. Evaluate with MPC reranking
python -m experiments.pipeline eval --config configs/eval_run.yaml --output_dir runs/eval/ev_v1Every bot (random, greedy heuristic, beam search, CNN-DQN, diffusion-MPC, or an open LLM on
Tensormesh serverless) plays through one interface in harness/. The same episode seed means the
same piece sequence for every bot, so GIFs and metrics compare bots on identical games.
GIFs use the gym's colored renderer (tetris_render.py, opt in with TetrisGym().enable_visual() and
env.render(info, mode="pretty")). Each board has:
- standard tetromino colors;
- pieces that fall to a landing ghost;
- full rows that flash before they clear;
- a NEXT box and score/lines;
- a GAME OVER overlay.
Comparison GIFs keep every board on the same piece and verify, at every move, that all agents face
the same current and next piece. Use --style classic for the old matplotlib frames.
# four agents on one logged game (iteration 6, seed 1001): three LLMs replayed exactly + beam bot live
python -m harness.compare_gif --steps runs/explore/iter06/steps.csv --seed 1001 --pieces 100 \
--arms gemma-4-31B/direct,gpt-oss-20b/low,DeepSeek-V4-Flash/direct --bots beam \
--out runs/explore/iter06/four_agents_seed1001.gif# train (CPU-friendly defaults, wraps the existing trainers)
python -m harness.train dqn --episodes 300 --max_steps 500 --runs_dir runs/train
python -m harness.train diffusion --episodes 100 --epochs 5 --out_dir runs/train/diffusion
# deploy: play seeded episodes, print score/lines/pieces, record a GIF of the first seed
python -m harness.play --bot beam --seeds 0,1,2 --pieces 100 --gif runs/play/beam.gif
python -m harness.play --bot dqn:runs/train/<run>/checkpoint.pt --seeds 0 --gif runs/play/dqn.gif
python -m harness.play --bot llm:openai/gpt-oss-20b --history stateless --seeds 1000 --pieces 50 --gif runs/play/oss20b.gif
# several bots side by side in one GIF (same pieces)
python -m harness.play --compare --bot greedy,beam,llm:openai/gpt-oss-20b --seeds 1000 --pieces 60 --gif runs/play/cmp.gif
# replay logged LLM pilots (llm/run_pilot.py) as side-by-side GIFs, one column per arm/model
python -m harness.replay --steps runs/explore/iter01/steps.csv --seed 1000LLM bots need TENSORMESH_API_KEY in the environment; add --mock to run them offline.
Paper draft: Not Just the Weights: What Tetris Reveals About Serving Open-Weight Agents
(source; python paper/build.py rebuilds every figure from runs/explore/ and the PDF).
For the proposed open-weight Tetris benchmark, see the
research protocol and experiment plan and
configs/benchmark/pilot.yaml. The protocol separates
gameplay quality, frozen-state prefix-cache replay, and serving-load/deadline experiments.
New runs distinguish model decisions from fallbacks and missing telemetry from reported zero hits.
Use python -m llm.cache_replay --help for paired cache probes and
python -m llm.benchmark_report --help for episode-level summaries. Offline --mock runs
validate the harness only; they are not real-model benchmark results.
llm/ uses Tetris as a controllable long-horizon workload for serverless open-model inference, with
dense per-step regret as the quality signal. See notes/tensormesh_probe.md (platform probe),
notes/exploration_log.md (iterations), and notes/exploration_summary.md (findings).
Long-context interference study (iteration 8). Do a tenant's long-context requests delay its short
decision requests on a hosted endpoint, and does client-side admission help? See
notes/interference_study.md (results, with LIVE and SIMULATED evidence
kept separate) and notes/interference_novelty_matrix.md
(overlap audit and preregistration).
python -m llm.interference_pilot sim --config configs/interference/pilot_v2.yaml --out <fresh dir> # SIMULATED
python -m llm.interference_pilot live --config configs/interference/pilot_v2.yaml --out <fresh dir> # LIVE, <= $1 est.
python -m llm.interference_analysis <run dir> # paired bootstrappython -m llm.probe_tensormesh --out_dir runs/explore/probe # Phase 0 probe
python -m llm.run_pilot --config configs/explore/iter01.yaml [--mock] # one pilot (paired seeds)
python -m llm.analyze --dir runs/explore/iter01 --kind history # table + figure
python -m pytest tests -q # offline tests (no API calls)The framework is structured into three stages:
- Dataset Generation — collect expert trajectories
- Training — learn diffusion-style policy (MaskGIT denoiser)
- Evaluation — MPC-style reranking with critic
Key features:
- Resumable runs (
--resume) - Structured outputs (
manifest.json) - Unified artifact management
To run large-scale jobs:
bash scripts/lightning_clone_ssh.shSubmit batch experiments:
python -m experiments.submit_lightning_jobs --plan_path runs/run_plans/<timestamp>_plan.yaml --teamspace <teamspace> --cluster <cluster> --machine <machine> --gpu <gpu>configs/ # Experiment configs
experiments/ # Pipeline + training/eval logic
scripts/ # Setup and automation
runs/ # Outputs (datasets, models, eval)
notes/ # Additional documentation
If you find this work useful, please cite:
@article{wang2026diffusion_tetris,
title={Diffusion-MPC in Discrete Domains: Feasibility Constraints, Horizon Effects, and Critic Alignment},
author={Wang, Haochuan},
year={2026},
eprint={2603.02348},
archivePrefix={arXiv}
}Additional details:
notes/lightning_runbook.md
