[albertvillanova]

This PR removes trl.experimental.xpo (XPOTrainer, XPOConfig), added in #1943 and moved to trl.experimental in #4485.

Motivation

Reporting the inputs discussed in #7182:

• Usage. 17 non-CPU users over the last two months. Eighteen models carry both the xpo and trl tags, none created this year. Nobody has spoken up for it in [RFC] Criteria for removing code from trl.experimental #7182.
• External issues and pull requests. Nine issues and eight pull requests from outside the maintainer team in the last twelve months. Almost all are code-reading findings rather than reports from training runs: six of the issues were filed on a single day by one account and are three distinct problems worded twice, and the pull requests are drive-by fixes sweeping Online DPO, XPO and Nash-MD together. The exception is XPO Trainer data update step #7007, a question about whether the implementation covers step 5 of Algorithm 1 in the paper, which is the only item here that came from someone actually working with the trainer.
• Subsumed. No. No stable trainer covers exploratory preference optimization, so this removes a capability rather than consolidating one.
• Cost. 569 lines of trainer and config. Fourteen commits have touched it in the last year and none was dedicated work on XPO itself.
• Downstream consumers. None. XPOTrainer is imported by no major training framework.
• Owner. None has come forward.
• Age. Available since 2024-09, in trl.experimental since 2025-11-06, deprecated import path dropped 2026-02-10. Long enough to have been found.

Solution

trl.experimental carries no stability guarantee, so there is no deprecation cycle. The code stays in the git history.

The paper index entry stays and points at the last release that shipped the implementation, following the discussion in #7182.

Changes

• Remove XPOTrainer, XPOConfig and their tests
• Delete the XPO trainer doc page and drop it from the toctree, the trainer index, the dataset format table and the vLLM integration page
• Drop XPOTrainer from the telemetry allowlist
• Keep the XPO paper index entry, pointing at the last release shipping the trainer

---

Note

Medium Risk
Removes a public experimental API and training capability with no in-library substitute, which will break any remaining trl.experimental.xpo imports after upgrade.

Overview
Removes the experimental Exploratory Preference Optimization stack (trl.experimental.xpo: XPOTrainer, XPOConfig), its tests, and the dedicated xpo_trainer doc page.

Documentation and integration guides no longer list XPO: experimental toctree, main trainer index, dataset-format compatibility table, and all vLLM examples/options for XPO are dropped. The paper index entry for XPO remains but now states the trainer shipped through v1.13 and lives only in git history (the XPOConfig usage snippet is removed). XPOTrainer is also removed from the telemetry trainer allowlist in base_trainer.py.

There is no replacement trainer for exploratory preference optimization in this change—callers must pin an older TRL release or recover the code from history.

^{Reviewed by Cursor Bugbot for commit 34985a8. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

LGTM.