Skip to content

Release 2.1.0 - Integration with wiff converter - #82

Merged
ypriverol merged 87 commits into
mainfrom
dev
May 8, 2026
Merged

ypriverol merged 87 commits into
mainfrom
dev

Conversation

@ypriverol

@ypriverol ypriverol commented May 5, 2026 •

Copy link
Copy Markdown
Contributor

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the bigbio/quantmsdiann branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core pipelines lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

Summary by CodeRabbit

  • New Features

    • Optional PRIDE Archive pre-downloads; WIFF → mzML conversion; experimental QPX export (Parquet/MuData)
    • New CLI flags for DIA‑NN channel normalization
  • Improvements

    • Nextflow requirement bumped to ≥25.10.4
    • Default input type changed to raw; mzML reindexing default → false; new --mzml_convert control
    • Broader Bruker .d archive support; more reliable native RAW handling (staging behavior)
  • Documentation

    • Expanded contributing guide, parameters, usage, README, and changelog; input now requires .sdrf.tsv only

Copilot AI and others added 30 commits April 21, 2026 13:28
- Move the --local_input_type validation to the top of CREATE_INPUT_CHANNEL
  so it fails fast before any channel processing.
- Bump pipeline version to 2.1.0dev for the next development cycle.
Extend the allowed values for --local_input_type to include the Bruker
.d compressed archive variants that the pipeline already decompresses
(.d.tar, .d.tar.gz, .d.zip). The DECOMPRESS process handles all three
and emits a .d directory, so users can point --root_folder at folders
of .d.zip archives and set --local_input_type=d.zip without tripping
the validator.
Revert the Math.min wrappers on memory and cpus; resourceLimits already
caps allocations at 1900.GB / 48 cpus, so the manual clamping was
redundant and cluttered the expression.
Move input type validation before SDRF parsing
)

DIA-NN 2.1.0 added native Thermo .raw support on Linux, making the
ThermoRawFileParser step optional. Introduce --mzml_convert with an
auto default that picks TRFP vs. native based on --diann_version, plus
a version guard that rejects explicit --mzml_convert false for < 2.1.0.
Emit a warning when --local_input_type under --root_folder means no
.raw will reach the conversion branch (parameter is a no-op).
…tution

The previous substitution stripped only the last '.ext', so SDRFs that
contain compound extensions (e.g. '.d.zip' as in PXD065380 / test_dia_dotd,
or '.d.tar.gz') produced wrong paths when combined with --root_folder.

Examples of the bug:
  /root/sample.d.zip    + local_input_type=d.zip -> /root/sample.d.d.zip
  /root/sample.d.tar.gz + local_input_type=d     -> /root/sample.d.tar.d
  /root/sample.d.zip    + local_input_type=raw   -> /root/sample.d.raw

Strip the longest matching known raw-data extension first so the resulting
path targets the real file on disk.

https://claude.ai/code/session_01CtVkdv9sgCm6WUnfZxYrPr
…ailure

DIA-NN's native Thermo reader fails when .raw files are staged as
symlinks in the Nextflow work directory (Thermo SDK limitation).

Add a dynamic stageInMode closure to PRELIMINARY_ANALYSIS,
INDIVIDUAL_ANALYSIS, and ASSEMBLE_EMPIRICAL_LIBRARY that switches to
'copy' staging when native .raw mode is active (DIA-NN >= 2.1.0 with
mzml_convert != true), and uses 'symlink' otherwise.

Extract the shared mode-detection logic into VersionUtils.isNativeRawMode()
to avoid duplication across modules.

Agent-Logs-Url: https://github.com/bigbio/quantmsdiann/sessions/657483e0-5833-4c95-a84d-76a61f5f5908

Co-authored-by: ypriverol <52113+ypriverol@users.noreply.github.com>
Add --mzml_convert parameter for native Thermo .raw support
Populate the [Unreleased] section (bumped to 2.1.0dev by the version
manifest change) with entries for the --local_input_type default switch,
the schema enum / runtime validation, the Bruker .d archive variants,
the pride_codon_slurm resource cleanup, and the native-raw stageInMode
fix merged from #68.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Default `local_input_type` to `raw` and enforce supported local file formats
Manual targeted update instead of `git merge TEMPLATE`. The TEMPLATE
branch in this repo accumulated project commits, so the merge-base
contains project files; a three-way merge silently deletes them
(workflows/dia.nf, modules/local/diann/*, lib/BlockedFlags.groovy,
conf/diann_versions/*, etc.). Cherry-picking template files by hand
avoids that trap.

Phase 1 — low-risk, template-only:

- Bump nextflowVersion to !>=25.10.4 in nextflow.config manifest
- Add new template CI: .github/workflows/{nf-test,download_pipeline}.yml
- Add .devcontainer/, .vscode/settings.json, tower.yml
- Update .github/workflows/* (branch, clean-up, fix_linting, linting,
  linting_comment, template-version-comment) and .github/actions/*
  (get-shards, nf-test) to 4.0.2 versions: action pin bumps, switch
  from pre-commit to prek for linting, add `paths` input to nf-test
  composite actions
- Update .github/.dockstore.yml (NFL → nfl), ISSUE_TEMPLATE config
  (slack channel quantms → quantmsdiann), PULL_REQUEST_TEMPLATE
  (CONTRIBUTING.md path)
- Update subworkflows/nf-core/utils_* to 4.0.2 (emit syntax fix,
  imNotification removed, params.help handling, nf-schema 2.6.1,
  new utils_nfcore_pipeline tests)
- Update .pre-commit-config.yaml (prettier 3.8.3, nf-lint hook),
  .nf-core.yml (3.5.2 → 4.0.2)
- Hybrid .gitignore: keep project entries, add .lineage/
- Hybrid .prettierignore: TEMPLATE baseline + project SKILL.md/AGENTS.md

No project source files (workflows/, modules/local/, conf/diann_versions/,
conf/tests/, lib/, subworkflows/local/) touched.
Follow-up to Phase 1 (d71d60c) addressing nf-core 4.0.2 lint warnings
introduced or surfaced by the template sync.

- Move CONTRIBUTING.md from .github/ to docs/ (4.0.2 convention; PR
  template already references the new path) and add a quantmsdiann-
  specific section covering DIA-NN version handling, SDRF inputs, the
  local-modules layout, and the test-profile reference table
- Update modules.json SHAs for the two nf-core utility subworkflows
  whose files were refreshed in Phase 1: utils_nfcore_pipeline →
  a3fb7351, utils_nfschema_plugin → fdc08b8b (matches what TEMPLATE
  records, fixes check_local_copy lint)
- Remove subworkflows/local/utils_nfcore_quantms_pipeline/meta.yml —
  4.0.2 convention drops meta.yml for local subworkflows (fixes
  meta_name lint warning)
- README.md / ro-crate-metadata.json: bump nf-core template badge
  3.5.2 → 4.0.2, fix CONTRIBUTING.md links to point at docs/, refresh
  Nextflow badge ≥25.10.4 to match the manifest, and tighten the
  pipeline description (DIA-NN 2.3.2 row, plexDIA / Bruker mention,
  test-profile quickstart, sdrf-pipelines / pmultiqc citations)

The remaining lint hits (main_nf_script_outputs on PIPELINE_COMPLETION,
main_nf_include on the single-module input_check / create_input_channel
wrappers) are intentional design choices and accepted as warnings — see
PR description.
- Restore imNotification locally in subworkflows/local/utils_nfcore_quantms_pipeline/main.nf.
  nf-core 4.0.2 dropped this helper from utils_nfcore_pipeline (we picked up the trimmed
  version in phase 1), but the project still uses it via params.hook_url for Slack/Teams
  notifications and ships assets/slackreport.json + assets/adaptivecard.json as templates.
  Switched the import from imNotification to getWorkflowVersion (still in the upstream
  utility) and inlined the function in the FUNCTIONS block; fixes nextflow-lint
  "imNotification is not defined" error.
- Re-add assets/adaptivecard.json, assets/slackreport.json, modules.json to
  .prettierignore — TEMPLATE 4.0.2 removed those entries because the upstream template
  no longer ships these files, but the project still does. Without the ignore, prettier
  fails on the embedded Groovy <% %> template syntax.
- Apply prettier's column alignment to the test-profile table in docs/CONTRIBUTING.md
  to match the format the CI hook emits.
The TEMPLATE workflow runs tests/default.nf.test under -profile test,
but the project has no 'test' profile (only test_dia, test_dia_dotd,
test_dda, ...) and the test file itself is a stale fork-skeleton
(snapshots nf_core_quantms_software_mqc_versions.yml — the project
is named quantmsdiann, not quantms). The job fails on every run.

Wiring up proper nf-test integration needs:
- a 'test' profile or alias for one of the existing test_dia profiles
- a real tests/default.nf.test body for quantmsdiann
- snapshot generation and ongoing maintenance

That's a separate effort from "absorb the 4.0.2 template" — defer to
a follow-up PR. Existing CI (ci.yml, merge_ci.yml, extended_ci.yml)
keeps using profile-based DIA-NN testing as before.

The composite actions (.github/actions/get-shards, .github/actions/nf-test)
stay as-is — they were on dev pre-sync, and the 4.0.2 refresh is harmless
since nothing invokes them now.
Nextflow >= 25.10 (which we adopted via the manifest bump) emits
"Unrecognized config option 'validation.defaultIgnoreParams'" /
"validation.monochromeLogs" warnings on every run. Root cause:
nf-schema 2.5.1 doesn't register its config schema with Nextflow at
parse time, so Nextflow doesn't know the validation namespace is
plugin-owned and warns. nf-schema 2.6.1 registers properly.

This is the version the test subworkflow's nextflow.config was already
using (we updated subworkflows/nf-core/utils_nfschema_plugin/tests/
nextflow.config to 2.6.1 in phase 1) — main pipeline config was just
out of sync.

API compatibility check:
- paramsSummaryMap, paramsSummaryLog, validateParameters, paramsHelp:
  all stable across 2.5 → 2.6
- params.help boolean vs string handling: already covered by the
  utils_nfschema_plugin/main.nf update absorbed in phase 1

Drop the now-stale "nf-schema@2.5.1 (version specified in nextflow.config)"
comment in utils_nfcore_quantms_pipeline/main.nf so the version pin
isn't duplicated.
The Phase 1 manifest bump set nextflowVersion = '!>=25.10.4', but the
project CI workflows pin NXF_VER = "25.04.0" directly, so CI was running
with an older Nextflow than the manifest required. With the nf-schema
plugin bump to 2.6.1 (also from this PR), the older Nextflow couldn't
resolve the plugin in its repository — "Plugin nf-schema with version
@2.6.1 does not exist in the repository" — and CI failed at startup.

Fixes:

- ci.yml, extended_ci.yml, merge_ci.yml: bump all NXF_VER pins from
  25.04.0 to 25.10.4 (7 occurrences across the three workflows; the
  "latest-everything" matrix entries in extended_ci.yml are unchanged
  since they intentionally test the absolute newest)
- ro-crate-metadata.json: bump engineVersion to "!>=25.10.4" so the
  RO-Crate metadata matches the manifest
- AGENTS.md: bump three references (DSL2 syntax requirement, the
  NXF_VER export example, the "Minimum Nextflow" header)
- modules.json: add trailing newline so end-of-file-fixer doesn't
  re-introduce noise (linter ran earlier and dropped it; multi-line
  array formatting kept since it matches nf-core convention)
chore: nf-core 4.0.2 template sync
The pipeline always normalised inputs to .sdrf.tsv internally
(SAMPLESHEET_CHECK had a pandas-based .csv → .tsv conversion and a
.sdrf → .sdrf.tsv rename) but the schema let any of .sdrf, .tsv, or
.csv through. That left users on three different conventions for the
same file and required runtime gymnastics in a process whose only
job is to validate.

Make .sdrf.tsv the contract:

- nextflow_schema.json: tighten --input pattern from
  ^\S+\.(?:csv|tsv|sdrf)$ to ^\S+\.sdrf\.tsv$, switch mimetype to
  text/tab-separated-values, and rewrite description / help_text to
  reflect the single accepted extension. nf-schema enforces this at
  startup, so wrong inputs now fail fast with a clear validation
  error rather than silently going through the conversion path.

- modules/local/samplesheet_check/main.nf: drop the in-process
  CSV→TSV conversion and the .sdrf→.sdrf.tsv basename rewrite. The
  staged file is guaranteed to be .sdrf.tsv, so just hand it to
  quantmsutilsc as-is. Removes the pandas dependency from this
  process's hot path and shaves the only nontrivial logic.

- README.md, docs/usage.md, docs/parameters.md, ro-crate-metadata.json:
  update the prose to "must use the .sdrf.tsv extension" and rename
  example inputs from `sdrf.tsv` / `dda_sdrf.tsv` to
  `experiment.sdrf.tsv` / `dda_experiment.sdrf.tsv` for consistency.

- CHANGELOG.md: entry under [Unreleased] § Changed with migration
  note (rename existing samplesheets, convert CSV inputs upstream).

`nf-core pipelines schema lint` and `nf-core pipelines lint` are
green (177 passes, 0 failures, 8 pre-existing warnings carried from
the 4.0.2 template sync).
Shen-YuFei and others added 16 commits May 6, 2026 19:04
Adds a GUI-flag → params.yml translation table, a worked example, and
common-pitfalls callouts (auto mass accuracy on Bruker timsTOF,
--reanalyse already covered by the pipeline architecture, modifications
via SDRF, version pinning).

Addresses #85.
docs: add guide for reproducing DIA-NN GUI runs
Integrate QPX Parquet and MuData export into workflow
Bump the SDRF_PARSING module container from 0.1.3 to 0.1.4 and expand
the 2.1.0 CHANGELOG dependencies table to reflect the cumulative deps
changes since 2.0.0 (sdrf-pipelines, quantms-utils, qpx).

https://claude.ai/code/session_01K8s9C1vc5CNHT8zzR9BSUf

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs/usage.md (1)

595-595: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Leftover open question to @vdemichev should not ship in user-facing docs.

docs/usage.md is the user manual; an in-line "[@vdemichev]: would this approach work correctly…" question reads like an unresolved review thread. Either move it to an issue/PR comment or convert it to a definitive statement now that the integrated fine-tuning section above describes a working flow.

📝 Suggested removal
-This would be gated by a `--enable_fine_tuning` parameter. [`@vdemichev`](https://github.com/vdemichev): would this approach work correctly — using the empirical library from assembly as `--tune-lib`, then regenerating the in-silico library with the tuned models before proceeding to individual analysis? Or would you recommend a different integration point?
+This is gated by the `--enable_fine_tuning` parameter described above.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usage.md` at line 595, Remove the inline question addressed to
`@vdemichev` in the docs ("[`@vdemichev`]: would this approach work correctly —
...") from docs/usage.md; either delete that sentence entirely or replace it
with a definitive, user-facing sentence that describes the chosen behavior for
--enable_fine_tuning/--tune-lib (e.g., confirm that empirical library is used
for tuning and the in-silico library is regenerated before individual analysis).
If this needs follow-up, move the question to the PR/issue tracker instead of
leaving it in the user manual.
🧹 Nitpick comments (3)
modules/bigbio/qpx/main.nf (2)

32-32: ⚡ Quick win

Avoid referencing params.* directly inside a module.

Modules under modules/bigbio/ should be self-contained and not reach into pipeline-level params. Direct access to params.matrix_qvalue couples this module to the pipeline namespace and breaks if it's reused elsewhere or if the param is renamed. Promote matrix_qvalue to a proper module input (or surface it via task.ext.args like the rest of the module's args).

♻️ Suggested refactor
     input:
     path(diann_report)
     path(pg_matrix)
     path(sdrf)
     path(diann_log)
     val(project_accession)
+    val(qvalue_threshold)

     ...

     def acc_arg = project_accession ? "--project-accession ${project_accession}" : ''
-    def qvalue  = params.matrix_qvalue ?: 0.05
+    def qvalue  = qvalue_threshold ?: 0.05

Then pass params.matrix_qvalue from workflows/dia.nf at the call site.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@modules/bigbio/qpx/main.nf` at line 32, The module currently reads
pipeline-level params directly via "def qvalue = params.matrix_qvalue ?: 0.05"
which couples modules/bigbio/qpx/main.nf to the pipeline; change the module to
accept matrix_qvalue (or qvalue) as a module input/parameter (e.g., add a module
input named matrix_qvalue or qvalue with default 0.05) and update uses of the
qvalue variable inside the module to reference that input; then update the call
site in workflows/dia.nf to pass params.matrix_qvalue into the module (or
surface it via task.ext.args like the other args) so the module no longer
references params directly.

50-59: 💤 Low value

Embedded Python build step is not resilient to qpxc output schema drift.

Two latent issues in this inline Python:

  1. Dataset("qpx_output") and build_mudata(ds) are called unconditionally — if qpxc convert diann succeeded but produced an incomplete dataset (e.g., empty pg.parquet for low-ID runs), this Python step will raise inside the task and there's no set -e / set -u declared for the Python block (though set -o pipefail is set above, it doesn't apply here). The task will still fail correctly because Python exits non-zero, but the failure surface is split across two tools.
  2. mdata.write("${prefix}.h5mu") — if prefix ever contained a shell-meaningful character (slash, quote), Groovy interpolates it directly into the Python string. project_accession is normally a PXD ID, so this is unlikely in practice, but worth sanitising or asserting upstream.

Consider folding the MuData build into qpxc convert diann itself (so the module just calls one tool and reports versions), or wrapping the Python in a small CLI provided by the qpx package. Lower priority — current code is functional for canonical PRIDE accessions.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@modules/bigbio/qpx/main.nf` around lines 50 - 59, The inline Python step
unconditionally calls Dataset("qpx_output") and build_mudata(ds) and
interpolates ${prefix} directly into mdata.write, which makes the task fragile
to qpxc output schema drift and unsafe string injection; update the step to (1)
validate the dataset before building MuData by checking for required parquet
files / non-empty tables and bail with a clear message if missing, (2) wrap
Dataset(...) and build_mudata(...) in a try/except that logs the original
exception and exits non-zero, and (3) avoid direct Groovy interpolation into the
Python string by passing prefix via a sanitized environment variable or using a
safe basename/escape routine before calling mdata.write("${prefix}.h5mu");
alternatively consider moving MuData creation into the qpx tool (e.g., add a
subcommand to qpxc convert diann) so the Nextflow task invokes a single robust
CLI.
conf/tests/test_dia_qpx.config (1)

1-52: ⚡ Quick win

Profile duplicates test_dia instead of extending it.

The header comment says "Extends test_dia with --enable_qpx_export", but this config redeclares all the test_dia parameters (input/database/m/z bounds/peptide constraints, etc.) verbatim instead of composing on top of conf/tests/test_dia.config. Any future tweak to the base test profile will silently drift here.

Consider including the base profile and only layering QPX-specific overrides:

♻️ Suggested refactor
+includeConfig 'test_dia.config'
+
 process {
     resourceLimits = [
         cpus: 4,
         memory: '6.GB',
         time: '48.h'
     ]
+    // thermorawfileparser
+    withName: 'BIGBIO_QUANTMSDIANN:QUANTMSDIANN:FILE_PREPARATION:THERMORAWFILEPARSER' {
+        publishDir = [path: { "${params.outdir}/${task.process.tokenize(':')[-1].toLowerCase()}" }, pattern: "*.log" ]
+    }
 }

 params {
     config_profile_name        = 'Test profile for DIA + QPX export'
     config_profile_description = 'Extends test_dia to exercise QPX Parquet + MuData export.'
-
-    outdir = './results_dia_qpx'
-
-    // Input data (same fixture as test_dia)
-    input    = '...'
-    database = '...'
-    ...

     // QPX export
     enable_qpx_export    = true
     project_accession    = 'PXD026600'
 }
-
-process {
-    // thermorawfileparser
-    withName: 'BIGBIO_QUANTMSDIANN:QUANTMSDIANN:FILE_PREPARATION:THERMORAWFILEPARSER' {
-        publishDir = [path: { "${params.outdir}/${task.process.tokenize(':')[-1].toLowerCase()}" }, pattern: "*.log" ]
-    }
-}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@conf/tests/test_dia_qpx.config` around lines 1 - 52, This profile duplicates
the entire test_dia params instead of extending it; remove the repeated params
(cpus/memory/time, input, database, m/z and peptide bounds, normalize, max_mods,
etc.) and instead load/extend the existing test_dia profile and only override
QPX-specific keys: set params.enable_qpx_export = true, params.project_accession
= 'PXD026600' and adjust params.outdir and config_profile_name/description as
needed; keep the specialized publishDir override for the process withName
'BIGBIO_QUANTMSDIANN:QUANTMSDIANN:FILE_PREPARATION:THERMORAWFILEPARSER' but
remove any duplicated general params to avoid drift from the base test_dia
profile.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/usage.md`:
- Around line 697-720: The docs claim QPX export is "Experimental" but also "On
by default" while nextflow_schema.json sets enable_qpx_export true and
project_accession default null; fix this by choosing one of two coordinated
changes: either (A) flip the nextflow_schema.json default for enable_qpx_export
to false and update docs/usage.md to state QPX export is experimental and off by
default, or (B) keep the schema default true but update docs/usage.md to
explicitly state QPX export runs unconditionally (how to disable with
--enable_qpx_export false) and mark project_accession as required when enabled;
additionally implement the schema conditional in nextflow_schema.json using an
if/then that requires project_accession when enable_qpx_export is true (mirror
pridepy_download pattern) and update the parameter table in docs to reflect the
new default/requirement and disable instructions.
- Line 95: Update the example usages that currently show "--input sdrf.tsv" to
use the required ".sdrf.tsv" extension instead (change the input filename string
from "sdrf.tsv" to ".sdrf.tsv") in both the PRIDE Archive Download example and
the QPX Export quickstart; ensure the text and any command examples referencing
the "--input" flag and the literal "sdrf.tsv" string are replaced so they match
the documented validation rule that accepts ".sdrf.tsv".

In `@modules/bigbio/qpx/main.nf`:
- Around line 7-9: The container image reference uses the bare Docker Hub name
'biocontainers/qpx:1.0.2--pyhdfd78af_1' which doesn't exist on Docker Hub;
update the Docker image path used in the container string (the conditional that
checks workflow.containerEngine and task.ext.singularity_pull_docker_container)
to use the Quay registry by replacing the Docker image name with
'quay.io/biocontainers/qpx:1.0.2--pyhdfd78af_1' while leaving the Singularity
Galaxy depot URL unchanged.

In `@nextflow_schema.json`:
- Around line 137-140: The CLI option reindex_mzml now defaults to false but the
usage docs still state default=true; locate all occurrences of the option name
reindex_mzml in the usage documentation and change the prose from "default: true
/ enabled by default" to "default: false / disabled by default" and update the
guidance to match the schema/help_text (i.e., enable only when supplying
pre-built/unindexed mzML files); if the default flip is intentional for the
2.1.0 release, add a short migration note in the changelog/migration notes
stating that reindex_mzml default changed to false in 2.1.0 and instructions for
users who relied on the previous default.
- Around line 596-610: Change the qpx_export block so enable_qpx_export defaults
to false and add an if/then schema guard that requires project_accession when
enable_qpx_export is true (mirror the existing
pridepy_download/pride_archive_download if/then pattern): update
"$defs.qpx_export" to set "enable_qpx_export" default to false, then add an
"if": { "properties": { "enable_qpx_export": { "const": true } } } with a
corresponding "then" that requires or references "project_accession" (use the
same reference/key used by pridepy_download/pride_archive_download) and produces
an error if missing; also verify the cross-section reference to
project_accession resolves in the flattened parameter model or move
project_accession to a shared defs section.

In `@workflows/dia.nf`:
- Around line 327-342: The branch controlled by enable_qpx_export currently runs
by default and can call QPX_EXPORT with an empty params.project_accession
causing silent fallback; either (A) change nextflow_schema.json to set
enable_qpx_export default to false, or (B) add a schema-level if/then requiring
project_accession when enable_qpx_export is true and add a runtime guard in the
workflow before calling QPX_EXPORT that checks if (params.enable_qpx_export) and
params.project_accession is empty then call error("enable_qpx_export requires a
non-empty project_accession"); update references to enable_qpx_export,
project_accession and QPX_EXPORT to locate and fix the code.
- Line 330: The pipeline currently re-reads params.input into ch_sdrf_original
for QPX_EXPORT, which breaks URI substitution done by
SDRF_PARSING/create_meta_channel_grouped; change QPX_EXPORT to consume the
processed design channel emitted by SDRF_PARSING (SDRF_PARSING.out.ch_expdesign)
instead of params.input (replace uses of ch_sdrf_original where passed to
QPX_EXPORT), or alternatively modify SDRF_PARSING to also emit the original SDRF
path under a distinct channel if qpxc truly needs the unaltered file; update
references to ch_sdrf_original and any downstream inputs to use the chosen
channel (SDRF_PARSING.out.ch_expdesign or the newly emitted original-SDRF
channel).

---

Outside diff comments:
In `@docs/usage.md`:
- Line 595: Remove the inline question addressed to `@vdemichev` in the docs
("[`@vdemichev`]: would this approach work correctly — ...") from docs/usage.md;
either delete that sentence entirely or replace it with a definitive,
user-facing sentence that describes the chosen behavior for
--enable_fine_tuning/--tune-lib (e.g., confirm that empirical library is used
for tuning and the in-silico library is regenerated before individual analysis).
If this needs follow-up, move the question to the PR/issue tracker instead of
leaving it in the user manual.

---

Nitpick comments:
In `@conf/tests/test_dia_qpx.config`:
- Around line 1-52: This profile duplicates the entire test_dia params instead
of extending it; remove the repeated params (cpus/memory/time, input, database,
m/z and peptide bounds, normalize, max_mods, etc.) and instead load/extend the
existing test_dia profile and only override QPX-specific keys: set
params.enable_qpx_export = true, params.project_accession = 'PXD026600' and
adjust params.outdir and config_profile_name/description as needed; keep the
specialized publishDir override for the process withName
'BIGBIO_QUANTMSDIANN:QUANTMSDIANN:FILE_PREPARATION:THERMORAWFILEPARSER' but
remove any duplicated general params to avoid drift from the base test_dia
profile.

In `@modules/bigbio/qpx/main.nf`:
- Line 32: The module currently reads pipeline-level params directly via "def
qvalue = params.matrix_qvalue ?: 0.05" which couples modules/bigbio/qpx/main.nf
to the pipeline; change the module to accept matrix_qvalue (or qvalue) as a
module input/parameter (e.g., add a module input named matrix_qvalue or qvalue
with default 0.05) and update uses of the qvalue variable inside the module to
reference that input; then update the call site in workflows/dia.nf to pass
params.matrix_qvalue into the module (or surface it via task.ext.args like the
other args) so the module no longer references params directly.
- Around line 50-59: The inline Python step unconditionally calls
Dataset("qpx_output") and build_mudata(ds) and interpolates ${prefix} directly
into mdata.write, which makes the task fragile to qpxc output schema drift and
unsafe string injection; update the step to (1) validate the dataset before
building MuData by checking for required parquet files / non-empty tables and
bail with a clear message if missing, (2) wrap Dataset(...) and
build_mudata(...) in a try/except that logs the original exception and exits
non-zero, and (3) avoid direct Groovy interpolation into the Python string by
passing prefix via a sanitized environment variable or using a safe
basename/escape routine before calling mdata.write("${prefix}.h5mu");
alternatively consider moving MuData creation into the qpx tool (e.g., add a
subcommand to qpxc convert diann) so the Nextflow task invokes a single robust
CLI.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1e6b8f22-090f-4cce-bda6-ced46b8decd8

📥 Commits

Reviewing files that changed from the base of the PR and between d26a7f0 and 1918a71.

⛔ Files ignored due to path filters (1)
  • modules/bigbio/qpx/tests/main.nf.test.snap is excluded by !**/*.snap
📒 Files selected for processing (16)
  • .github/workflows/extended_ci.yml
  • CHANGELOG.md
  • conf/modules/shared.config
  • conf/tests/test_dia_qpx.config
  • docs/output.md
  • docs/usage.md
  • modules.json
  • modules/bigbio/qpx/environment.yml
  • modules/bigbio/qpx/main.nf
  • modules/bigbio/qpx/meta.yml
  • modules/bigbio/qpx/tests/main.nf.test
  • modules/bigbio/qpx/tests/nextflow.config
  • modules/local/sdrf_parsing/main.nf
  • nextflow.config
  • nextflow_schema.json
  • workflows/dia.nf
✅ Files skipped from review due to trivial changes (4)
  • modules/bigbio/qpx/environment.yml
  • modules/bigbio/qpx/meta.yml
  • modules/bigbio/qpx/tests/main.nf.test
  • modules.json
🚧 Files skipped from review as they are similar to previous changes (6)
  • modules/local/sdrf_parsing/main.nf
  • docs/output.md
  • conf/modules/shared.config
  • .github/workflows/extended_ci.yml
  • CHANGELOG.md
  • nextflow.config

Comment thread docs/usage.md Resolved
Comment thread docs/usage.md Resolved
Comment on lines +7 to +9
container "${workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container
? 'https://depot.galaxyproject.org/singularity/qpx:1.0.2--pyhdfd78af_1'
: 'biocontainers/qpx:1.0.2--pyhdfd78af_1'}"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Galaxy depot
curl -sIL "https://depot.galaxyproject.org/singularity/qpx:1.0.2--pyhdfd78af_1" | head -n1

# Quay.io biocontainers
curl -s "https://quay.io/api/v1/repository/biocontainers/qpx/tag/" | jq '.tags[] | select(.name | test("1\\.0\\.2"))'

# Docker Hub fallback
curl -s "https://hub.docker.com/v2/repositories/biocontainers/qpx/tags/?page_size=100" | jq '.results[].name' | rg '1\.0\.2'

Repository: bigbio/quantmsdiann

Length of output: 724


Use quay.io/biocontainers/qpx:1.0.2--pyhdfd78af_1 for the Docker image path.

The bare biocontainers/qpx:1.0.2--pyhdfd78af_1 form resolves to Docker Hub, where this image is not available. The correct registry is Quay.io, where both the exact Docker image tag and the corresponding Singularity image on Galaxy depot are confirmed to exist.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@modules/bigbio/qpx/main.nf` around lines 7 - 9, The container image reference
uses the bare Docker Hub name 'biocontainers/qpx:1.0.2--pyhdfd78af_1' which
doesn't exist on Docker Hub; update the Docker image path used in the container
string (the conditional that checks workflow.containerEngine and
task.ext.singularity_pull_docker_container) to use the Quay registry by
replacing the Docker image name with
'quay.io/biocontainers/qpx:1.0.2--pyhdfd78af_1' while leaving the Singularity
Galaxy depot URL unchanged.

Comment thread nextflow_schema.json Resolved
Comment thread nextflow_schema.json Resolved
Comment thread workflows/dia.nf Resolved
Comment thread workflows/dia.nf
qpx_dataset_ch = Channel.empty()
mudata_ch = Channel.empty()
if (params.enable_qpx_export) {
ch_sdrf_original = channel.fromPath(params.input, checkIfExists: true).first()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Locate where the SDRF is parsed/rewritten and whether a parsed SDRF channel exists
fd -e nf | xargs rg -nP '\bSDRF_PARSING\b|sdrf.*parsing|out\.sdrf|emit:\s*sdrf' -C2
rg -nP '\bpridepy_download\b' -C3

Repository: bigbio/quantmsdiann

Length of output: 6894


🏁 Script executed:

# Check SDRF_PARSING module outputs
cat -n modules/local/sdrf_parsing/main.nf | head -50

Repository: bigbio/quantmsdiann

Length of output: 1570


🏁 Script executed:

# Check the context around line 330 in workflows/dia.nf
sed -n '310,350p' workflows/dia.nf | cat -n

Repository: bigbio/quantmsdiann

Length of output: 1842


🏁 Script executed:

# Find references to ch_sdrf_original and QPX_EXPORT
rg -nP 'ch_sdrf_original|QPX_EXPORT' -A 3 -B 1

Repository: bigbio/quantmsdiann

Length of output: 3108


🏁 Script executed:

# Check if CREATE_INPUT_CHANNEL or SDRF_PARSING handles SDRF rewriting
cat -n subworkflows/local/create_input_channel/main.nf

Repository: bigbio/quantmsdiann

Length of output: 12835


🏁 Script executed:

# Check QPX_EXPORT module to see what it expects
cat -n modules/bigbio/qpx/main.nf | head -80

Repository: bigbio/quantmsdiann

Length of output: 2900


🏁 Script executed:

# Search for any SDRF rewriting or emission logic
rg -nP 'parse_sdrf|sdrf.*rewrite|comment\[file uri\]' -A 2 -B 2

Repository: bigbio/quantmsdiann

Length of output: 7220


Pass the processed design file from SDRF_PARSING to QPX_EXPORT instead of re-reading params.input.

Line 330 re-reads the original user-supplied SDRF directly from params.input, but SDRF_PARSING has already converted it. When --pridepy_download is used, CREATE_INPUT_CHANNEL substitutes remote URIs with local file paths (lines 111–124 in create_meta_channel_grouped), and DIA-NN processes files using those local paths. The DIA-NN report's run identifiers will match the local substituted paths, but qpxc convert diann --sdrf-file will receive the original SDRF with unsubstituted remote URIs, causing a potential mismatch.

Confirm whether qpxc should consume the original SDRF or the design file emitted by SDRF_PARSING, then either:

  • Pass SDRF_PARSING.out.ch_expdesign (the design file) to QPX_EXPORT, or
  • Have SDRF_PARSING emit the original SDRF it processes so downstream tools receive the same file.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@workflows/dia.nf` at line 330, The pipeline currently re-reads params.input
into ch_sdrf_original for QPX_EXPORT, which breaks URI substitution done by
SDRF_PARSING/create_meta_channel_grouped; change QPX_EXPORT to consume the
processed design channel emitted by SDRF_PARSING (SDRF_PARSING.out.ch_expdesign)
instead of params.input (replace uses of ch_sdrf_original where passed to
QPX_EXPORT), or alternatively modify SDRF_PARSING to also emit the original SDRF
path under a distinct channel if qpxc truly needs the unaltered file; update
references to ch_sdrf_original and any downstream inputs to use the chosen
channel (SDRF_PARSING.out.ch_expdesign or the newly emitted original-SDRF
channel).

claude and others added 9 commits May 8, 2026 09:04
- docs/usage.md: replace `sdrf.tsv` with `experiment.sdrf.tsv` in pridepy
  and QPX examples so they pass the new `.sdrf.tsv` schema validation.
- docs/usage.md: align `--reindex_mzml` description with the flipped
  schema default (now `false`); update both occurrences.
- modules/bigbio/qpx: point Docker container at
  `quay.io/biocontainers/qpx:1.0.2` (the bare `biocontainers/...`
  form resolves to Docker Hub, where this tag is not published).
- nextflow_schema.json + nextflow.config: flip `enable_qpx_export`
  default to `false` (consistent with its "Experimental" labelling) and
  add an `if/then` guard requiring `project_accession` whenever it is
  enabled, mirroring the `pridepy_download` contract.
- workflows/dia.nf: fail fast with `error()` if QPX export is enabled
  without `project_accession`, so the QPX prefix never silently falls
  back to "diann" and PRIDE/PX provenance is never lost.
nf-core lint's schema_lint walks every entry in the top-level allOf
assuming each is a $ref, so a bare if/then guard makes it crash with
KeyError: '$ref'. The workflows/dia.nf early error() already enforces
that --project_accession is set when --enable_qpx_export is on, which
is sufficient to prevent the silent 'diann' fallback prefix.
The qpx module is pinned via modules.json to bigbio/modules at
69529f0bf418a82b75c4d29255f9f73f8ed84dbf, so editing it locally fails
nf-core lint's check_local_copy. The biocontainers/qpx -> quay.io fix
needs to land upstream in bigbio/modules and then be pulled in via
`nf-core modules update qpx`.
Fix QPX export: disable by default and require project_accession
Update WIFF support and update documentation for QPX
@ypriverol
ypriverol merged commit 9af0b39 into main May 8, 2026
53 of 54 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants