Skip to content

fix(diann): benchmark-review bug fixes (--kb, QuantUMS, mass-accuracy, --no-prot-inf) - #104

Merged
ypriverol merged 9 commits into
devfrom
feat/bug-fixes-from-benchmarks
Jun 9, 2026
Merged

ypriverol merged 9 commits into
devfrom
feat/bug-fixes-from-benchmarks

Conversation

@ypriverol

Copy link
Copy Markdown
Contributor

Summary

Bug fixes for the DIA-NN modules, found while benchmarking quantmsdiann against ProteoBench (DIA-NN 1.8.1 / 2.5.1 / 2.5.1-enterprise) and reviewed with DIA-NN's author (V. Demichev). These address cross-version comparison confounders and a few configuration issues on the Enterprise / 2.5.x path.

Changes

  1. --kb (Enterprise Knowledge Base) on the search passes — it was only wired into PRELIMINARY_ANALYSIS (quick mass-acc calibration), not where identification happens. Added to ASSEMBLE_EMPIRICAL_LIBRARY (first-pass search) and INDIVIDUAL_ANALYSIS (per-file search), gated by params.enable_kb.

  2. QuantUMS on by default for DIA-NN ≥ 1.9.2 — params.quantums defaulted to false, forcing --direct-quant and disabling QuantUMS (DIA-NN's recommended quantification). Flipped to true; the --direct-quant opt-out stays version-gated, so 1.8.1 is unaffected.

  3. --quant-sel-runs 20 default + version-gated QuantUMS tuning flags — QuantUMS over thousands of runs is slow; auto-select 20 runs to optimise its params. --quant-sel-runs/--quant-train-runs/--quant-params now only apply when QuantUMS is active (params.quantums && DIA-NN ≥ 1.9.2), so they're never passed to versions that don't support them.

  4. Mass-accuracy resolution (the main confounder) — the per-file search mass accuracy was read from the ASSEMBLE_EMPIRICAL_LIBRARY "Averaged recommended settings" log line. On DIA-NN ≥ 2.5.0 that step combines --use-quant with --individual-mass-acc, which DIA-NN flags as "strongly not recommended" and then prints a default MS1 = 20 ppm (vs the genuine ~3 ppm), so 2.5.x searched at MS1 = 20 ppm while 1.8.1 used 3 ppm. Now:

    • DIA-NN ≥ 2.5.0: read the optimised values from each PRELIMINARY log (Recommended MS1 mass accuracy setting / Suggested mass accuracy / Scan window radius) and average across runs.
    • DIA-NN < 2.5.0: keep the (correct for that format) ASSEMBLE averaged line.
    • Resolution priority — mass_acc_automatic=true: calibration > SDRF ppm > defaults; mass_acc_automatic=false: SDRF ppm > defaults (calibration excluded). Defaults updated 15/15 → 20/10 ppm.
    • When calibration can't be parsed, a 0,0,0 sentinel makes INDIVIDUAL_ANALYSIS fall back to the previous reliable value (SDRF if annotated, else params) instead of silently using defaults.
  5. --no-prot-inf in FINAL_QUANTIFICATION — protein groups are already inferred when the empirical library is built; reuse that inference in the final quant instead of re-inferring with --relaxed-prot-inf, so the final report matches the library.

  6. Docs — record the decision to keep pg_level = 2.

Validation

  • nextflow lint is clean (only the expected lib/ class false positives).
  • Needs a -profile test run to validate the new preliminary-log parsing / averaging / sentinel path at runtime.

Not in this PR

  • Stray --mass-acc 20 / --mass-acc-ms1 10 written into diann_config.cfg by quantms-utils (dianncfg) — inert here (not extracted into mod_flags); to be fixed in quantms-utils.
  • Empty-signatures warning on every 2.5.x empirical library (all precursors) — open question for DIA-NN upstream.

Reported/reviewed by V. Demichev.

ypriverol added 8 commits June 8, 2026 21:59
--kb was only added to PRELIMINARY_ANALYSIS, which is the quick mass-accuracy
calibration pass — not where identification happens. The Knowledge Base
boosts IDs and belongs on the actual search steps. Add --kb (gated by
params.enable_kb) to ASSEMBLE_EMPIRICAL_LIBRARY (first-pass search that builds
the empirical library) and INDIVIDUAL_ANALYSIS (per-file search); clarify the
PRELIMINARY_ANALYSIS comment. Found during the ProteoBench benchmark review
(reported by V. Demichev).
params.quantums defaulted to false, which forced --direct-quant on every
DIA-NN >= 1.9.2 run and thereby disabled QuantUMS (DIA-NN's recommended
quantification), affecting quantification accuracy. Flip the default to true.
The --direct-quant opt-out remains version-gated in FINAL_QUANTIFICATION, so
DIA-NN 1.8.1 (no QuantUMS) is unaffected; only >= 1.9.2 now uses QuantUMS.
…s by version

QuantUMS over thousands of runs (e.g. ProCan, single-cell sweeps) is very slow
when it optimises params over every run. Default --quant-sel-runs to 20 (DIA-NN
auto-selects 20 runs to optimise QuantUMS params) so QuantUMS scales now that it
is on by default. Also gate --quant-sel-runs/--quant-train-runs/--quant-params
on QuantUMS actually being active (params.quantums && DIA-NN >= 1.9.2) so they
are never passed to versions that don't support them (e.g. 1.8.1).
…A-NN >= 2.5.0

The mass accuracy / scan window for the per-file search were parsed from the
ASSEMBLE_EMPIRICAL_LIBRARY 'Averaged recommended settings' line. On DIA-NN >= 2.5.0
that step combines --use-quant with --individual-mass-acc, which DIA-NN flags as
'strongly not recommended' and then prints a default MS1 = 20 ppm (far too wide for
e.g. Astral) decoupled from the genuine per-run optimisation -- so the per-file
search ran at MS1 = 20 ppm while 1.8.1 (no such warning) correctly used ~3 ppm,
confounding cross-version comparisons.

Capture the real optimised values version-aware: for DIA-NN >= 2.5.0 read
'Recommended MS1 mass accuracy setting' / 'Suggested mass accuracy' / 'Scan window
radius set to' from each PRELIMINARY log and average across runs; < 2.5.0 keeps the
(correct for that format) ASSEMBLE averaged line. Reported by V. Demichev.

NOTE: needs a -profile test run to validate the channel/averaging at runtime.
…alibrating

Set the fallback mass accuracy to MS2=20 / MS1=10 ppm (was 15/15). Priority is now:
  mass_acc_automatic=true  -> calibration (preliminary logs) > SDRF ppm > defaults
  mass_acc_automatic=false -> SDRF ppm > defaults
Previously the non-automatic branch also fell back to calibration values, which is
contradictory when the user has explicitly disabled auto-calibration; remove that so
disabling calibration is honoured.
Protein groups are already inferred when the empirical library is built
(ASSEMBLE_EMPIRICAL_LIBRARY). Re-inferring in the final quant with --relaxed-prot-inf
can yield protein groups inconsistent with the library; --no-prot-inf reuses the
library's inference so the final report matches it. Suggested by V. Demichev.
…be parsed

The preliminary/assemble mass-acc parsers previously baked in the param defaults on
a parse miss, which short-circuited the SDRF tolerances. Emit a 0,0,0 sentinel
instead so INDIVIDUAL_ANALYSIS's existing cascade resolves the fallback in the
intended order: calibration (if parseable) > SDRF ppm (if annotated) > param
defaults. Matches the agreed priority: param is the base reliable value, SDRF
overrides it, calibration overrides that, and a failed calibration parse drops back
to the previous reliable value (SDRF, else param).
@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@coderabbitai

coderabbitai Bot commented Jun 8, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6edafd5c-44e3-4268-8f6f-166e8a9f14a6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/bug-fixes-from-benchmarks

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Jun 8, 2026 •

Copy link
Copy Markdown

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 4057d5e

+| ✅ 108 tests passed       |+
#| ❔  18 tests were ignored |#
#| ❔   1 tests had warnings |#
!| ❗   4 tests had warnings |!
Details

❗ Test warnings:

  • files_exist - File not found: conf/igenomes.config
  • files_exist - File not found: conf/igenomes_ignored.config
  • files_exist - File not found: .github/workflows/awstest.yml
  • files_exist - File not found: .github/workflows/awsfulltest.yml

❔ Tests ignored:

❔ Tests fixed:

✅ Tests passed:

Run details

  • nf-core/tools version 4.0.2
  • Run at 2026-06-08 22:29:49

@ypriverol

Copy link
Copy Markdown
Contributor Author

Code review

Found 3 issues (2 convention-required, 1 minor):

  1. --kb is added to PRELIMINARY/ASSEMBLE/INDIVIDUAL but not registered in lib/BlockedFlags.groovy. Every pipeline-managed DIA-NN flag must be in the blocked list (established in refactor: centralize blocked flags + fix missing guards from PR #36 review #48) so it can't be silently double-passed via extra_args. --kb is now pipeline-managed in three modules but absent from all blocked lists, so a user could still inject --kb (or override it) through extra_args even with enable_kb=false.

],
PRELIMINARY_ANALYSIS: [
// Pipeline-managed: set from params and SDRF calibration metadata
'--use-quant', '--gen-spec-lib', '--out-lib', '--matrices', '--out',
'--mass-acc', '--mass-acc-ms1', '--window',
'--quick-mass-acc', '--min-corr', '--corr-diff', '--time-corr-only',
'--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz',
'--channel-run-norm', '--channel-spec-norm',
// Pipeline-managed: preliminary step disables protein inference (--no-prot-inf)
'--no-prot-inf',
],
ASSEMBLE_EMPIRICAL_LIBRARY: [
// Pipeline-managed: set from params and calibration results
'--no-main-report', '--no-ifs-removal', '--matrices', '--out',
'--mass-acc', '--mass-acc-ms1', '--window',
'--individual-mass-acc', '--individual-windows',
'--out-lib', '--use-quant', '--gen-spec-lib', '--rt-profiling',
'--channel-run-norm', '--channel-spec-norm',
// No-effect guard: protein inference flags have no effect during library assembly
// (--gen-spec-lib produces a spectral library, not a quantified report).
// Blocked to prevent users from thinking they affect this step.
'--no-prot-inf', '--relaxed-prot-inf', '--pg-level',
],
INDIVIDUAL_ANALYSIS: [
// Pipeline-managed: set from params and calibrated values from assembly step
'--use-quant', '--gen-spec-lib', '--out-lib', '--matrices', '--out', '--rt-profiling',
'--mass-acc', '--mass-acc-ms1', '--window',
'--no-ifs-removal', '--no-main-report',
// Pipeline-managed: protein inference set by pipeline (--relaxed-prot-inf --pg-level)
'--relaxed-prot-inf', '--pg-level',
'--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz',
'--channel-run-norm', '--channel-spec-norm',
'--no-prot-inf',
],

  1. Parameter defaults changed without updating nextflow_schema.json. mass_acc_ms2 15→20, mass_acc_ms1 15→10, quantums false→true, and quantums_sel_runs null→20, but the schema still declares the old defaults (15/15, no quantums/quantums_sel_runs default). AGENTS.md §3 "Schema Validation (REQUIRED for parameter changes)" requires nf-core pipelines schema build.

// mass_acc_automatic=true -> calibration > SDRF ppm > these defaults
// mass_acc_automatic=false -> SDRF ppm > these defaults
mass_acc_ms2 = 20
mass_acc_ms1 = 10
// DIA-NN: FINAL_QUANTIFICATION — summarization & output
pg_level = 2 // protein-group inference level (2 = gene-level). Correct for UniProt FASTAs with gene names; gene-less FASTAs (e.g. ProteoBench HYE) are unaffected (DIA-NN falls back), so kept at 2 by default.
species_genes = false
normalize = true
report_decoys = false
export_xic = false
quantums = true // use QuantUMS (DIA-NN's recommended quantification) for DIA-NN >= 1.9.2; false adds '--direct-quant' (legacy quant)
quantums_train_runs = null
quantums_sel_runs = 20 // '--quant-sel-runs N': auto-select N runs to optimise QuantUMS params; keeps QuantUMS fast on large cohorts (only applied when QuantUMS is active)

  1. Minor — stale comment in BlockedFlags.groovy. The FINAL_QUANTIFICATION block comment still reads "Pipeline-managed: protein inference (--relaxed-prot-inf --pg-level)", but FINAL now emits --no-prot-inf (this PR). The blocked entries are fine; only the comment is out of date.

'--use-quant', '--matrices', '--out',
// Pipeline-managed: protein inference (--relaxed-prot-inf --pg-level)
'--relaxed-prot-inf', '--pg-level',
// Pipeline-managed: FDR controls (precursor_qvalue, matrix_qvalue, matrix_spec_q)

Lower-confidence notes (not blocking): the INDIVIDUAL_ANALYSIS (--relaxed-prot-inf) vs FINAL_QUANTIFICATION (--no-prot-inf) asymmetry may be intentional (per-file pass needs its own inference) but is worth a one-line comment; and quantums_on / quantums are two expressions of overlapping state.

…ma defaults

Address code-review findings on PR #104:
- Add --kb to the PRELIMINARY/ASSEMBLE/INDIVIDUAL blocked lists (it is now
  pipeline-managed via params.enable_kb) so it can't be double-passed via extra_args.
- Register the pipeline-managed QuantUMS flags (--direct-quant, --quant-sel-runs,
  --quant-train-runs, --quant-params) in the FINAL_QUANTIFICATION blocked list, and
  fix the stale protein-inference comment (FINAL now emits --no-prot-inf).
- Sync nextflow_schema.json defaults to nextflow.config: mass_acc_ms2 15->20,
  mass_acc_ms1 15->10, quantums default true, quantums_sel_runs default 20; fix the
  contradictory quantums description.
@ypriverol

Copy link
Copy Markdown
Contributor Author

Review findings addressed — 4057d5e

All three code-review items fixed:

  1. --kb registered in BlockedFlags.groovy — added to the PRELIMINARY / ASSEMBLE / INDIVIDUAL blocked lists (it's pipeline-managed via enable_kb). Also registered the now pipeline-managed QuantUMS flags (--direct-quant, --quant-sel-runs, --quant-train-runs, --quant-params) in FINAL.
  2. nextflow_schema.json synced — mass_acc_ms2 15→20, mass_acc_ms1 15→10, quantums default true, quantums_sel_runs default 20; also fixed the contradictory quantums description.
  3. Stale BlockedFlags comment for FINAL protein inference updated (now --no-prot-inf).

CodeRabbit (coderabbit review --agent -t all --base dev): 3 findings, all in modules/local/utils/wiff_convert/main.nf — a file this PR does not touch (pre-existing: process naming, hardcoded 0.10 version, missing .wiff null-check). Out of scope here; worth a separate cleanup PR.

Still open before merge: a -profile test run to validate the new preliminary-log mass-accuracy parsing / averaging / sentinel path at runtime.

@ypriverol
ypriverol merged commit 0d65a6d into dev Jun 9, 2026
21 checks passed
ypriverol added a commit that referenced this pull request Jun 10, 2026
…o-prot-inf opt-in

Per DIA-NN docs, --relaxed-prot-inf (FragPipe/Spectronaut-style grouping) and
--no-prot-inf (reuse library inference) are both OPT-IN deviations from DIA-NN's
default standard inference. The pipeline was forcing --relaxed-prot-inf
(INDIVIDUAL_ANALYSIS) and --no-prot-inf (FINAL, #104) -- both non-default.

Replace with two mutually-exclusive opt-in params (relaxed_prot_inf / no_prot_inf,
default false) applied to INDIVIDUAL_ANALYSIS + FINAL_QUANTIFICATION. Default (both
off) = DIA-NN standard inference -- what a typical user gets. Benchmarks opt into
whichever inference a given comparison needs (Vadim's guidance). Supersedes the
single relaxed_protein_inference toggle.
yueqixuan pushed a commit to yueqixuan/quantmsdiann that referenced this pull request Jun 16, 2026
…in_inference)

FINAL_QUANTIFICATION hardcoded --no-prot-inf (bigbio#104). Make it a param:
relaxed_protein_inference=false (default) -> --no-prot-inf (reuse empirical-library
inference); true -> --relaxed-prot-inf (re-infer in final). Enables an A/B to test
whether --no-prot-inf is what lifts 1.8.1 protein counts and erases the 1.8.1->2.5
run-specific protein advantage Vadim expects (+12%/+18%).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant