Repository navigation
fix(diann): benchmark-review bug fixes (--kb, QuantUMS, mass-accuracy, --no-prot-inf) - #104
Conversation
--kb was only added to PRELIMINARY_ANALYSIS, which is the quick mass-accuracy calibration pass — not where identification happens. The Knowledge Base boosts IDs and belongs on the actual search steps. Add --kb (gated by params.enable_kb) to ASSEMBLE_EMPIRICAL_LIBRARY (first-pass search that builds the empirical library) and INDIVIDUAL_ANALYSIS (per-file search); clarify the PRELIMINARY_ANALYSIS comment. Found during the ProteoBench benchmark review (reported by V. Demichev).
params.quantums defaulted to false, which forced --direct-quant on every DIA-NN >= 1.9.2 run and thereby disabled QuantUMS (DIA-NN's recommended quantification), affecting quantification accuracy. Flip the default to true. The --direct-quant opt-out remains version-gated in FINAL_QUANTIFICATION, so DIA-NN 1.8.1 (no QuantUMS) is unaffected; only >= 1.9.2 now uses QuantUMS.
…s by version QuantUMS over thousands of runs (e.g. ProCan, single-cell sweeps) is very slow when it optimises params over every run. Default --quant-sel-runs to 20 (DIA-NN auto-selects 20 runs to optimise QuantUMS params) so QuantUMS scales now that it is on by default. Also gate --quant-sel-runs/--quant-train-runs/--quant-params on QuantUMS actually being active (params.quantums && DIA-NN >= 1.9.2) so they are never passed to versions that don't support them (e.g. 1.8.1).
…A-NN >= 2.5.0 The mass accuracy / scan window for the per-file search were parsed from the ASSEMBLE_EMPIRICAL_LIBRARY 'Averaged recommended settings' line. On DIA-NN >= 2.5.0 that step combines --use-quant with --individual-mass-acc, which DIA-NN flags as 'strongly not recommended' and then prints a default MS1 = 20 ppm (far too wide for e.g. Astral) decoupled from the genuine per-run optimisation -- so the per-file search ran at MS1 = 20 ppm while 1.8.1 (no such warning) correctly used ~3 ppm, confounding cross-version comparisons. Capture the real optimised values version-aware: for DIA-NN >= 2.5.0 read 'Recommended MS1 mass accuracy setting' / 'Suggested mass accuracy' / 'Scan window radius set to' from each PRELIMINARY log and average across runs; < 2.5.0 keeps the (correct for that format) ASSEMBLE averaged line. Reported by V. Demichev. NOTE: needs a -profile test run to validate the channel/averaging at runtime.
…alibrating Set the fallback mass accuracy to MS2=20 / MS1=10 ppm (was 15/15). Priority is now: mass_acc_automatic=true -> calibration (preliminary logs) > SDRF ppm > defaults mass_acc_automatic=false -> SDRF ppm > defaults Previously the non-automatic branch also fell back to calibration values, which is contradictory when the user has explicitly disabled auto-calibration; remove that so disabling calibration is honoured.
Protein groups are already inferred when the empirical library is built (ASSEMBLE_EMPIRICAL_LIBRARY). Re-inferring in the final quant with --relaxed-prot-inf can yield protein groups inconsistent with the library; --no-prot-inf reuses the library's inference so the final report matches it. Suggested by V. Demichev.
…be parsed The preliminary/assemble mass-acc parsers previously baked in the param defaults on a parse miss, which short-circuited the SDRF tolerances. Emit a 0,0,0 sentinel instead so INDIVIDUAL_ANALYSIS's existing cascade resolves the fallback in the intended order: calibration (if parseable) > SDRF ppm (if annotated) > param defaults. Matches the agreed priority: param is the base reliable value, SDRF overrides it, calibration overrides that, and a failed calibration parse drops back to the previous reliable value (SDRF, else param).
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Code reviewFound 3 issues (2 convention-required, 1 minor):
quantmsdiann/lib/BlockedFlags.groovy Lines 66 to 99 in 4596bc3
Lines 115 to 128 in 4596bc3
quantmsdiann/lib/BlockedFlags.groovy Lines 103 to 106 in 4596bc3 Lower-confidence notes (not blocking): the |
…ma defaults Address code-review findings on PR #104: - Add --kb to the PRELIMINARY/ASSEMBLE/INDIVIDUAL blocked lists (it is now pipeline-managed via params.enable_kb) so it can't be double-passed via extra_args. - Register the pipeline-managed QuantUMS flags (--direct-quant, --quant-sel-runs, --quant-train-runs, --quant-params) in the FINAL_QUANTIFICATION blocked list, and fix the stale protein-inference comment (FINAL now emits --no-prot-inf). - Sync nextflow_schema.json defaults to nextflow.config: mass_acc_ms2 15->20, mass_acc_ms1 15->10, quantums default true, quantums_sel_runs default 20; fix the contradictory quantums description.
Review findings addressed — 4057d5eAll three code-review items fixed:
CodeRabbit ( Still open before merge: a |
…o-prot-inf opt-in Per DIA-NN docs, --relaxed-prot-inf (FragPipe/Spectronaut-style grouping) and --no-prot-inf (reuse library inference) are both OPT-IN deviations from DIA-NN's default standard inference. The pipeline was forcing --relaxed-prot-inf (INDIVIDUAL_ANALYSIS) and --no-prot-inf (FINAL, #104) -- both non-default. Replace with two mutually-exclusive opt-in params (relaxed_prot_inf / no_prot_inf, default false) applied to INDIVIDUAL_ANALYSIS + FINAL_QUANTIFICATION. Default (both off) = DIA-NN standard inference -- what a typical user gets. Benchmarks opt into whichever inference a given comparison needs (Vadim's guidance). Supersedes the single relaxed_protein_inference toggle.
…in_inference) FINAL_QUANTIFICATION hardcoded --no-prot-inf (bigbio#104). Make it a param: relaxed_protein_inference=false (default) -> --no-prot-inf (reuse empirical-library inference); true -> --relaxed-prot-inf (re-infer in final). Enables an A/B to test whether --no-prot-inf is what lifts 1.8.1 protein counts and erases the 1.8.1->2.5 run-specific protein advantage Vadim expects (+12%/+18%).
Summary
Bug fixes for the DIA-NN modules, found while benchmarking quantmsdiann against ProteoBench (DIA-NN 1.8.1 / 2.5.1 / 2.5.1-enterprise) and reviewed with DIA-NN's author (V. Demichev). These address cross-version comparison confounders and a few configuration issues on the Enterprise / 2.5.x path.
Changes
--kb(Enterprise Knowledge Base) on the search passes — it was only wired intoPRELIMINARY_ANALYSIS(quick mass-acc calibration), not where identification happens. Added toASSEMBLE_EMPIRICAL_LIBRARY(first-pass search) andINDIVIDUAL_ANALYSIS(per-file search), gated byparams.enable_kb.QuantUMS on by default for DIA-NN ≥ 1.9.2 —
params.quantumsdefaulted tofalse, forcing--direct-quantand disabling QuantUMS (DIA-NN's recommended quantification). Flipped totrue; the--direct-quantopt-out stays version-gated, so 1.8.1 is unaffected.--quant-sel-runs 20default + version-gated QuantUMS tuning flags — QuantUMS over thousands of runs is slow; auto-select 20 runs to optimise its params.--quant-sel-runs/--quant-train-runs/--quant-paramsnow only apply when QuantUMS is active (params.quantums && DIA-NN ≥ 1.9.2), so they're never passed to versions that don't support them.Mass-accuracy resolution (the main confounder) — the per-file search mass accuracy was read from the
ASSEMBLE_EMPIRICAL_LIBRARY"Averaged recommended settings" log line. On DIA-NN ≥ 2.5.0 that step combines--use-quantwith--individual-mass-acc, which DIA-NN flags as "strongly not recommended" and then prints a default MS1 = 20 ppm (vs the genuine ~3 ppm), so 2.5.x searched at MS1 = 20 ppm while 1.8.1 used 3 ppm. Now:Recommended MS1 mass accuracy setting/Suggested mass accuracy/Scan window radius) and average across runs.mass_acc_automatic=true: calibration > SDRF ppm > defaults;mass_acc_automatic=false: SDRF ppm > defaults (calibration excluded). Defaults updated 15/15 → 20/10 ppm.0,0,0sentinel makesINDIVIDUAL_ANALYSISfall back to the previous reliable value (SDRF if annotated, else params) instead of silently using defaults.--no-prot-infinFINAL_QUANTIFICATION— protein groups are already inferred when the empirical library is built; reuse that inference in the final quant instead of re-inferring with--relaxed-prot-inf, so the final report matches the library.Docs — record the decision to keep
pg_level = 2.Validation
nextflow lintis clean (only the expectedlib/class false positives).-profile testrun to validate the new preliminary-log parsing / averaging / sentinel path at runtime.Not in this PR
--mass-acc 20 / --mass-acc-ms1 10written intodiann_config.cfgbyquantms-utils(dianncfg) — inert here (not extracted intomod_flags); to be fixed inquantms-utils.Reported/reviewed by V. Demichev.