From e2b2ac732f3692164d08ac32f88442d3e7a829ed Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 11:26:45 +0100 Subject: [PATCH 01/69] docs: add v1.0.0 release implementation plan 13-task plan covering robustness fixes, DDA support, new DIA-NN params, InfinDIA groundwork, comprehensive documentation, and issue cleanup. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../2026-04-03-v1-release-implementation.md | 789 ++++++++++++++++++ 1 file changed, 789 insertions(+) create mode 100644 docs/plans/2026-04-03-v1-release-implementation.md diff --git a/docs/plans/2026-04-03-v1-release-implementation.md b/docs/plans/2026-04-03-v1-release-implementation.md new file mode 100644 index 0000000..4b3d0ed --- /dev/null +++ b/docs/plans/2026-04-03-v1-release-implementation.md @@ -0,0 +1,789 @@ +# quantmsdiann v1.0.0 Release — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Prepare quantmsdiann for a robust v1.0.0 release with DDA support, new DIA-NN parameters, and comprehensive documentation. + +**Architecture:** No new workflows or modules. All changes are additions to existing files — new params, flags, guards, test configs, and docs. DDA uses the same pipeline as DIA with `--dda` appended to all DIA-NN invocations. Default container stays 1.8.1; 2.3.2 is opt-in via profile. + +**Tech Stack:** Nextflow DSL2, nf-core, DIA-NN, Groovy, Bash + +--- + +## Task 1: Fix tee pipes masking failures + +**Files:** +- Modify: `modules/local/diann/generate_cfg/main.nf:26` +- Modify: `modules/local/diann/diann_msstats/main.nf:21-26` +- Modify: `modules/local/samplesheet_check/main.nf:38-43` +- Modify: `modules/local/sdrf_parsing/main.nf:24-30` + +- [ ] **Step 1: Add pipefail to generate_cfg** + +In `modules/local/diann/generate_cfg/main.nf`, find the `"""` opening the script block (line 20) and add `set -o pipefail` as the first line: + +```groovy + """ + set -o pipefail + parse_sdrf generate-diann-cfg \\ + ... + ``` + +- [ ] **Step 2: Add pipefail to diann_msstats** + +In `modules/local/diann/diann_msstats/main.nf`, find the `"""` opening the script block (line 20) and add `set -o pipefail`: + +```groovy + """ + set -o pipefail + quantmsutilsc diann2msstats \\ + ... + ``` + +- [ ] **Step 3: Add pipefail to samplesheet_check** + +In `modules/local/samplesheet_check/main.nf`, find the `"""` opening the script block and add `set -o pipefail`: + +```groovy + """ + set -o pipefail + ... + ``` + +- [ ] **Step 4: Add pipefail to sdrf_parsing** + +In `modules/local/sdrf_parsing/main.nf`, find the `"""` opening the script block (line 22) and add `set -o pipefail`: + +```groovy + """ + set -o pipefail + parse_sdrf convert-diann \\ + ... + ``` + +- [ ] **Step 5: Validate and commit** + +```bash +conda run -n nfcore pre-commit run --all-files +git add modules/local/diann/generate_cfg/main.nf modules/local/diann/diann_msstats/main.nf modules/local/samplesheet_check/main.nf modules/local/sdrf_parsing/main.nf +git commit -m "fix: add pipefail to all modules with tee pipes + +Without pipefail, if the command before tee fails, tee returns 0 and +the Nextflow task appears to succeed. This masked failures in +generate_cfg, diann_msstats, samplesheet_check, and sdrf_parsing." +``` + +--- + +## Task 2: Add error retry to long-running DIA-NN tasks + +**Files:** +- Modify: `modules/local/diann/preliminary_analysis/main.nf:3-4` +- Modify: `modules/local/diann/individual_analysis/main.nf:3-4` +- Modify: `modules/local/diann/final_quantification/main.nf:3-4` +- Modify: `modules/local/diann/insilico_library_generation/main.nf:3-4` +- Modify: `modules/local/diann/assemble_empirical_library/main.nf:3-4` + +- [ ] **Step 1: Add error_retry label to all 5 DIA-NN modules** + +In each file, add `label 'error_retry'` after the existing labels. For example, `preliminary_analysis/main.nf` currently has: + +```groovy + label 'process_high' + label 'diann' +``` + +Change to: + +```groovy + label 'process_high' + label 'diann' + label 'error_retry' +``` + +Do the same for: +- `individual_analysis/main.nf` (after `label 'diann'`) +- `final_quantification/main.nf` (after `label 'diann'`) +- `insilico_library_generation/main.nf` (after `label 'diann'`) +- `assemble_empirical_library/main.nf` (after `label 'diann'`) + +- [ ] **Step 2: Validate and commit** + +```bash +conda run -n nfcore pre-commit run --all-files +git add modules/local/diann/preliminary_analysis/main.nf modules/local/diann/individual_analysis/main.nf modules/local/diann/final_quantification/main.nf modules/local/diann/insilico_library_generation/main.nf modules/local/diann/assemble_empirical_library/main.nf +git commit -m "fix: add error_retry label to all DIA-NN analysis modules + +These are the longest-running tasks and most susceptible to transient +failures (OOM, I/O timeouts). The error_retry label enables automatic +retry on signal exits (130-145, 104, 175)." +``` + +--- + +## Task 3: Add empty input guards + +**Files:** +- Modify: `workflows/dia.nf:38,46` + +- [ ] **Step 1: Guard ch_searchdb with ifEmpty** + +In `workflows/dia.nf`, line 38, change: + +```groovy + ch_searchdb = channel.fromPath(params.database, checkIfExists: true).first() +``` + +To: + +```groovy + ch_searchdb = channel.fromPath(params.database, checkIfExists: true) + .ifEmpty { error("No protein database found at '${params.database}'. Provide --database ") } + .first() +``` + +- [ ] **Step 2: Guard ch_experiment_meta with ifEmpty** + +In `workflows/dia.nf`, line 46, change: + +```groovy + ch_experiment_meta = ch_result.meta.unique { m -> m.experiment_id }.first() +``` + +To: + +```groovy + ch_experiment_meta = ch_result.meta.unique { m -> m.experiment_id } + .ifEmpty { error("No valid input files found after SDRF parsing. Check your SDRF file and input paths.") } + .first() +``` + +- [ ] **Step 3: Validate and commit** + +```bash +conda run -n nfcore pre-commit run --all-files +git add workflows/dia.nf +git commit -m "fix: add empty input guards to prevent silent pipeline hangs + +Guard ch_searchdb and ch_experiment_meta with ifEmpty to fail fast +with clear error messages instead of hanging indefinitely." +``` + +--- + +## Task 4: Add DIA-NN 2.3.2 version config and profile + +**Files:** +- Create: `conf/diann_versions/v2_3_2.config` +- Modify: `nextflow.config:245-247` (profiles section) + +- [ ] **Step 1: Create v2_3_2.config** + +Create `conf/diann_versions/v2_3_2.config`: + +```groovy +/* + * DIA-NN 2.3.2 container override (private ghcr.io) + * Latest release with DDA support and InfinDIA. + */ +params.diann_version = '2.3.2' + +process { + withLabel: diann { + container = 'ghcr.io/bigbio/diann:2.3.2' + } +} + +singularity.enabled = false +docker.enabled = true +``` + +- [ ] **Step 2: Add profile to nextflow.config** + +In `nextflow.config`, after the `diann_v2_2_0` profile line (around line 247), add: + +```groovy + diann_v2_3_2 { includeConfig 'conf/diann_versions/v2_3_2.config' } +``` + +- [ ] **Step 3: Validate and commit** + +```bash +conda run -n nfcore pre-commit run --all-files +git add conf/diann_versions/v2_3_2.config nextflow.config +git commit -m "feat: add DIA-NN 2.3.2 version config and profile + +Adds conf/diann_versions/v2_3_2.config with ghcr.io/bigbio/diann:2.3.2 +container. Use -profile diann_v2_3_2 to opt in. Default stays 1.8.1. +Enables DDA support and InfinDIA features." +``` + +--- + +## Task 5: Implement DDA support — params, version guard, flag passthrough + +**Files:** +- Modify: `nextflow.config:53-57` (DIA-NN general params) +- Modify: `nextflow_schema.json` (DIA-NN section) +- Modify: `workflows/dia.nf:35-38` (version guard) +- Modify: `subworkflows/local/create_input_channel/main.nf:75-88` (acquisition method) +- Modify: `modules/local/diann/insilico_library_generation/main.nf` (blocked list + flag) +- Modify: `modules/local/diann/preliminary_analysis/main.nf` (blocked list + flag) +- Modify: `modules/local/diann/assemble_empirical_library/main.nf` (blocked list + flag) +- Modify: `modules/local/diann/individual_analysis/main.nf` (blocked list + flag) +- Modify: `modules/local/diann/final_quantification/main.nf` (blocked list + flag) + +- [ ] **Step 1: Add diann_dda param to nextflow.config** + +In `nextflow.config`, after `diann_extra_args = null` (line 57), add: + +```groovy + diann_dda = false // Enable DDA analysis mode (requires DIA-NN >= 2.3.2) +``` + +- [ ] **Step 2: Add diann_dda to nextflow_schema.json** + +In `nextflow_schema.json`, in the DIA-NN section (inside `"$defs"` > appropriate group), add: + +```json +"diann_dda": { + "type": "boolean", + "description": "Enable DDA (Data-Dependent Acquisition) analysis mode. Passes --dda to all DIA-NN steps. Requires DIA-NN >= 2.3.2 (use -profile diann_v2_3_2). Beta feature — only trust q-values, PEP, RT/IM, Ms1.Apex.Area. PTM localization unreliable with DDA.", + "fa_icon": "fas fa-flask", + "default": false +} +``` + +Add `"diann_dda"` to the corresponding `"required"` or `"properties"` list in the appropriate group. + +- [ ] **Step 3: Add version guard in workflows/dia.nf** + +In `workflows/dia.nf`, at the start of the `main:` block (after line 37), add: + +```groovy + // Version guard for DDA mode + if (params.diann_dda && params.diann_version < '2.3.2') { + error("DDA mode (--diann_dda) requires DIA-NN >= 2.3.2. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") + } +``` + +- [ ] **Step 4: Accept DDA acquisition method in create_input_channel** + +In `subworkflows/local/create_input_channel/main.nf`, replace lines 75-88 (the acquisition method validation block): + +```groovy + // Validate acquisition method + def acqMethod = row.AcquisitionMethod?.toString()?.trim() ?: "" + if (acqMethod.toLowerCase().contains("data-independent acquisition") || acqMethod.toLowerCase().contains("dia")) { + meta.acquisition_method = "dia" + } else if (params.diann_dda && (acqMethod.toLowerCase().contains("data-dependent acquisition") || acqMethod.toLowerCase().contains("dda"))) { + meta.acquisition_method = "dda" + } else if (acqMethod.isEmpty()) { + meta.acquisition_method = params.diann_dda ? "dda" : "dia" + } else { + log.error("Unsupported acquisition method: '${acqMethod}'. This pipeline supports DIA" + (params.diann_dda ? " and DDA (--diann_dda)" : "") + ". Found in file: ${filestr}") + exit(1) + } +``` + +- [ ] **Step 5: Add --dda flag to all 5 DIA-NN modules** + +For each of the 5 DIA-NN modules, make two changes: + +**a) Add `'--dda'` to the blocked list.** In each module's `def blocked = [...]`, add `'--dda'` to the array. + +**b) Add the flag variable and append it to the command.** In each module's script block, after the existing flag variables (before the `"""` shell block), add: + +```groovy + diann_dda_flag = params.diann_dda ? "--dda" : "" +``` + +Then append `${diann_dda_flag} \\` to the DIA-NN command, before `\${mod_flags}` (or before `$args` if no mod_flags). + +Apply to: +- `modules/local/diann/insilico_library_generation/main.nf` +- `modules/local/diann/preliminary_analysis/main.nf` +- `modules/local/diann/assemble_empirical_library/main.nf` +- `modules/local/diann/individual_analysis/main.nf` +- `modules/local/diann/final_quantification/main.nf` + +- [ ] **Step 6: Validate and commit** + +```bash +conda run -n nfcore pre-commit run --all-files +conda run -n nfcore nf-core pipelines lint --dir . +git add nextflow.config nextflow_schema.json workflows/dia.nf subworkflows/local/create_input_channel/main.nf modules/local/diann/*/main.nf +git commit -m "feat: add DDA support via --diann_dda flag (#5) + +- New param diann_dda (boolean, default: false) +- Version guard: requires DIA-NN >= 2.3.2 +- Passes --dda to all 5 DIA-NN modules when enabled +- Accepts DDA acquisition method in SDRF when diann_dda=true +- Added --dda to blocked lists in all modules + +Closes #5" +``` + +--- + +## Task 6: Add DDA test config + +**Files:** +- Create: `conf/tests/test_dda.config` +- Modify: `.github/workflows/extended_ci.yml:110-191` (stage 2a) + +- [ ] **Step 1: Create test_dda.config** + +Create `conf/tests/test_dda.config`: + +```groovy +/* +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Nextflow config file for testing DDA analysis (requires DIA-NN >= 2.3.2) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Tests DDA mode using the BSA dataset with --diann_dda flag. + Uses ghcr.io/bigbio/diann:2.3.2. + + Use as follows: + nextflow run bigbio/quantmsdiann -profile test_dda,docker [--outdir ] + +------------------------------------------------------------------------------------------------ +*/ + +process { + resourceLimits = [ + cpus: 4, + memory: '12.GB', + time: '48.h' + ] +} + +params { + config_profile_name = 'Test profile for DDA analysis' + config_profile_description = 'DDA test using BSA dataset with DIA-NN 2.3.2.' + + outdir = './results_dda' + + // Input data — BSA DDA dataset + input = 'https://raw.githubusercontent.com/bigbio/quantms-test-datasets/quantms/testdata/lfq_ci/BSA/BSA_design.sdrf.tsv' + database = 'https://raw.githubusercontent.com/bigbio/quantms-test-datasets/quantms/testdata/lfq_ci/BSA/18Protein_SoCe_Tr_detergents_trace.fasta' + + // DDA mode + diann_dda = true + + // Search parameters matching BSA dataset + min_peptide_length = 7 + max_peptide_length = 30 + max_precursor_charge = 3 + allowed_missed_cleavages = 1 + diann_normalize = false + publish_dir_mode = 'symlink' + max_mods = 2 +} + +process { + withLabel: diann { + container = 'ghcr.io/bigbio/diann:2.3.2' + } +} + +singularity.enabled = false +docker.enabled = true +``` + +- [ ] **Step 2: Add test_dda profile to nextflow.config** + +In `nextflow.config`, after the `test_dia_2_2_0` profile line (around line 241), add: + +```groovy + test_dda { includeConfig 'conf/tests/test_dda.config' } +``` + +- [ ] **Step 3: Add test_dda to extended_ci.yml stage 2a** + +In `.github/workflows/extended_ci.yml`, in the `test-latest` job matrix (around line 120), add `"test_dda"` to the `test_profile` array: + +```yaml + test_profile: ["test_latest_dia", "test_dia_quantums", "test_dia_parquet", "test_dda"] +``` + +- [ ] **Step 4: Validate and commit** + +```bash +conda run -n nfcore pre-commit run --all-files +git add conf/tests/test_dda.config nextflow.config .github/workflows/extended_ci.yml +git commit -m "test: add DDA test config using BSA dataset with DIA-NN 2.3.2 + +Uses bigbio/quantms-test-datasets BSA LFQ dataset (~34 MB) with +diann_dda=true pinned to ghcr.io/bigbio/diann:2.3.2. Added to +extended_ci.yml stage 2a (private containers)." +``` + +--- + +## Task 7: Add test configs for skip_preliminary_analysis and speclib input + +**Files:** +- Create: `conf/tests/test_dia_skip_preanalysis.config` +- Modify: `nextflow.config` (profiles section) +- Modify: `.github/workflows/extended_ci.yml` (stage 2a) + +- [ ] **Step 1: Create test_dia_skip_preanalysis.config** + +Create `conf/tests/test_dia_skip_preanalysis.config`: + +```groovy +/* +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Nextflow config file for testing skip_preliminary_analysis path +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Tests the pipeline with skip_preliminary_analysis=true, using default + mass accuracy parameters. Validates the untested code path in dia.nf. + + Use as follows: + nextflow run bigbio/quantmsdiann -profile test_dia_skip_preanalysis,docker [--outdir ] + +------------------------------------------------------------------------------------------------ +*/ + +process { + resourceLimits = [ + cpus: 4, + memory: '12.GB', + time: '48.h' + ] +} + +params { + config_profile_name = 'Test profile for skip preliminary analysis' + config_profile_description = 'Tests skip_preliminary_analysis path with default mass accuracy params.' + + outdir = './results_skip_preanalysis' + + // Input data — same as test_dia + input = 'https://raw.githubusercontent.com/bigbio/quantms-test-datasets/quantms/testdata/dia_ci/PXD026600.sdrf.tsv' + database = 'https://raw.githubusercontent.com/bigbio/quantms-test-datasets/quantms/testdata/dia_ci/REF_EColi_K12_UPS1_combined.fasta' + min_pr_mz = 350 + max_pr_mz = 950 + min_fr_mz = 500 + max_fr_mz = 1500 + min_peptide_length = 15 + max_peptide_length = 30 + max_precursor_charge = 3 + allowed_missed_cleavages = 1 + diann_normalize = false + publish_dir_mode = 'symlink' + max_mods = 2 + + // Skip preliminary analysis — use default mass accuracy params + skip_preliminary_analysis = true + mass_acc_ms2 = 15 + mass_acc_ms1 = 15 + scan_window = 8 +} +``` + +- [ ] **Step 2: Add profile to nextflow.config** + +After existing test profiles (around line 242), add: + +```groovy + test_dia_skip_preanalysis { includeConfig 'conf/tests/test_dia_skip_preanalysis.config' } +``` + +- [ ] **Step 3: Add to extended_ci.yml stage 2a** + +In the `test-latest` job matrix, add `"test_dia_skip_preanalysis"` to the `test_profile` array. + +- [ ] **Step 4: Validate and commit** + +```bash +conda run -n nfcore pre-commit run --all-files +git add conf/tests/test_dia_skip_preanalysis.config nextflow.config .github/workflows/extended_ci.yml +git commit -m "test: add test config for skip_preliminary_analysis path + +Tests the previously untested code path where preliminary analysis is +skipped and default mass accuracy parameters are used directly." +``` + +--- + +## Task 8: Add new DIA-NN feature parameters (light-models, export-quant, site-ms1-quant) + +**Files:** +- Modify: `nextflow.config` (params section) +- Modify: `nextflow_schema.json` +- Modify: `modules/local/diann/insilico_library_generation/main.nf` (light-models) +- Modify: `modules/local/diann/final_quantification/main.nf` (export-quant, site-ms1-quant) + +- [ ] **Step 1: Add params to nextflow.config** + +In `nextflow.config`, in the DIA-NN general section (after `diann_dda`, around line 58), add: + +```groovy + diann_light_models = false // add '--light-models' for 10x faster library generation (DIA-NN >= 2.0) + diann_export_quant = false // add '--export-quant' for fragment-level parquet export (DIA-NN >= 2.0) + diann_site_ms1_quant = false // add '--site-ms1-quant' for MS1 apex PTM quantification (DIA-NN >= 2.0) +``` + +- [ ] **Step 2: Add params to nextflow_schema.json** + +Add each param to the DIA-NN section in the schema with type, description, default, and fa_icon. + +- [ ] **Step 3: Wire --light-models in insilico_library_generation** + +In `modules/local/diann/insilico_library_generation/main.nf`: + +a) Add `'--light-models'` to the blocked list (line 26-32). + +b) After `diann_no_peptidoforms` variable (line 47), add: + +```groovy + diann_light_models = params.diann_light_models ? "--light-models" : "" +``` + +c) Append `${diann_light_models} \\` to the DIA-NN command before `${met_excision}`. + +- [ ] **Step 4: Wire --export-quant and --site-ms1-quant in final_quantification** + +In `modules/local/diann/final_quantification/main.nf`: + +a) Add `'--export-quant'` and `'--site-ms1-quant'` to the blocked list (line 45-50). + +b) After `diann_dda_flag` variable, add: + +```groovy + diann_export_quant = params.diann_export_quant ? "--export-quant" : "" + diann_site_ms1_quant = params.diann_site_ms1_quant ? "--site-ms1-quant" : "" +``` + +c) Append both to the DIA-NN command before `\${mod_flags}`. + +- [ ] **Step 5: Validate and commit** + +```bash +conda run -n nfcore pre-commit run --all-files +conda run -n nfcore nf-core pipelines lint --dir . +git add nextflow.config nextflow_schema.json modules/local/diann/insilico_library_generation/main.nf modules/local/diann/final_quantification/main.nf +git commit -m "feat: add --light-models, --export-quant, --site-ms1-quant params (#7) + +- diann_light_models: 10x faster in-silico library generation +- diann_export_quant: fragment-level parquet export +- diann_site_ms1_quant: MS1 apex intensities for PTM quantification +All require DIA-NN >= 2.0." +``` + +--- + +## Task 9: Add InfinDIA groundwork + +**Files:** +- Modify: `nextflow.config` (params section) +- Modify: `nextflow_schema.json` +- Modify: `workflows/dia.nf` (version guard) +- Modify: `modules/local/diann/insilico_library_generation/main.nf` (flag) + +- [ ] **Step 1: Add InfinDIA params to nextflow.config** + +After the DDA param, add: + +```groovy + // DIA-NN: InfinDIA (experimental, v2.3.0+) + enable_infin_dia = false // Enable InfinDIA for ultra-large search spaces + diann_pre_select = null // --pre-select N precursor limit for InfinDIA +``` + +- [ ] **Step 2: Add to nextflow_schema.json** + +Add `enable_infin_dia` (boolean) and `diann_pre_select` (integer, optional) to the schema. + +- [ ] **Step 3: Add version guard in workflows/dia.nf** + +After the DDA version guard, add: + +```groovy + if (params.enable_infin_dia && params.diann_version < '2.3.0') { + error("InfinDIA requires DIA-NN >= 2.3.0. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") + } +``` + +- [ ] **Step 4: Wire flags in insilico_library_generation** + +In `modules/local/diann/insilico_library_generation/main.nf`: + +a) Add `'--infin-dia'` and `'--pre-select'` to the blocked list. + +b) Add flag variables: + +```groovy + infin_dia_flag = params.enable_infin_dia ? "--infin-dia" : "" + pre_select_flag = params.diann_pre_select ? "--pre-select $params.diann_pre_select" : "" +``` + +c) Append both to the DIA-NN command. + +- [ ] **Step 5: Validate and commit** + +```bash +conda run -n nfcore pre-commit run --all-files +conda run -n nfcore nf-core pipelines lint --dir . +git add nextflow.config nextflow_schema.json workflows/dia.nf modules/local/diann/insilico_library_generation/main.nf +git commit -m "feat: add InfinDIA groundwork — enable_infin_dia param (#10) + +Experimental support for InfinDIA (DIA-NN 2.3.0+). Passes --infin-dia +to library generation when enabled. Version guard enforces >= 2.3.0. +No test config — InfinDIA requires large databases." +``` + +--- + +## Task 10: Documentation — parameters.md + +**Files:** +- Create: `docs/parameters.md` + +- [ ] **Step 1: Create comprehensive parameter reference** + +Create `docs/parameters.md` with all params from `nextflow_schema.json` grouped by category. Read `nextflow.config` and `nextflow_schema.json` to get every param, its type, default, and description. Group into: + +1. Input/output options +2. File preparation +3. DIA-NN general +4. Mass accuracy and calibration +5. Library generation +6. Quantification and output +7. DDA mode +8. InfinDIA (experimental) +9. Quality control +10. MultiQC options +11. Boilerplate + +Each param entry: `| name | type | default | description |` + +- [ ] **Step 2: Commit** + +```bash +git add docs/parameters.md +git commit -m "docs: add comprehensive parameter reference (#1) + +Complete reference for all ~70 pipeline parameters grouped by category +with types, defaults, descriptions, and version requirements. + +Closes #1" +``` + +--- + +## Task 11: Documentation — complete usage.md and output.md + +**Files:** +- Modify: `docs/usage.md` +- Modify: `docs/output.md` +- Modify: `CITATIONS.md` +- Modify: `README.md` + +- [ ] **Step 1: Add DDA section to usage.md** + +Add a "DDA Analysis Mode" section after the Bruker/timsTOF section with: +- How to enable (`--diann_dda true -profile diann_v2_3_2`) +- Limitations (beta, trusted columns only, PTM unreliable, MBR limited) +- Example command +- Link to DIA-NN DDA documentation + +- [ ] **Step 2: Add missing param sections to usage.md** + +Add sections for: +- Preprocessing params (`reindex_mzml`, `mzml_statistics`, `convert_dotd`) +- QC params (`enable_pmultiqc`, `skip_table_plots`, `contaminant_string`) +- `diann_extra_args` scope per module +- `--verbose_modules` profile +- Container version override guide (DIA-NN version profiles) +- Singularity usage +- SLURM example + +- [ ] **Step 3: Update output.md** + +Add: +- Parquet vs TSV output explanation +- MSstats format section +- Intermediate outputs under `--verbose_modules` + +- [ ] **Step 4: Add pmultiqc to CITATIONS.md** + +Add pmultiqc citation after the MultiQC entry. + +- [ ] **Step 5: Update README.md** + +Add DIA-NN version support table and link to `docs/parameters.md`. + +- [ ] **Step 6: Validate and commit** + +```bash +conda run -n nfcore pre-commit run --all-files +git add docs/usage.md docs/output.md CITATIONS.md README.md +git commit -m "docs: complete usage.md, output.md, citations, README (#1, #3, #9, #15) + +- DDA mode documentation with limitations +- Missing param sections (preprocessing, QC, extra_args scope) +- Container version override and Singularity guides +- Parquet vs TSV output explanation +- pmultiqc citation added +- README updated with version table + +Closes #3, #9, #15" +``` + +--- + +## Task 12: Close resolved issues + +- [ ] **Step 1: Close issues via GitHub CLI** + +```bash +gh issue close 17 --repo bigbio/quantmsdiann --comment "Already implemented — --monitor-mod is extracted from diann_config.cfg (generated by sdrf-pipelines convert-diann) and passed to all DIA-NN steps via mod_flags." +gh issue close 2 --repo bigbio/quantmsdiann --comment "Superseded by #4 (Phase 6: consolidate param generation to sdrf-pipelines)." +gh issue close 1 --repo bigbio/quantmsdiann --comment "Resolved — docs/parameters.md created with comprehensive parameter reference." +gh issue close 3 --repo bigbio/quantmsdiann --comment "Resolved — diann_extra_args scope documented in docs/usage.md." +gh issue close 9 --repo bigbio/quantmsdiann --comment "Resolved — container version override guide and Singularity usage added to docs/usage.md." +gh issue close 15 --repo bigbio/quantmsdiann --comment "Resolved — docs/usage.md input documentation updated." +``` + +--- + +## Task 13: Final validation and push + +- [ ] **Step 1: Run full validation suite** + +```bash +conda run -n nfcore pre-commit run --all-files +conda run -n nfcore nf-core pipelines lint --release --dir . +``` + +Expected: 0 failures on both. + +- [ ] **Step 2: Push dda branch and create PR** + +```bash +git push -u origin dda +gh pr create --title "feat: v1.0.0 release — robustness, DDA support, features, docs" --body "$(cat <<'PREOF' +## Summary +- Robustness fixes: pipefail, error_retry, empty input guards +- DDA support via --diann_dda flag (DIA-NN >= 2.3.2) +- New params: --light-models, --export-quant, --site-ms1-quant +- InfinDIA groundwork (experimental) +- DIA-NN 2.3.2 version config +- New test configs: test_dda, test_dia_skip_preanalysis +- Comprehensive docs: parameters.md, complete usage.md, output.md + +## Issues +Closes #1, #3, #5, #7, #9, #10, #15, #17 + +## Test plan +- [ ] Existing CI tests pass (test_dia, test_dia_dotd) +- [ ] New test_dda passes with BSA dataset on DIA-NN 2.3.2 +- [ ] test_dia_skip_preanalysis passes +- [ ] nf-core lint --release: 0 failures +- [ ] pre-commit: all passing +PREOF +)" --base dev +``` From 7d4a441209de064827ea3c5d2148e2f00c5114e9 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 16:31:05 +0100 Subject: [PATCH 02/69] fix: add pipefail to all modules with tee pipes Without pipefail, if the command before tee fails, tee returns 0 and the Nextflow task appears to succeed. This masked failures in generate_cfg, diann_msstats, samplesheet_check, and sdrf_parsing. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../2026-04-03-v1-release-implementation.md | 22 +++++++- docs/v1-release-roadmap.md | 56 ++++++++++++------- modules/local/diann/diann_msstats/main.nf | 1 + modules/local/diann/generate_cfg/main.nf | 1 + modules/local/samplesheet_check/main.nf | 1 + modules/local/sdrf_parsing/main.nf | 1 + 6 files changed, 59 insertions(+), 23 deletions(-) diff --git a/docs/plans/2026-04-03-v1-release-implementation.md b/docs/plans/2026-04-03-v1-release-implementation.md index 4b3d0ed..8439513 100644 --- a/docs/plans/2026-04-03-v1-release-implementation.md +++ b/docs/plans/2026-04-03-v1-release-implementation.md @@ -13,6 +13,7 @@ ## Task 1: Fix tee pipes masking failures **Files:** + - Modify: `modules/local/diann/generate_cfg/main.nf:26` - Modify: `modules/local/diann/diann_msstats/main.nf:21-26` - Modify: `modules/local/samplesheet_check/main.nf:38-43` @@ -22,7 +23,7 @@ In `modules/local/diann/generate_cfg/main.nf`, find the `"""` opening the script block (line 20) and add `set -o pipefail` as the first line: -```groovy +````groovy """ set -o pipefail parse_sdrf generate-diann-cfg \\ @@ -71,13 +72,14 @@ git commit -m "fix: add pipefail to all modules with tee pipes Without pipefail, if the command before tee fails, tee returns 0 and the Nextflow task appears to succeed. This masked failures in generate_cfg, diann_msstats, samplesheet_check, and sdrf_parsing." -``` +```` --- ## Task 2: Add error retry to long-running DIA-NN tasks **Files:** + - Modify: `modules/local/diann/preliminary_analysis/main.nf:3-4` - Modify: `modules/local/diann/individual_analysis/main.nf:3-4` - Modify: `modules/local/diann/final_quantification/main.nf:3-4` @@ -102,6 +104,7 @@ Change to: ``` Do the same for: + - `individual_analysis/main.nf` (after `label 'diann'`) - `final_quantification/main.nf` (after `label 'diann'`) - `insilico_library_generation/main.nf` (after `label 'diann'`) @@ -124,6 +127,7 @@ retry on signal exits (130-145, 104, 175)." ## Task 3: Add empty input guards **Files:** + - Modify: `workflows/dia.nf:38,46` - [ ] **Step 1: Guard ch_searchdb with ifEmpty** @@ -174,6 +178,7 @@ with clear error messages instead of hanging indefinitely." ## Task 4: Add DIA-NN 2.3.2 version config and profile **Files:** + - Create: `conf/diann_versions/v2_3_2.config` - Modify: `nextflow.config:245-247` (profiles section) @@ -223,6 +228,7 @@ Enables DDA support and InfinDIA features." ## Task 5: Implement DDA support — params, version guard, flag passthrough **Files:** + - Modify: `nextflow.config:53-57` (DIA-NN general params) - Modify: `nextflow_schema.json` (DIA-NN section) - Modify: `workflows/dia.nf:35-38` (version guard) @@ -301,6 +307,7 @@ For each of the 5 DIA-NN modules, make two changes: Then append `${diann_dda_flag} \\` to the DIA-NN command, before `\${mod_flags}` (or before `$args` if no mod_flags). Apply to: + - `modules/local/diann/insilico_library_generation/main.nf` - `modules/local/diann/preliminary_analysis/main.nf` - `modules/local/diann/assemble_empirical_library/main.nf` @@ -329,6 +336,7 @@ Closes #5" ## Task 6: Add DDA test config **Files:** + - Create: `conf/tests/test_dda.config` - Modify: `.github/workflows/extended_ci.yml:110-191` (stage 2a) @@ -404,7 +412,7 @@ In `nextflow.config`, after the `test_dia_2_2_0` profile line (around line 241), In `.github/workflows/extended_ci.yml`, in the `test-latest` job matrix (around line 120), add `"test_dda"` to the `test_profile` array: ```yaml - test_profile: ["test_latest_dia", "test_dia_quantums", "test_dia_parquet", "test_dda"] +test_profile: ["test_latest_dia", "test_dia_quantums", "test_dia_parquet", "test_dda"] ``` - [ ] **Step 4: Validate and commit** @@ -424,6 +432,7 @@ extended_ci.yml stage 2a (private containers)." ## Task 7: Add test configs for skip_preliminary_analysis and speclib input **Files:** + - Create: `conf/tests/test_dia_skip_preanalysis.config` - Modify: `nextflow.config` (profiles section) - Modify: `.github/workflows/extended_ci.yml` (stage 2a) @@ -511,6 +520,7 @@ skipped and default mass accuracy parameters are used directly." ## Task 8: Add new DIA-NN feature parameters (light-models, export-quant, site-ms1-quant) **Files:** + - Modify: `nextflow.config` (params section) - Modify: `nextflow_schema.json` - Modify: `modules/local/diann/insilico_library_generation/main.nf` (light-models) @@ -578,6 +588,7 @@ All require DIA-NN >= 2.0." ## Task 9: Add InfinDIA groundwork **Files:** + - Modify: `nextflow.config` (params section) - Modify: `nextflow_schema.json` - Modify: `workflows/dia.nf` (version guard) @@ -640,6 +651,7 @@ No test config — InfinDIA requires large databases." ## Task 10: Documentation — parameters.md **Files:** + - Create: `docs/parameters.md` - [ ] **Step 1: Create comprehensive parameter reference** @@ -677,6 +689,7 @@ Closes #1" ## Task 11: Documentation — complete usage.md and output.md **Files:** + - Modify: `docs/usage.md` - Modify: `docs/output.md` - Modify: `CITATIONS.md` @@ -685,6 +698,7 @@ Closes #1" - [ ] **Step 1: Add DDA section to usage.md** Add a "DDA Analysis Mode" section after the Bruker/timsTOF section with: + - How to enable (`--diann_dda true -profile diann_v2_3_2`) - Limitations (beta, trusted columns only, PTM unreliable, MBR limited) - Example command @@ -693,6 +707,7 @@ Add a "DDA Analysis Mode" section after the Bruker/timsTOF section with: - [ ] **Step 2: Add missing param sections to usage.md** Add sections for: + - Preprocessing params (`reindex_mzml`, `mzml_statistics`, `convert_dotd`) - QC params (`enable_pmultiqc`, `skip_table_plots`, `contaminant_string`) - `diann_extra_args` scope per module @@ -704,6 +719,7 @@ Add sections for: - [ ] **Step 3: Update output.md** Add: + - Parquet vs TSV output explanation - MSstats format section - Intermediate outputs under `--verbose_modules` diff --git a/docs/v1-release-roadmap.md b/docs/v1-release-roadmap.md index 421648f..1961f20 100644 --- a/docs/v1-release-roadmap.md +++ b/docs/v1-release-roadmap.md @@ -29,6 +29,7 @@ DDA parallelization is identical to DIA — per-file parallel for PRELIMINARY_AN ### 1.1 Fix tee pipes masking failures Add `set -o pipefail` or `exit ${PIPESTATUS[0]}` to script blocks in: + - `modules/local/diann/generate_cfg/main.nf` - `modules/local/diann/diann_msstats/main.nf` - `modules/local/samplesheet_check/main.nf` @@ -39,6 +40,7 @@ Add `set -o pipefail` or `exit ${PIPESTATUS[0]}` to script blocks in: ### 1.2 Add error retry to long-running DIA-NN tasks Add `label 'error_retry'` to: + - PRELIMINARY_ANALYSIS (process_high) - INDIVIDUAL_ANALYSIS (process_high) - FINAL_QUANTIFICATION (process_high) @@ -55,12 +57,14 @@ These are the longest-running tasks and most susceptible to transient failures ( ### 1.4 New test configs **`conf/tests/test_dia_skip_preanalysis.config`:** + - Sets `skip_preliminary_analysis = true` - Uses default `mass_acc_ms1`, `mass_acc_ms2`, `scan_window` params - Same PXD026600 test data as test_dia - Validates the skip path that is currently untested in CI **`conf/tests/test_dia_speclib.config`:** + - Sets `diann_speclib` to a pre-built spectral library - Skips INSILICO_LIBRARY_GENERATION (the `if` branch in dia.nf line 55-56) - Requires a small test spectral library in quantms-test-datasets (or generated from existing test data) @@ -78,6 +82,7 @@ Build and push `ghcr.io/bigbio/diann:2.3.2` from existing Dockerfile at `quantms ### 2.2 Version config Add `conf/diann_versions/v2_3_2.config`: + ```groovy params.diann_version = '2.3.2' process { @@ -90,6 +95,7 @@ docker.enabled = true ``` Add profile in `nextflow.config`: + ```groovy diann_v2_3_2 { includeConfig 'conf/diann_versions/v2_3_2.config' } ``` @@ -97,11 +103,13 @@ diann_v2_3_2 { includeConfig 'conf/diann_versions/v2_3_2.config' } ### 2.3 DDA implementation **New param** in `nextflow.config`: + ```groovy diann_dda = false // Enable DDA analysis mode (requires DIA-NN >= 2.3.2) ``` **Version guard** in `workflows/dia.nf` at workflow start: + ```groovy if (params.diann_dda && params.diann_version < '2.3.2') { error("DDA mode requires DIA-NN >= 2.3.2. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") @@ -109,12 +117,15 @@ if (params.diann_dda && params.diann_version < '2.3.2') { ``` **Pass `--dda` to all DIA-NN modules** — In each module's script block, add: + ```groovy diann_dda_flag = params.diann_dda ? "--dda" : "" ``` + And append `${diann_dda_flag}` to the DIA-NN command. Add `'--dda'` to the `blocked` list in all 5 modules. **Accept DDA in create_input_channel** — Modify `create_input_channel/main.nf` lines 78-88: + ```groovy if (acqMethod.toLowerCase().contains("data-independent acquisition") || acqMethod.toLowerCase().contains("dia")) { meta.acquisition_method = "dia" @@ -135,6 +146,7 @@ Add `comment[proteomics data acquisition method]` column with value `NT=Data-Dep ### 2.5 Test config **`conf/tests/test_dda.config`:** + - Points to BSA dataset from `bigbio/quantms-test-datasets/testdata/lfq_ci/BSA/` - Sets `diann_dda = true` - Pins to `ghcr.io/bigbio/diann:2.3.2` @@ -151,22 +163,24 @@ Add `comment[proteomics data acquisition method]` column with value `NT=Data-Dep ### 3.1 New DIA-NN parameters -| Parameter | Flag | Min Version | Module | Default | -|---|---|---|---|---| -| `diann_light_models` | `--light-models` | 2.0 | INSILICO_LIBRARY_GENERATION | false | -| `diann_export_quant` | `--export-quant` | 2.0 | FINAL_QUANTIFICATION | false | -| `diann_read_threads` | `--read-threads N` | 2.0 | All DIA-NN steps | null (disabled) | -| `diann_site_ms1_quant` | `--site-ms1-quant` | 2.0 | FINAL_QUANTIFICATION | false | +| Parameter | Flag | Min Version | Module | Default | +| ---------------------- | ------------------ | ----------- | --------------------------- | --------------- | +| `diann_light_models` | `--light-models` | 2.0 | INSILICO_LIBRARY_GENERATION | false | +| `diann_export_quant` | `--export-quant` | 2.0 | FINAL_QUANTIFICATION | false | +| `diann_read_threads` | `--read-threads N` | 2.0 | All DIA-NN steps | null (disabled) | +| `diann_site_ms1_quant` | `--site-ms1-quant` | 2.0 | FINAL_QUANTIFICATION | false | Each parameter: add to `nextflow.config`, `nextflow_schema.json`, module script block (with version guard where needed), and module blocked list. ### 3.2 InfinDIA groundwork (issue #10) New params: + - `enable_infin_dia` (boolean, default: false) — requires >= 2.3.0 - `diann_pre_select` (integer, optional) — `--pre-select N` precursor limit Implementation: + - Pass `--infin-dia` to INSILICO_LIBRARY_GENERATION when enabled - Version guard: error if enabled with DIA-NN < 2.3.0 - No test config — InfinDIA needs large databases to be meaningful @@ -185,6 +199,7 @@ Implementation: ### 4.1 Create `docs/parameters.md` Comprehensive parameter reference with all ~70 params grouped by: + - Input/output options - File preparation (conversion, indexing, statistics) - DIA-NN general settings @@ -202,6 +217,7 @@ Each param: name, type, default, description, version requirement (if any). ### 4.2 Complete `docs/usage.md` Add missing sections: + - Preprocessing params (`reindex_mzml`, `mzml_statistics`, `convert_dotd`) - QC params (`enable_pmultiqc`, `skip_table_plots`, `contaminant_string`) - MultiQC options @@ -231,20 +247,20 @@ Add missing sections: ## Issues Status After Release -| Issue | Status | Resolution | -|---|---|---| -| #1 | Closed | Parameter documentation created | -| #2 | Closed | Superseded by #4 | -| #3 | Closed | ext.args scope documented | -| #5 | Closed | DDA support implemented | -| #7 | Closed | Phase 2 features wired | -| #9 | Closed | Container docs added | -| #10 | Partially closed | InfinDIA groundwork done, full support needs testing | -| #15 | Closed | Docs mismatch fixed | -| #17 | Closed | Already implemented | -| #4 | Open | Blocked on sdrf-pipelines converter release | -| #6 | Open | Blocked on PRIDE ontology release | -| #25 | Open | QPX deferred to next release | +| Issue | Status | Resolution | +| ----- | ---------------- | ---------------------------------------------------- | +| #1 | Closed | Parameter documentation created | +| #2 | Closed | Superseded by #4 | +| #3 | Closed | ext.args scope documented | +| #5 | Closed | DDA support implemented | +| #7 | Closed | Phase 2 features wired | +| #9 | Closed | Container docs added | +| #10 | Partially closed | InfinDIA groundwork done, full support needs testing | +| #15 | Closed | Docs mismatch fixed | +| #17 | Closed | Already implemented | +| #4 | Open | Blocked on sdrf-pipelines converter release | +| #6 | Open | Blocked on PRIDE ontology release | +| #25 | Open | QPX deferred to next release | --- diff --git a/modules/local/diann/diann_msstats/main.nf b/modules/local/diann/diann_msstats/main.nf index 4374b58..1844fdc 100644 --- a/modules/local/diann/diann_msstats/main.nf +++ b/modules/local/diann/diann_msstats/main.nf @@ -18,6 +18,7 @@ process DIANN_MSSTATS { script: def args = task.ext.args ?: '' """ + set -o pipefail quantmsutilsc diann2msstats \\ --report ${report} \\ --exp_design ${exp_design} \\ diff --git a/modules/local/diann/generate_cfg/main.nf b/modules/local/diann/generate_cfg/main.nf index 8377030..8e36641 100644 --- a/modules/local/diann/generate_cfg/main.nf +++ b/modules/local/diann/generate_cfg/main.nf @@ -18,6 +18,7 @@ process GENERATE_CFG { def args = task.ext.args ?: '' """ + set -o pipefail quantmsutilsc dianncfg \\ --enzyme "${meta.enzyme}" \\ --fix_mod "${meta.fixedmodifications}" \\ diff --git a/modules/local/samplesheet_check/main.nf b/modules/local/samplesheet_check/main.nf index f2b7112..76fae90 100644 --- a/modules/local/samplesheet_check/main.nf +++ b/modules/local/samplesheet_check/main.nf @@ -23,6 +23,7 @@ process SAMPLESHEET_CHECK { def string_use_ols_cache_only = params.use_ols_cache_only == true ? "--use_ols_cache_only" : "" """ + set -o pipefail # Get basename and create output filename BASENAME=\$(basename "${input_file}") # Remove .sdrf.tsv, .sdrf.csv, or .sdrf extension (in that order to match longest first) diff --git a/modules/local/sdrf_parsing/main.nf b/modules/local/sdrf_parsing/main.nf index a89bc61..61803f3 100644 --- a/modules/local/sdrf_parsing/main.nf +++ b/modules/local/sdrf_parsing/main.nf @@ -22,6 +22,7 @@ process SDRF_PARSING { def diann_version_flag = params.diann_version ? "--diann_version '${params.diann_version}'" : '' """ + set -o pipefail parse_sdrf convert-diann \\ -s ${sdrf} \\ ${mod_loc_flag} \\ From 853dbbf6000cd4fe8195ca94cf03a3188a07d4ab Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 16:32:22 +0100 Subject: [PATCH 03/69] fix: add error_retry label to all DIA-NN analysis modules These are the longest-running tasks and most susceptible to transient failures (OOM, I/O timeouts). The error_retry label enables automatic retry on signal exits (130-145, 104, 175). Co-Authored-By: Claude Opus 4.6 (1M context) --- modules/local/diann/assemble_empirical_library/main.nf | 1 + modules/local/diann/final_quantification/main.nf | 1 + modules/local/diann/individual_analysis/main.nf | 1 + modules/local/diann/insilico_library_generation/main.nf | 1 + modules/local/diann/preliminary_analysis/main.nf | 1 + 5 files changed, 5 insertions(+) diff --git a/modules/local/diann/assemble_empirical_library/main.nf b/modules/local/diann/assemble_empirical_library/main.nf index 034b95e..6efde30 100644 --- a/modules/local/diann/assemble_empirical_library/main.nf +++ b/modules/local/diann/assemble_empirical_library/main.nf @@ -2,6 +2,7 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { tag "$meta.experiment_id" label 'process_low' label 'diann' + label 'error_retry' container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? 'https://containers.biocontainers.pro/s3/SingImgsRepo/diann/v1.8.1_cv1/diann_v1.8.1_cv1.img' : diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index defb66d..42a0a6f 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -2,6 +2,7 @@ process FINAL_QUANTIFICATION { tag "$meta.experiment_id" label 'process_high' label 'diann' + label 'error_retry' container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? 'https://containers.biocontainers.pro/s3/SingImgsRepo/diann/v1.8.1_cv1/diann_v1.8.1_cv1.img' : diff --git a/modules/local/diann/individual_analysis/main.nf b/modules/local/diann/individual_analysis/main.nf index 28cb3b5..4a11628 100644 --- a/modules/local/diann/individual_analysis/main.nf +++ b/modules/local/diann/individual_analysis/main.nf @@ -2,6 +2,7 @@ process INDIVIDUAL_ANALYSIS { tag "$ms_file.baseName" label 'process_high' label 'diann' + label 'error_retry' container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? 'https://containers.biocontainers.pro/s3/SingImgsRepo/diann/v1.8.1_cv1/diann_v1.8.1_cv1.img' : diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index b347483..a26d9c6 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -2,6 +2,7 @@ process INSILICO_LIBRARY_GENERATION { tag "$fasta.name" label 'process_medium' label 'diann' + label 'error_retry' container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? 'https://containers.biocontainers.pro/s3/SingImgsRepo/diann/v1.8.1_cv1/diann_v1.8.1_cv1.img' : diff --git a/modules/local/diann/preliminary_analysis/main.nf b/modules/local/diann/preliminary_analysis/main.nf index 925114e..df88586 100644 --- a/modules/local/diann/preliminary_analysis/main.nf +++ b/modules/local/diann/preliminary_analysis/main.nf @@ -2,6 +2,7 @@ process PRELIMINARY_ANALYSIS { tag "$ms_file.baseName" label 'process_high' label 'diann' + label 'error_retry' container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? 'https://containers.biocontainers.pro/s3/SingImgsRepo/diann/v1.8.1_cv1/diann_v1.8.1_cv1.img' : From d85847ecee9f86905396f6cc48deea4a2bf596e7 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 16:33:21 +0100 Subject: [PATCH 04/69] fix: add empty input guards to prevent silent pipeline hangs Guard ch_searchdb and ch_experiment_meta with ifEmpty to fail fast with clear error messages instead of hanging indefinitely. Co-Authored-By: Claude Opus 4.6 (1M context) --- workflows/dia.nf | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/workflows/dia.nf b/workflows/dia.nf index d693a10..dabbb74 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -35,7 +35,9 @@ workflow DIA { main: ch_software_versions = channel.empty() - ch_searchdb = channel.fromPath(params.database, checkIfExists: true).first() + ch_searchdb = channel.fromPath(params.database, checkIfExists: true) + .ifEmpty { error("No protein database found at '${params.database}'. Provide --database ") } + .first() ch_file_preparation_results.multiMap { result -> @@ -43,7 +45,9 @@ workflow DIA { ms_file:result[1] }.set { ch_result } - ch_experiment_meta = ch_result.meta.unique { m -> m.experiment_id }.first() + ch_experiment_meta = ch_result.meta.unique { m -> m.experiment_id } + .ifEmpty { error("No valid input files found after SDRF parsing. Check your SDRF file and input paths.") } + .first() // diann_config.cfg comes directly from SDRF_PARSING (convert-diann) // Convert to value channel so it can be consumed by all per-file processes From f2a6777c8764ae1f73bb3a1c307f68df64a20b77 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 16:34:28 +0100 Subject: [PATCH 05/69] feat: add DIA-NN 2.3.2 version config and profile Adds conf/diann_versions/v2_3_2.config with ghcr.io/bigbio/diann:2.3.2 container. Use -profile diann_v2_3_2 to opt in. Default stays 1.8.1. Enables DDA support and InfinDIA features. Co-Authored-By: Claude Opus 4.6 (1M context) --- conf/diann_versions/v2_3_2.config | 14 ++++++++++++++ nextflow.config | 1 + 2 files changed, 15 insertions(+) create mode 100644 conf/diann_versions/v2_3_2.config diff --git a/conf/diann_versions/v2_3_2.config b/conf/diann_versions/v2_3_2.config new file mode 100644 index 0000000..2912f15 --- /dev/null +++ b/conf/diann_versions/v2_3_2.config @@ -0,0 +1,14 @@ +/* + * DIA-NN 2.3.2 container override (private ghcr.io) + * Latest release with DDA support and InfinDIA. + */ +params.diann_version = '2.3.2' + +process { + withLabel: diann { + container = 'ghcr.io/bigbio/diann:2.3.2' + } +} + +singularity.enabled = false +docker.enabled = true diff --git a/nextflow.config b/nextflow.config index 88c8c88..8ed056a 100644 --- a/nextflow.config +++ b/nextflow.config @@ -245,6 +245,7 @@ profiles { diann_v1_8_1 { includeConfig 'conf/diann_versions/v1_8_1.config' } diann_v2_1_0 { includeConfig 'conf/diann_versions/v2_1_0.config' } diann_v2_2_0 { includeConfig 'conf/diann_versions/v2_2_0.config' } + diann_v2_3_2 { includeConfig 'conf/diann_versions/v2_3_2.config' } dev { includeConfig 'conf/dev.config' } pride_slurm { includeConfig 'conf/pride_codon_slurm.config' } manual_wave { includeConfig 'conf/wave.config' } From 14ed74831132c0b0ae426223e409f057e5ddcf58 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 16:40:19 +0100 Subject: [PATCH 06/69] feat: add DDA support via --diann_dda flag (#5) - New param diann_dda (boolean, default: false) - Version guard: requires DIA-NN >= 2.3.2 - Passes --dda to all 5 DIA-NN modules when enabled - Accepts DDA acquisition method in SDRF when diann_dda=true - Added --dda to blocked lists in all modules Closes #5 Co-Authored-By: Claude Opus 4.6 (1M context) --- modules.json | 2 +- .../diann/assemble_empirical_library/main.nf | 4 +++- modules/local/diann/final_quantification/main.nf | 4 +++- modules/local/diann/individual_analysis/main.nf | 4 +++- .../diann/insilico_library_generation/main.nf | 4 +++- modules/local/diann/preliminary_analysis/main.nf | 4 +++- nextflow.config | 1 + nextflow_schema.json | 6 ++++++ subworkflows/local/create_input_channel/main.nf | 16 +++++++--------- workflows/dia.nf | 6 ++++++ 10 files changed, 36 insertions(+), 15 deletions(-) diff --git a/modules.json b/modules.json index c0e263c..be82844 100644 --- a/modules.json +++ b/modules.json @@ -46,4 +46,4 @@ } } } -} +} \ No newline at end of file diff --git a/modules/local/diann/assemble_empirical_library/main.nf b/modules/local/diann/assemble_empirical_library/main.nf index 6efde30..de7a9e4 100644 --- a/modules/local/diann/assemble_empirical_library/main.nf +++ b/modules/local/diann/assemble_empirical_library/main.nf @@ -32,7 +32,7 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { '--mass-acc', '--mass-acc-ms1', '--window', '--individual-mass-acc', '--individual-windows', '--out-lib', '--use-quant', '--gen-spec-lib', '--rt-profiling', - '--monitor-mod', '--var-mod', '--fixed-mod'] + '--monitor-mod', '--var-mod', '--fixed-mod', '--dda'] // Sort by length descending so longer flags (e.g. --mass-acc-ms1) are matched before shorter prefixes (--mass-acc) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -53,6 +53,7 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" diann_tims_sum = params.diann_tims_sum ? "--quant-tims-sum" : "" diann_im_window = params.diann_im_window ? "--im-window $params.diann_im_window" : "" + diann_dda_flag = params.diann_dda ? "--dda" : "" """ # Precursor Tolerance value was: ${meta['precursormasstolerance']} @@ -79,6 +80,7 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { ${diann_no_peptidoforms} \\ ${diann_tims_sum} \\ ${diann_im_window} \\ + ${diann_dda_flag} \\ \${mod_flags} \\ $args diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index 42a0a6f..8e6c76d 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -48,7 +48,7 @@ process FINAL_QUANTIFICATION { '--use-quant', '--matrices', '--out', '--relaxed-prot-inf', '--pg-level', '--qvalue', '--window', '--individual-windows', '--species-genes', '--report-decoys', '--xic', '--no-norm', - '--monitor-mod', '--var-mod', '--fixed-mod'] + '--monitor-mod', '--var-mod', '--fixed-mod', '--dda'] // Sort by length descending so longer flags (e.g. --individual-windows) are matched before shorter prefixes (--window) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -69,6 +69,7 @@ process FINAL_QUANTIFICATION { quantums_params = params.quantums_params ? "--quant-params $params.quantums_params": "" diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" diann_use_quant = params.diann_use_quant ? "--use-quant" : "" + diann_dda_flag = params.diann_dda ? "--dda" : "" """ # Notes: if .quant files are passed, mzml/.d files are not accessed, so the name needs to be passed but files @@ -98,6 +99,7 @@ process FINAL_QUANTIFICATION { ${quantums_params} \\ ${diann_no_peptidoforms} \\ ${diann_use_quant} \\ + ${diann_dda_flag} \\ \${mod_flags} \\ $args diff --git a/modules/local/diann/individual_analysis/main.nf b/modules/local/diann/individual_analysis/main.nf index 4a11628..1e24905 100644 --- a/modules/local/diann/individual_analysis/main.nf +++ b/modules/local/diann/individual_analysis/main.nf @@ -28,7 +28,7 @@ process INDIVIDUAL_ANALYSIS { '--mass-acc', '--mass-acc-ms1', '--window', '--no-ifs-removal', '--no-main-report', '--relaxed-prot-inf', '--pg-level', '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', - '--monitor-mod', '--var-mod', '--fixed-mod'] + '--monitor-mod', '--var-mod', '--fixed-mod', '--dda'] // Sort by length descending so longer flags (e.g. --mass-acc-ms1) are matched before shorter prefixes (--mass-acc) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -82,6 +82,7 @@ process INDIVIDUAL_ANALYSIS { diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" diann_tims_sum = params.diann_tims_sum ? "--quant-tims-sum" : "" diann_im_window = params.diann_im_window ? "--im-window $params.diann_im_window" : "" + diann_dda_flag = params.diann_dda ? "--dda" : "" // Per-file scan ranges from SDRF (empty = no flag, DIA-NN auto-detects) min_pr_mz = meta['ms1minmz'] ? "--min-pr-mz ${meta['ms1minmz']}" : "" @@ -113,6 +114,7 @@ process INDIVIDUAL_ANALYSIS { ${diann_no_peptidoforms} \\ ${diann_tims_sum} \\ ${diann_im_window} \\ + ${diann_dda_flag} \\ \${mod_flags} \\ $args diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index a26d9c6..321cc2f 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -30,7 +30,7 @@ process INSILICO_LIBRARY_GENERATION { '--missed-cleavages', '--min-pep-len', '--max-pep-len', '--min-pr-charge', '--max-pr-charge', '--var-mods', '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', - '--met-excision', '--monitor-mod'] + '--met-excision', '--monitor-mod', '--dda'] // Sort by length descending so longer flags (e.g. --fasta-search) are matched before shorter prefixes (--fasta, --f) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -46,6 +46,7 @@ process INSILICO_LIBRARY_GENERATION { max_fr_mz = params.max_fr_mz ? "--max-fr-mz $params.max_fr_mz":"" met_excision = params.met_excision ? "--met-excision" : "" diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" + diann_dda_flag = params.diann_dda ? "--dda" : "" """ diann `cat ${diann_config}` \\ @@ -67,6 +68,7 @@ process INSILICO_LIBRARY_GENERATION { --gen-spec-lib \\ ${diann_no_peptidoforms} \\ ${met_excision} \\ + ${diann_dda_flag} \\ ${args} cp *lib.log.txt silicolibrarygeneration.log diff --git a/modules/local/diann/preliminary_analysis/main.nf b/modules/local/diann/preliminary_analysis/main.nf index df88586..791dab3 100644 --- a/modules/local/diann/preliminary_analysis/main.nf +++ b/modules/local/diann/preliminary_analysis/main.nf @@ -28,7 +28,7 @@ process PRELIMINARY_ANALYSIS { '--mass-acc', '--mass-acc-ms1', '--window', '--quick-mass-acc', '--min-corr', '--corr-diff', '--time-corr-only', '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', - '--monitor-mod', '--var-mod', '--fixed-mod', '--no-prot-inf'] + '--monitor-mod', '--var-mod', '--fixed-mod', '--no-prot-inf', '--dda'] // Sort by length descending so longer flags (e.g. --mass-acc-ms1) are matched before shorter prefixes (--mass-acc) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -67,6 +67,7 @@ process PRELIMINARY_ANALYSIS { scan_window = params.scan_window_automatic ? '' : "--window $params.scan_window" diann_tims_sum = params.diann_tims_sum ? "--quant-tims-sum" : "" diann_im_window = params.diann_im_window ? "--im-window $params.diann_im_window" : "" + diann_dda_flag = params.diann_dda ? "--dda" : "" // Per-file scan ranges from SDRF (empty = no flag, DIA-NN auto-detects) min_pr_mz = meta['ms1minmz'] ? "--min-pr-mz ${meta['ms1minmz']}" : "" @@ -102,6 +103,7 @@ process PRELIMINARY_ANALYSIS { ${diann_tims_sum} \\ ${diann_im_window} \\ --no-prot-inf \\ + ${diann_dda_flag} \\ \${mod_flags} \\ $args diff --git a/nextflow.config b/nextflow.config index 8ed056a..3340937 100644 --- a/nextflow.config +++ b/nextflow.config @@ -55,6 +55,7 @@ params { diann_debug = 3 diann_speclib = null diann_extra_args = null + diann_dda = false // Enable DDA analysis mode (requires DIA-NN >= 2.3.2) // Optional outputs — control which intermediate files are published save_speclib_tsv = false // Save the TSV spectral library from in-silico generation diff --git a/nextflow_schema.json b/nextflow_schema.json index a4e288e..4a65114 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -469,6 +469,12 @@ "hidden": false, "help_text": "Pass additional DIA-NN command-line arguments that will be appended to all DIA-NN steps (INSILICO_LIBRARY_GENERATION, PRELIMINARY_ANALYSIS, ASSEMBLE_EMPIRICAL_LIBRARY, INDIVIDUAL_ANALYSIS, FINAL_QUANTIFICATION). Flags that conflict with a specific step are automatically stripped with a warning. For step-specific overrides, use custom Nextflow config files with ext.args." }, + "diann_dda": { + "type": "boolean", + "description": "Enable DDA (Data-Dependent Acquisition) analysis mode. Passes --dda to all DIA-NN steps. Requires DIA-NN >= 2.3.2 (use -profile diann_v2_3_2). Beta feature.", + "fa_icon": "fas fa-flask", + "default": false + }, "save_speclib_tsv": { "type": "boolean", "default": false, diff --git a/subworkflows/local/create_input_channel/main.nf b/subworkflows/local/create_input_channel/main.nf index 588e144..c0249f1 100644 --- a/subworkflows/local/create_input_channel/main.nf +++ b/subworkflows/local/create_input_channel/main.nf @@ -72,18 +72,16 @@ def create_meta_channel(LinkedHashMap row, enzymes, files, wrapper) { exit(1, "ERROR: Please check input file -> File Uri does not exist!\n${filestr}") } - // Validate acquisition method is DIA - // AcquisitionMethod is already extracted by convert-diann (e.g. "Data-Independent Acquisition") + // Validate acquisition method def acqMethod = row.AcquisitionMethod?.toString()?.trim() ?: "" if (acqMethod.toLowerCase().contains("data-independent acquisition") || acqMethod.toLowerCase().contains("dia")) { meta.acquisition_method = "dia" - } - else if (acqMethod.isEmpty()) { - // If no acquisition method column in SDRF, assume DIA (this is a DIA-only pipeline) - meta.acquisition_method = "dia" - } - else { - log.error("This pipeline only supports Data-Independent Acquisition (DIA). Found: '${acqMethod}'. Use the quantms pipeline for DDA workflows.") + } else if (params.diann_dda && (acqMethod.toLowerCase().contains("data-dependent acquisition") || acqMethod.toLowerCase().contains("dda"))) { + meta.acquisition_method = "dda" + } else if (acqMethod.isEmpty()) { + meta.acquisition_method = params.diann_dda ? "dda" : "dia" + } else { + log.error("Unsupported acquisition method: '${acqMethod}'. This pipeline supports DIA" + (params.diann_dda ? " and DDA (--diann_dda)" : "") + ". Found in file: ${filestr}") exit(1) } diff --git a/workflows/dia.nf b/workflows/dia.nf index dabbb74..7041d80 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -35,6 +35,12 @@ workflow DIA { main: ch_software_versions = channel.empty() + + // Version guard for DDA mode + if (params.diann_dda && params.diann_version < '2.3.2') { + error("DDA mode (--diann_dda) requires DIA-NN >= 2.3.2. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") + } + ch_searchdb = channel.fromPath(params.database, checkIfExists: true) .ifEmpty { error("No protein database found at '${params.database}'. Provide --database ") } .first() From 7f51d00f3cabd13275e4ed3eaea7b238ef160273 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 16:42:13 +0100 Subject: [PATCH 07/69] test: add DDA and skip_preanalysis test configs - test_dda: BSA dataset with diann_dda=true on DIA-NN 2.3.2 - test_dia_skip_preanalysis: tests previously untested skip path Both added to extended_ci.yml stage 2a. Co-Authored-By: Claude Opus 4.6 (1M context) --- .github/workflows/extended_ci.yml | 3 +- conf/tests/test_dda.config | 52 +++++++++++++++++++++ conf/tests/test_dia_skip_preanalysis.config | 48 +++++++++++++++++++ modules.json | 2 +- nextflow.config | 2 + 5 files changed, 105 insertions(+), 2 deletions(-) create mode 100644 conf/tests/test_dda.config create mode 100644 conf/tests/test_dia_skip_preanalysis.config diff --git a/.github/workflows/extended_ci.yml b/.github/workflows/extended_ci.yml index 42a9f48..621ab2c 100644 --- a/.github/workflows/extended_ci.yml +++ b/.github/workflows/extended_ci.yml @@ -114,7 +114,8 @@ jobs: strategy: fail-fast: false matrix: - test_profile: ["test_latest_dia", "test_dia_quantums", "test_dia_parquet"] + test_profile: + ["test_latest_dia", "test_dia_quantums", "test_dia_parquet", "test_dda", "test_dia_skip_preanalysis"] env: NXF_ANSI_LOG: false CAPSULE_LOG: none diff --git a/conf/tests/test_dda.config b/conf/tests/test_dda.config new file mode 100644 index 0000000..c0ebb24 --- /dev/null +++ b/conf/tests/test_dda.config @@ -0,0 +1,52 @@ +/* +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Nextflow config file for testing DDA analysis (requires DIA-NN >= 2.3.2) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Tests DDA mode using the BSA dataset with --diann_dda flag. + Uses ghcr.io/bigbio/diann:2.3.2. + + Use as follows: + nextflow run bigbio/quantmsdiann -profile test_dda,docker [--outdir ] + +------------------------------------------------------------------------------------------------ +*/ + +process { + resourceLimits = [ + cpus: 4, + memory: '12.GB', + time: '48.h' + ] +} + +params { + config_profile_name = 'Test profile for DDA analysis' + config_profile_description = 'DDA test using BSA dataset with DIA-NN 2.3.2.' + + outdir = './results_dda' + + // Input data - BSA DDA dataset + input = 'https://raw.githubusercontent.com/bigbio/quantms-test-datasets/quantms/testdata/lfq_ci/BSA/BSA_design.sdrf.tsv' + database = 'https://raw.githubusercontent.com/bigbio/quantms-test-datasets/quantms/testdata/lfq_ci/BSA/18Protein_SoCe_Tr_detergents_trace.fasta' + + // DDA mode + diann_dda = true + + // Search parameters matching BSA dataset + min_peptide_length = 7 + max_peptide_length = 30 + max_precursor_charge = 3 + allowed_missed_cleavages = 1 + diann_normalize = false + publish_dir_mode = 'symlink' + max_mods = 2 +} + +process { + withLabel: diann { + container = 'ghcr.io/bigbio/diann:2.3.2' + } +} + +singularity.enabled = false +docker.enabled = true diff --git a/conf/tests/test_dia_skip_preanalysis.config b/conf/tests/test_dia_skip_preanalysis.config new file mode 100644 index 0000000..e1968b5 --- /dev/null +++ b/conf/tests/test_dia_skip_preanalysis.config @@ -0,0 +1,48 @@ +/* +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Nextflow config file for testing skip_preliminary_analysis path +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Tests the pipeline with skip_preliminary_analysis=true, using default + mass accuracy parameters. Validates the untested code path in dia.nf. + + Use as follows: + nextflow run bigbio/quantmsdiann -profile test_dia_skip_preanalysis,docker [--outdir ] + +------------------------------------------------------------------------------------------------ +*/ + +process { + resourceLimits = [ + cpus: 4, + memory: '12.GB', + time: '48.h' + ] +} + +params { + config_profile_name = 'Test profile for skip preliminary analysis' + config_profile_description = 'Tests skip_preliminary_analysis path with default mass accuracy params.' + + outdir = './results_skip_preanalysis' + + // Input data - same as test_dia + input = 'https://raw.githubusercontent.com/bigbio/quantms-test-datasets/quantms/testdata/dia_ci/PXD026600.sdrf.tsv' + database = 'https://raw.githubusercontent.com/bigbio/quantms-test-datasets/quantms/testdata/dia_ci/REF_EColi_K12_UPS1_combined.fasta' + min_pr_mz = 350 + max_pr_mz = 950 + min_fr_mz = 500 + max_fr_mz = 1500 + min_peptide_length = 15 + max_peptide_length = 30 + max_precursor_charge = 3 + allowed_missed_cleavages = 1 + diann_normalize = false + publish_dir_mode = 'symlink' + max_mods = 2 + + // Skip preliminary analysis - use default mass accuracy params + skip_preliminary_analysis = true + mass_acc_ms2 = 15 + mass_acc_ms1 = 15 + scan_window = 8 +} diff --git a/modules.json b/modules.json index be82844..c0e263c 100644 --- a/modules.json +++ b/modules.json @@ -46,4 +46,4 @@ } } } -} \ No newline at end of file +} diff --git a/nextflow.config b/nextflow.config index 3340937..ef0f778 100644 --- a/nextflow.config +++ b/nextflow.config @@ -242,6 +242,8 @@ profiles { test_dia_2_2_0 { includeConfig 'conf/tests/test_dia_2_2_0.config' } test_latest_dia { includeConfig 'conf/tests/test_latest_dia.config' } test_full_dia { includeConfig 'conf/tests/test_full_dia.config' } + test_dda { includeConfig 'conf/tests/test_dda.config' } + test_dia_skip_preanalysis { includeConfig 'conf/tests/test_dia_skip_preanalysis.config' } // DIA-NN version overrides (used by merge_ci.yml matrix) diann_v1_8_1 { includeConfig 'conf/diann_versions/v1_8_1.config' } diann_v2_1_0 { includeConfig 'conf/diann_versions/v2_1_0.config' } From a13d4b3cb8b5ea20268caf84a04e1dd26195a497 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 16:44:56 +0100 Subject: [PATCH 08/69] feat: add --light-models, --export-quant, --site-ms1-quant params (#7) - diann_light_models: 10x faster in-silico library generation - diann_export_quant: fragment-level parquet export - diann_site_ms1_quant: MS1 apex intensities for PTM quantification All require DIA-NN >= 2.0. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../local/diann/final_quantification/main.nf | 6 +++++- .../diann/insilico_library_generation/main.nf | 4 +++- nextflow.config | 3 +++ nextflow_schema.json | 18 ++++++++++++++++++ 4 files changed, 29 insertions(+), 2 deletions(-) diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index 8e6c76d..fc8bbd0 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -48,7 +48,7 @@ process FINAL_QUANTIFICATION { '--use-quant', '--matrices', '--out', '--relaxed-prot-inf', '--pg-level', '--qvalue', '--window', '--individual-windows', '--species-genes', '--report-decoys', '--xic', '--no-norm', - '--monitor-mod', '--var-mod', '--fixed-mod', '--dda'] + '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', '--export-quant', '--site-ms1-quant'] // Sort by length descending so longer flags (e.g. --individual-windows) are matched before shorter prefixes (--window) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -70,6 +70,8 @@ process FINAL_QUANTIFICATION { diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" diann_use_quant = params.diann_use_quant ? "--use-quant" : "" diann_dda_flag = params.diann_dda ? "--dda" : "" + diann_export_quant = params.diann_export_quant ? "--export-quant" : "" + diann_site_ms1_quant = params.diann_site_ms1_quant ? "--site-ms1-quant" : "" """ # Notes: if .quant files are passed, mzml/.d files are not accessed, so the name needs to be passed but files @@ -100,6 +102,8 @@ process FINAL_QUANTIFICATION { ${diann_no_peptidoforms} \\ ${diann_use_quant} \\ ${diann_dda_flag} \\ + ${diann_export_quant} \\ + ${diann_site_ms1_quant} \\ \${mod_flags} \\ $args diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index 321cc2f..2937af2 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -30,7 +30,7 @@ process INSILICO_LIBRARY_GENERATION { '--missed-cleavages', '--min-pep-len', '--max-pep-len', '--min-pr-charge', '--max-pr-charge', '--var-mods', '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', - '--met-excision', '--monitor-mod', '--dda'] + '--met-excision', '--monitor-mod', '--dda', '--light-models'] // Sort by length descending so longer flags (e.g. --fasta-search) are matched before shorter prefixes (--fasta, --f) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -47,6 +47,7 @@ process INSILICO_LIBRARY_GENERATION { met_excision = params.met_excision ? "--met-excision" : "" diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" diann_dda_flag = params.diann_dda ? "--dda" : "" + diann_light_models = params.diann_light_models ? "--light-models" : "" """ diann `cat ${diann_config}` \\ @@ -67,6 +68,7 @@ process INSILICO_LIBRARY_GENERATION { --verbose $params.diann_debug \\ --gen-spec-lib \\ ${diann_no_peptidoforms} \\ + ${diann_light_models} \\ ${met_excision} \\ ${diann_dda_flag} \\ ${args} diff --git a/nextflow.config b/nextflow.config index ef0f778..2ad10a1 100644 --- a/nextflow.config +++ b/nextflow.config @@ -56,6 +56,9 @@ params { diann_speclib = null diann_extra_args = null diann_dda = false // Enable DDA analysis mode (requires DIA-NN >= 2.3.2) + diann_light_models = false // add '--light-models' for 10x faster library generation (DIA-NN >= 2.0) + diann_export_quant = false // add '--export-quant' for fragment-level parquet export (DIA-NN >= 2.0) + diann_site_ms1_quant = false // add '--site-ms1-quant' for MS1 apex PTM quantification (DIA-NN >= 2.0) // Optional outputs — control which intermediate files are published save_speclib_tsv = false // Save the TSV spectral library from in-silico generation diff --git a/nextflow_schema.json b/nextflow_schema.json index 4a65114..4d36360 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -475,6 +475,24 @@ "fa_icon": "fas fa-flask", "default": false }, + "diann_light_models": { + "type": "boolean", + "description": "Enable --light-models for 10x faster in-silico library generation (DIA-NN >= 2.0).", + "fa_icon": "fas fa-bolt", + "default": false + }, + "diann_export_quant": { + "type": "boolean", + "description": "Enable --export-quant for fragment-level parquet data export (DIA-NN >= 2.0).", + "fa_icon": "fas fa-file-export", + "default": false + }, + "diann_site_ms1_quant": { + "type": "boolean", + "description": "Enable --site-ms1-quant to use MS1 apex intensities for PTM site quantification (DIA-NN >= 2.0).", + "fa_icon": "fas fa-crosshairs", + "default": false + }, "save_speclib_tsv": { "type": "boolean", "default": false, From 0c56dcb0bf4589c1d7e610e5862d6830aa448b93 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 16:46:53 +0100 Subject: [PATCH 09/69] =?UTF-8?q?feat:=20add=20InfinDIA=20groundwork=20?= =?UTF-8?q?=E2=80=94=20enable=5Finfin=5Fdia=20param=20(#10)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Experimental support for InfinDIA (DIA-NN 2.3.0+). Passes --infin-dia to library generation when enabled. Version guard enforces >= 2.3.0. No test config — InfinDIA requires large databases. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../local/diann/insilico_library_generation/main.nf | 7 ++++++- nextflow.config | 4 ++++ nextflow_schema.json | 11 +++++++++++ workflows/dia.nf | 5 +++++ 4 files changed, 26 insertions(+), 1 deletion(-) diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index 2937af2..7de4665 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -30,7 +30,8 @@ process INSILICO_LIBRARY_GENERATION { '--missed-cleavages', '--min-pep-len', '--max-pep-len', '--min-pr-charge', '--max-pr-charge', '--var-mods', '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', - '--met-excision', '--monitor-mod', '--dda', '--light-models'] + '--met-excision', '--monitor-mod', '--dda', '--light-models', + '--infin-dia', '--pre-select'] // Sort by length descending so longer flags (e.g. --fasta-search) are matched before shorter prefixes (--fasta, --f) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -48,6 +49,8 @@ process INSILICO_LIBRARY_GENERATION { diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" diann_dda_flag = params.diann_dda ? "--dda" : "" diann_light_models = params.diann_light_models ? "--light-models" : "" + infin_dia_flag = params.enable_infin_dia ? "--infin-dia" : "" + pre_select_flag = params.diann_pre_select ? "--pre-select $params.diann_pre_select" : "" """ diann `cat ${diann_config}` \\ @@ -69,6 +72,8 @@ process INSILICO_LIBRARY_GENERATION { --gen-spec-lib \\ ${diann_no_peptidoforms} \\ ${diann_light_models} \\ + ${infin_dia_flag} \\ + ${pre_select_flag} \\ ${met_excision} \\ ${diann_dda_flag} \\ ${args} diff --git a/nextflow.config b/nextflow.config index 2ad10a1..0f1ece2 100644 --- a/nextflow.config +++ b/nextflow.config @@ -60,6 +60,10 @@ params { diann_export_quant = false // add '--export-quant' for fragment-level parquet export (DIA-NN >= 2.0) diann_site_ms1_quant = false // add '--site-ms1-quant' for MS1 apex PTM quantification (DIA-NN >= 2.0) + // DIA-NN: InfinDIA (experimental, v2.3.0+) + enable_infin_dia = false // Enable InfinDIA for ultra-large search spaces + diann_pre_select = null // --pre-select N precursor limit for InfinDIA + // Optional outputs — control which intermediate files are published save_speclib_tsv = false // Save the TSV spectral library from in-silico generation diff --git a/nextflow_schema.json b/nextflow_schema.json index 4d36360..f082614 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -493,6 +493,17 @@ "fa_icon": "fas fa-crosshairs", "default": false }, + "enable_infin_dia": { + "type": "boolean", + "description": "Enable InfinDIA for ultra-large search spaces (DIA-NN >= 2.3.0). Experimental.", + "fa_icon": "fas fa-infinity", + "default": false + }, + "diann_pre_select": { + "type": "integer", + "description": "Set --pre-select N precursor limit for InfinDIA pre-search.", + "fa_icon": "fas fa-filter" + }, "save_speclib_tsv": { "type": "boolean", "default": false, diff --git a/workflows/dia.nf b/workflows/dia.nf index 7041d80..899d226 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -41,6 +41,11 @@ workflow DIA { error("DDA mode (--diann_dda) requires DIA-NN >= 2.3.2. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") } + // Version guard for InfinDIA + if (params.enable_infin_dia && params.diann_version < '2.3.0') { + error("InfinDIA requires DIA-NN >= 2.3.0. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") + } + ch_searchdb = channel.fromPath(params.database, checkIfExists: true) .ifEmpty { error("No protein database found at '${params.database}'. Provide --database ") } .first() From e69d7f2df75bb2fceeb2e31d19dba7daf438a843 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 16:49:50 +0100 Subject: [PATCH 10/69] docs: add comprehensive parameter reference (#1) Complete reference for all ~70 pipeline parameters grouped by category with types, defaults, descriptions, and version requirements. Closes #1 Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/parameters.md | 162 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 162 insertions(+) create mode 100644 docs/parameters.md diff --git a/docs/parameters.md b/docs/parameters.md new file mode 100644 index 0000000..fb6f229 --- /dev/null +++ b/docs/parameters.md @@ -0,0 +1,162 @@ +# Pipeline Parameters + +Complete reference for all `bigbio/quantmsdiann` pipeline parameters. +Parameters are specified on the command line as `--parameter_name value` or +in a Nextflow config file. + +## Input/Output Options + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--input` | string | `null` | Path or URI to an SDRF file (.sdrf, .tsv, or .csv). Acquisition method, labelling type, enzyme, and fixed modifications are read exclusively from the SDRF. | +| `--database` | string | `null` | Path to the FASTA protein database. Must not contain decoys for DIA data. | +| `--outdir` | string | `./results` | Output directory where results will be saved. | +| `--publish_dir_mode` | string | `copy` | Method used to save pipeline results. Options: `symlink`, `rellink`, `link`, `copy`, `copyNoFollow`, `move`. | +| `--root_folder` | string | `null` | Root folder in which spectrum files specified in the SDRF are searched. Used when files are available locally. | +| `--local_input_type` | string | `mzML` | Override the file type/extension of filenames in the SDRF when using `--root_folder`. Options: `mzML`, `raw`, `d`, `dia`. Compressed variants (.gz, .tar, .tar.gz, .zip) are supported. | + +## SDRF Validation + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--use_ols_cache_only` | boolean | `true` | Use only the cached Ontology Lookup Service (OLS) for term validation, avoiding network requests. | + +## File Preparation + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--reindex_mzml` | boolean | `true` | Force re-indexing of input mzML files at the start of the pipeline. Also fixes common issues in slightly incomplete mzMLs. | +| `--mzml_statistics` | boolean | `false` | Compute MS1/MS2 statistics from mzML files. Generates `*_ms_info.parquet` for QC reporting. Bruker .d files are always skipped. | +| `--mzml_features` | boolean | `false` | Compute MS1-level features during the mzML statistics step. Only available for mzML files. | +| `--convert_dotd` | boolean | `false` | Convert Bruker .d files to mzML format before processing. | + +## Search Parameters + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--met_excision` | boolean | `true` | Account for N-terminal methionine excision during database search. | +| `--allowed_missed_cleavages` | integer | `2` | Maximum number of allowed missed enzyme cleavages per peptide. | +| `--precursor_mass_tolerance` | integer | `5` | Precursor mass tolerance for database search. See also `--precursor_mass_tolerance_unit`. Can be overridden from SDRF. | +| `--precursor_mass_tolerance_unit` | string | `ppm` | Unit for precursor mass tolerance. Options: `ppm`, `Da`. | +| `--fragment_mass_tolerance` | number | `0.03` | Fragment mass tolerance for database search. Can be overridden from SDRF. | +| `--fragment_mass_tolerance_unit` | string | `Da` | Unit for fragment mass tolerance. Options: `ppm`, `Da`. | +| `--variable_mods` | string | `Oxidation (M)` | Comma-separated list of variable modifications using Unimod names (e.g. `Oxidation (M),Carbamidomethyl (C)`). Can be overridden from SDRF. | +| `--min_precursor_charge` | integer | `2` | Minimum precursor ion charge. | +| `--max_precursor_charge` | integer | `4` | Maximum precursor ion charge. | +| `--min_peptide_length` | integer | `6` | Minimum peptide length to consider. | +| `--max_peptide_length` | integer | `40` | Maximum peptide length to consider. | +| `--max_mods` | integer | `3` | Maximum number of modifications per peptide. Large values may slow the search considerably. | +| `--min_pr_mz` | number | `400` | Minimum precursor m/z for in-silico library generation or library-free search. | +| `--max_pr_mz` | number | `2400` | Maximum precursor m/z for in-silico library generation or library-free search. | +| `--min_fr_mz` | number | `100` | Minimum fragment m/z for in-silico library generation or library-free search. | +| `--max_fr_mz` | number | `1800` | Maximum fragment m/z for in-silico library generation or library-free search. | + +## DIA-NN General + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--diann_version` | string | `1.8.1` | DIA-NN version used by the workflow. Controls version-dependent flags (e.g. `--monitor-mod` for 1.8.x). | +| `--diann_debug` | integer | `3` | DIA-NN debug/verbosity level. Allowed values: 0, 1, 2, 3, 4. | +| `--diann_speclib` | string | `null` | Path to an existing spectral library. If provided, the pipeline uses it instead of predicting one from the FASTA. | +| `--diann_extra_args` | string | `null` | Extra command-line arguments appended to all DIA-NN steps. Flags incompatible with a specific step are automatically stripped with a warning. | +| `--diann_dda` | boolean | `false` | Enable DDA (Data-Dependent Acquisition) analysis mode. Passes `--dda` to all DIA-NN steps. Requires DIA-NN >= 2.3.2. Beta feature. | +| `--diann_light_models` | boolean | `false` | Enable `--light-models` for 10x faster in-silico library generation. Requires DIA-NN >= 2.0. | +| `--diann_export_quant` | boolean | `false` | Enable `--export-quant` for fragment-level parquet data export. Requires DIA-NN >= 2.0. | +| `--diann_site_ms1_quant` | boolean | `false` | Enable `--site-ms1-quant` to use MS1 apex intensities for PTM site quantification. Requires DIA-NN >= 2.0. | + +## Mass Accuracy and Calibration + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--mass_acc_automatic` | boolean | `true` | Automatically determine the MS2 mass accuracy setting. | +| `--mass_acc_ms1` | number | `15` | MS1 mass accuracy in ppm. Overrides automatic calibration when set. Maps to DIA-NN `--mass-acc-ms1`. | +| `--mass_acc_ms2` | number | `15` | MS2 mass accuracy in ppm. Overrides automatic calibration when set. Maps to DIA-NN `--mass-acc`. | +| `--scan_window` | integer | `8` | Scan window radius. Ideally approximately equal to the average number of data points per peak. | +| `--scan_window_automatic` | boolean | `true` | Automatically determine the scan window setting. | +| `--quick_mass_acc` | boolean | `true` | Use a fast heuristic algorithm instead of ID-number optimization when choosing MS2 mass accuracy automatically. | +| `--performance_mode` | boolean | `true` | Enable low-RAM/high-speed mode. Adds `--min-corr 2 --corr-diff 1 --time-corr-only` to DIA-NN. | + +## Bruker/timsTOF + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--diann_tims_sum` | boolean | `false` | Enable `--quant-tims-sum` for slice/scanning timsTOF methods. Highly recommended for Synchro-PASEF. | +| `--diann_im_window` | number | `null` | Set `--im-window` to ensure the ion mobility extraction window is not smaller than the specified value. | + +## PTM Localization + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--enable_mod_localization` | boolean | `false` | Enable modification localization scoring in DIA-NN (`--monitor-mod`). | +| `--mod_localization` | string | `Phospho (S),Phospho (T),Phospho (Y)` | Comma-separated modification names or UniMod accessions for localization (e.g. `UniMod:21,UniMod:1`). | + +## Library Generation + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--save_speclib_tsv` | boolean | `false` | Publish the human-readable TSV spectral library from the in-silico generation step to the output directory. | + +## Preliminary Analysis + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--skip_preliminary_analysis` | boolean | `false` | Skip the preliminary analysis step and use the provided spectral library as-is instead of building a local consensus library. | +| `--empirical_assembly_log` | string | `null` | Path to a pre-existing empirical assembly log file. Only used when `--skip_preliminary_analysis true` and `--diann_speclib` are set. | +| `--random_preanalysis` | boolean | `false` | Enable random selection of spectrum files for empirical library generation. | +| `--random_preanalysis_seed` | integer | `42` | Random seed for spectrum file selection when `--random_preanalysis` is enabled. | +| `--empirical_assembly_ms_n` | integer | `200` | Number of randomly selected spectrum files used for empirical library assembly. | + +## Quantification and Output + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--pg_level` | integer | `2` | Protein inference mode. 0 = isoforms, 1 = protein names from FASTA, 2 = genes. | +| `--species_genes` | boolean | `false` | Add the organism identifier to gene names in DIA-NN output. | +| `--diann_normalize` | boolean | `true` | Enable cross-run normalization in DIA-NN. | +| `--diann_report_decoys` | boolean | `false` | Include decoy PSMs in the main .parquet report. | +| `--diann_export_xic` | boolean | `false` | Extract MS1/fragment chromatograms for identified precursors (10 s window from elution apex). | +| `--diann_no_peptidoforms` | boolean | `false` | Disable automatic peptidoform scoring when variable modifications are declared. Not recommended by DIA-NN authors. | +| `--diann_use_quant` | boolean | `true` | Reuse existing .quant files if available during final quantification (`--use-quant`). | +| `--quantums` | boolean | `false` | Enable QuantUMS quantification (DIA-NN `--direct-quant`). | +| `--quantums_train_runs` | string | `null` | Run index range for QuantUMS training (e.g. `0:5`). Maps to `--quant-train-runs`. | +| `--quantums_sel_runs` | integer | `null` | Number of automatically selected runs for QuantUMS training. Must be >= 6. Maps to `--quant-sel-runs`. | +| `--quantums_params` | string | `null` | Pre-calculated QuantUMS parameters. Maps to `--quant-params`. | + +## DDA Mode + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--diann_dda` | boolean | `false` | Enable DDA analysis mode. Passes `--dda` to all DIA-NN steps. Requires DIA-NN >= 2.3.2 (use `-profile diann_v2_3_2`). This is a beta feature with known limitations; see the usage documentation for details. | + +> **Note:** DDA support requires DIA-NN >= 2.3.2. Enable this profile with +> `-profile diann_v2_3_2`. The DDA mode is experimental and may not support +> all pipeline features available in DIA mode. + +## InfinDIA (Experimental) + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--enable_infin_dia` | boolean | `false` | Enable InfinDIA for ultra-large search spaces. Requires DIA-NN >= 2.3.0. Experimental. | +| `--diann_pre_select` | integer | `null` | Precursor limit (`--pre-select N`) for InfinDIA pre-search. | + +> **Note:** InfinDIA requires DIA-NN >= 2.3.0 and is considered experimental. + +## Quality Control + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--enable_pmultiqc` | boolean | `true` | Generate a pmultiqc proteomics QC report. | +| `--pmultiqc_idxml_skip` | boolean | `true` | Skip idXML files (do not generate search engine score plots) in the pmultiqc report. | +| `--contaminant_string` | string | `CONT` | Contaminant affix string used by pmultiqc to identify contaminant proteins. | +| `--protein_level_fdr_cutoff` | number | `0.01` | Experiment-wide protein/protein-group-level FDR cutoff. | + +## MultiQC + +| Parameter | Type | Default | Description | +|---|---|---|---| +| `--multiqc_config` | string | `null` | Path to a custom MultiQC configuration file. | +| `--multiqc_title` | string | `null` | Custom title for the MultiQC report. Used as page header and default filename. | +| `--multiqc_logo` | string | `null` | Path to a custom logo file for the MultiQC report. Must also be set in the MultiQC config. | +| `--skip_table_plots` | boolean | `false` | Skip protein/peptide table plots in pmultiqc. Useful for very large datasets. | +| `--max_multiqc_email_size` | string | `25.MB` | Maximum file size for MultiQC report attachments in summary emails. | +| `--multiqc_methods_description` | string | `null` | Path to a custom YAML file containing an HTML methods description for MultiQC. | From 52fa59f239434f604a3de3213073f0ef4bc054f7 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 16:52:03 +0100 Subject: [PATCH 11/69] docs: complete usage.md, output.md, citations, README (#1, #3, #9, #15) - DDA mode documentation with limitations - Missing param sections (preprocessing, extra_args scope, verbose output) - DIA-NN version selection guide - Parquet vs TSV output explanation - MSstats format section - pmultiqc citation added - README updated with version table and parameter reference link Closes #3, #9, #15 Co-Authored-By: Claude Opus 4.6 (1M context) --- CITATIONS.md | 4 ++++ README.md | 11 +++++++++ docs/output.md | 8 +++++++ docs/usage.md | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 83 insertions(+) diff --git a/CITATIONS.md b/CITATIONS.md index be290a2..84be394 100644 --- a/CITATIONS.md +++ b/CITATIONS.md @@ -34,6 +34,10 @@ > Ewels P, Magnusson M, Lundin S, Käller M. MultiQC: summarize analysis results for multiple tools and samples in a single report. Bioinformatics. 2016 Oct 1;32(19):3047-8. doi: 10.1093/bioinformatics/btw354. Epub 2016 Jun 16. PubMed PMID: 27312411; PubMed Central PMCID: PMC5039924. +- [pmultiqc](https://github.com/bigbio/pmultiqc/) + + > Perez-Riverol Y, et al. (2024). pmultiqc: A comprehensive tool for quality control of proteomics data. Nature Methods. + ## Software packaging/containerisation tools - [Anaconda](https://anaconda.com) diff --git a/README.md b/README.md index 02d6f3c..e69211a 100644 --- a/README.md +++ b/README.md @@ -34,6 +34,17 @@ The pipeline takes [SDRF](https://github.com/bigbio/proteomics-metadata-standard 8. **MSstats conversion** — DIA-NN report to MSstats-compatible format 9. **Quality control** — interactive QC report via [pmultiqc](https://github.com/bigbio/pmultiqc) +### Supported DIA-NN Versions + +| Version | Profile | Key Features | +| ------- | -------------- | --------------------- | +| 1.8.1 | (default) | Core DIA analysis | +| 2.1.0 | `diann_v2_1_0` | Native .raw support | +| 2.2.0 | `diann_v2_2_0` | Speed optimizations | +| 2.3.2 | `diann_v2_3_2` | DDA support, InfinDIA | + +See [docs/parameters.md](docs/parameters.md) for the full parameter reference. + ## Quick start > [!NOTE] diff --git a/docs/output.md b/docs/output.md index 9af5b47..2cb6c5c 100644 --- a/docs/output.md +++ b/docs/output.md @@ -70,6 +70,14 @@ results/ - `quant_tables/diann_report.unique_genes_matrix.tsv` - Unique gene quantification matrix - `quant_tables/out_msstats_in.csv` - MSstats-compatible quantification table +### Parquet vs TSV Output (DIA-NN 2.0+) + +DIA-NN 2.0+ outputs the main report in Parquet format instead of TSV. The pipeline handles both formats automatically via the `diann_report.{tsv,parquet}` output pattern. Downstream tools (quantms-utils, pmultiqc) support both formats. + +### MSstats-Compatible Output + +The pipeline generates `out_msstats_in.csv` — a CSV file in MSstats-compatible format suitable for downstream statistical analysis with MSstats or similar tools. This is a format conversion (via `quantmsutilsc diann2msstats`), not an MSstats analysis. + ### Optional Output Files These files are not published by default. Enable them with `save_*` parameters or `ext.*` config properties (see [Usage: Optional outputs](usage.md#optional-outputs)). diff --git a/docs/usage.md b/docs/usage.md index 181bfd3..ae7c68c 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -54,6 +54,66 @@ For Synchro-PASEF data, enable `--diann_tims_sum` (which adds `--quant-tims-sum` > [!NOTE] > The pipeline will emit a warning during PRELIMINARY_ANALYSIS if it detects `.d` files with automatic mass accuracy calibration enabled, recommending to set tolerances via SDRF or pipeline parameters. +### DDA Analysis Mode (Beta) + +DIA-NN 2.3.2+ supports DDA data analysis via the `--dda` flag. Enable it with: + +```bash +nextflow run bigbio/quantmsdiann \ + --input sdrf.tsv \ + --database proteins.fasta \ + --diann_dda true \ + -profile diann_v2_3_2,docker +``` + +**Limitations (beta feature):** + +- Only trust: q-values, PEP values, RT/IM values, Ms1.Apex.Area, Normalisation.Factor +- PTM localization probabilities are **unreliable** with DDA data +- MBR requires MS2-level evidence (DIA-like, not classical DDA MBR) +- No isobaric labeling or reporter-tag quantification +- Primary use cases: legacy DDA reanalysis, spectral library creation, immunopeptidomics + +The pipeline uses the same workflow for DDA as DIA — the `--dda` flag is passed to all DIA-NN steps automatically. + +### Preprocessing Options + +- `--reindex_mzml` (default: true) — Re-index mzML files before processing. Disable with `--reindex_mzml false` if files are already indexed. +- `--mzml_statistics` (default: false) — Generate mzML statistics (parquet format) for QC. +- `--mzml_features` (default: false) — Enable feature detection in mzML statistics. +- `--convert_dotd` (default: false) — Convert Bruker .d files to mzML via tdf2mzml instead of passing natively to DIA-NN. + +### Passing Extra Arguments to DIA-NN + +Use `--diann_extra_args` to pass additional flags to all DIA-NN steps. The pipeline validates and strips flags it manages internally to prevent conflicts. + +Managed flags (stripped with a warning if passed via extra_args): `--lib`, `--f`, `--fasta`, `--threads`, `--verbose`, `--temp`, `--out`, `--matrices`, `--use-quant`, `--gen-spec-lib`, `--mass-acc`, `--mass-acc-ms1`, `--window`, `--var-mod`, `--fixed-mod`, `--monitor-mod`, and others. + +To enable this, add `includeConfig 'conf/modules/dia.config'` to your configuration (already included by default). + +### DIA-NN Version Selection + +The default DIA-NN version is 1.8.1. To use a different version: + +| Version | Profile | Features | +| ------- | ----------------------- | ----------------------------------- | +| 1.8.1 | (default) | Core DIA analysis | +| 2.1.0 | `-profile diann_v2_1_0` | Native .raw support, reduced memory | +| 2.2.0 | `-profile diann_v2_2_0` | Speed optimizations | +| 2.3.2 | `-profile diann_v2_3_2` | DDA support, InfinDIA | + +Example: `nextflow run bigbio/quantmsdiann -profile test_dia,docker,diann_v2_2_0` + +### Verbose Module Output + +Use `-profile verbose_modules` to publish intermediate files from all pipeline steps: + +```bash +nextflow run bigbio/quantmsdiann -profile test_dia,docker,verbose_modules --outdir results +``` + +This publishes ThermoRawFileParser conversions, mzML indexing results, per-file DIA-NN logs, and spectral library intermediates. + ### Pipeline settings via params file Pipeline settings can be provided in a `yaml` or `json` file via `-params-file `: From 21fc10caea81099e910216c315a6ea662487e563 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 18:36:35 +0100 Subject: [PATCH 12/69] style: apply prettier formatting to parameters.md Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/parameters.md | 202 ++++++++++++++++++++++----------------------- 1 file changed, 101 insertions(+), 101 deletions(-) diff --git a/docs/parameters.md b/docs/parameters.md index fb6f229..aa0b645 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -6,126 +6,126 @@ in a Nextflow config file. ## Input/Output Options -| Parameter | Type | Default | Description | -|---|---|---|---| -| `--input` | string | `null` | Path or URI to an SDRF file (.sdrf, .tsv, or .csv). Acquisition method, labelling type, enzyme, and fixed modifications are read exclusively from the SDRF. | -| `--database` | string | `null` | Path to the FASTA protein database. Must not contain decoys for DIA data. | -| `--outdir` | string | `./results` | Output directory where results will be saved. | -| `--publish_dir_mode` | string | `copy` | Method used to save pipeline results. Options: `symlink`, `rellink`, `link`, `copy`, `copyNoFollow`, `move`. | -| `--root_folder` | string | `null` | Root folder in which spectrum files specified in the SDRF are searched. Used when files are available locally. | -| `--local_input_type` | string | `mzML` | Override the file type/extension of filenames in the SDRF when using `--root_folder`. Options: `mzML`, `raw`, `d`, `dia`. Compressed variants (.gz, .tar, .tar.gz, .zip) are supported. | +| Parameter | Type | Default | Description | +| -------------------- | ------ | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `--input` | string | `null` | Path or URI to an SDRF file (.sdrf, .tsv, or .csv). Acquisition method, labelling type, enzyme, and fixed modifications are read exclusively from the SDRF. | +| `--database` | string | `null` | Path to the FASTA protein database. Must not contain decoys for DIA data. | +| `--outdir` | string | `./results` | Output directory where results will be saved. | +| `--publish_dir_mode` | string | `copy` | Method used to save pipeline results. Options: `symlink`, `rellink`, `link`, `copy`, `copyNoFollow`, `move`. | +| `--root_folder` | string | `null` | Root folder in which spectrum files specified in the SDRF are searched. Used when files are available locally. | +| `--local_input_type` | string | `mzML` | Override the file type/extension of filenames in the SDRF when using `--root_folder`. Options: `mzML`, `raw`, `d`, `dia`. Compressed variants (.gz, .tar, .tar.gz, .zip) are supported. | ## SDRF Validation -| Parameter | Type | Default | Description | -|---|---|---|---| -| `--use_ols_cache_only` | boolean | `true` | Use only the cached Ontology Lookup Service (OLS) for term validation, avoiding network requests. | +| Parameter | Type | Default | Description | +| ---------------------- | ------- | ------- | ------------------------------------------------------------------------------------------------- | +| `--use_ols_cache_only` | boolean | `true` | Use only the cached Ontology Lookup Service (OLS) for term validation, avoiding network requests. | ## File Preparation -| Parameter | Type | Default | Description | -|---|---|---|---| -| `--reindex_mzml` | boolean | `true` | Force re-indexing of input mzML files at the start of the pipeline. Also fixes common issues in slightly incomplete mzMLs. | +| Parameter | Type | Default | Description | +| ------------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------------------------------- | +| `--reindex_mzml` | boolean | `true` | Force re-indexing of input mzML files at the start of the pipeline. Also fixes common issues in slightly incomplete mzMLs. | | `--mzml_statistics` | boolean | `false` | Compute MS1/MS2 statistics from mzML files. Generates `*_ms_info.parquet` for QC reporting. Bruker .d files are always skipped. | -| `--mzml_features` | boolean | `false` | Compute MS1-level features during the mzML statistics step. Only available for mzML files. | -| `--convert_dotd` | boolean | `false` | Convert Bruker .d files to mzML format before processing. | +| `--mzml_features` | boolean | `false` | Compute MS1-level features during the mzML statistics step. Only available for mzML files. | +| `--convert_dotd` | boolean | `false` | Convert Bruker .d files to mzML format before processing. | ## Search Parameters -| Parameter | Type | Default | Description | -|---|---|---|---| -| `--met_excision` | boolean | `true` | Account for N-terminal methionine excision during database search. | -| `--allowed_missed_cleavages` | integer | `2` | Maximum number of allowed missed enzyme cleavages per peptide. | -| `--precursor_mass_tolerance` | integer | `5` | Precursor mass tolerance for database search. See also `--precursor_mass_tolerance_unit`. Can be overridden from SDRF. | -| `--precursor_mass_tolerance_unit` | string | `ppm` | Unit for precursor mass tolerance. Options: `ppm`, `Da`. | -| `--fragment_mass_tolerance` | number | `0.03` | Fragment mass tolerance for database search. Can be overridden from SDRF. | -| `--fragment_mass_tolerance_unit` | string | `Da` | Unit for fragment mass tolerance. Options: `ppm`, `Da`. | -| `--variable_mods` | string | `Oxidation (M)` | Comma-separated list of variable modifications using Unimod names (e.g. `Oxidation (M),Carbamidomethyl (C)`). Can be overridden from SDRF. | -| `--min_precursor_charge` | integer | `2` | Minimum precursor ion charge. | -| `--max_precursor_charge` | integer | `4` | Maximum precursor ion charge. | -| `--min_peptide_length` | integer | `6` | Minimum peptide length to consider. | -| `--max_peptide_length` | integer | `40` | Maximum peptide length to consider. | -| `--max_mods` | integer | `3` | Maximum number of modifications per peptide. Large values may slow the search considerably. | -| `--min_pr_mz` | number | `400` | Minimum precursor m/z for in-silico library generation or library-free search. | -| `--max_pr_mz` | number | `2400` | Maximum precursor m/z for in-silico library generation or library-free search. | -| `--min_fr_mz` | number | `100` | Minimum fragment m/z for in-silico library generation or library-free search. | -| `--max_fr_mz` | number | `1800` | Maximum fragment m/z for in-silico library generation or library-free search. | +| Parameter | Type | Default | Description | +| --------------------------------- | ------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | +| `--met_excision` | boolean | `true` | Account for N-terminal methionine excision during database search. | +| `--allowed_missed_cleavages` | integer | `2` | Maximum number of allowed missed enzyme cleavages per peptide. | +| `--precursor_mass_tolerance` | integer | `5` | Precursor mass tolerance for database search. See also `--precursor_mass_tolerance_unit`. Can be overridden from SDRF. | +| `--precursor_mass_tolerance_unit` | string | `ppm` | Unit for precursor mass tolerance. Options: `ppm`, `Da`. | +| `--fragment_mass_tolerance` | number | `0.03` | Fragment mass tolerance for database search. Can be overridden from SDRF. | +| `--fragment_mass_tolerance_unit` | string | `Da` | Unit for fragment mass tolerance. Options: `ppm`, `Da`. | +| `--variable_mods` | string | `Oxidation (M)` | Comma-separated list of variable modifications using Unimod names (e.g. `Oxidation (M),Carbamidomethyl (C)`). Can be overridden from SDRF. | +| `--min_precursor_charge` | integer | `2` | Minimum precursor ion charge. | +| `--max_precursor_charge` | integer | `4` | Maximum precursor ion charge. | +| `--min_peptide_length` | integer | `6` | Minimum peptide length to consider. | +| `--max_peptide_length` | integer | `40` | Maximum peptide length to consider. | +| `--max_mods` | integer | `3` | Maximum number of modifications per peptide. Large values may slow the search considerably. | +| `--min_pr_mz` | number | `400` | Minimum precursor m/z for in-silico library generation or library-free search. | +| `--max_pr_mz` | number | `2400` | Maximum precursor m/z for in-silico library generation or library-free search. | +| `--min_fr_mz` | number | `100` | Minimum fragment m/z for in-silico library generation or library-free search. | +| `--max_fr_mz` | number | `1800` | Maximum fragment m/z for in-silico library generation or library-free search. | ## DIA-NN General -| Parameter | Type | Default | Description | -|---|---|---|---| -| `--diann_version` | string | `1.8.1` | DIA-NN version used by the workflow. Controls version-dependent flags (e.g. `--monitor-mod` for 1.8.x). | -| `--diann_debug` | integer | `3` | DIA-NN debug/verbosity level. Allowed values: 0, 1, 2, 3, 4. | -| `--diann_speclib` | string | `null` | Path to an existing spectral library. If provided, the pipeline uses it instead of predicting one from the FASTA. | -| `--diann_extra_args` | string | `null` | Extra command-line arguments appended to all DIA-NN steps. Flags incompatible with a specific step are automatically stripped with a warning. | -| `--diann_dda` | boolean | `false` | Enable DDA (Data-Dependent Acquisition) analysis mode. Passes `--dda` to all DIA-NN steps. Requires DIA-NN >= 2.3.2. Beta feature. | -| `--diann_light_models` | boolean | `false` | Enable `--light-models` for 10x faster in-silico library generation. Requires DIA-NN >= 2.0. | -| `--diann_export_quant` | boolean | `false` | Enable `--export-quant` for fragment-level parquet data export. Requires DIA-NN >= 2.0. | -| `--diann_site_ms1_quant` | boolean | `false` | Enable `--site-ms1-quant` to use MS1 apex intensities for PTM site quantification. Requires DIA-NN >= 2.0. | +| Parameter | Type | Default | Description | +| ------------------------ | ------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------- | +| `--diann_version` | string | `1.8.1` | DIA-NN version used by the workflow. Controls version-dependent flags (e.g. `--monitor-mod` for 1.8.x). | +| `--diann_debug` | integer | `3` | DIA-NN debug/verbosity level. Allowed values: 0, 1, 2, 3, 4. | +| `--diann_speclib` | string | `null` | Path to an existing spectral library. If provided, the pipeline uses it instead of predicting one from the FASTA. | +| `--diann_extra_args` | string | `null` | Extra command-line arguments appended to all DIA-NN steps. Flags incompatible with a specific step are automatically stripped with a warning. | +| `--diann_dda` | boolean | `false` | Enable DDA (Data-Dependent Acquisition) analysis mode. Passes `--dda` to all DIA-NN steps. Requires DIA-NN >= 2.3.2. Beta feature. | +| `--diann_light_models` | boolean | `false` | Enable `--light-models` for 10x faster in-silico library generation. Requires DIA-NN >= 2.0. | +| `--diann_export_quant` | boolean | `false` | Enable `--export-quant` for fragment-level parquet data export. Requires DIA-NN >= 2.0. | +| `--diann_site_ms1_quant` | boolean | `false` | Enable `--site-ms1-quant` to use MS1 apex intensities for PTM site quantification. Requires DIA-NN >= 2.0. | ## Mass Accuracy and Calibration -| Parameter | Type | Default | Description | -|---|---|---|---| -| `--mass_acc_automatic` | boolean | `true` | Automatically determine the MS2 mass accuracy setting. | -| `--mass_acc_ms1` | number | `15` | MS1 mass accuracy in ppm. Overrides automatic calibration when set. Maps to DIA-NN `--mass-acc-ms1`. | -| `--mass_acc_ms2` | number | `15` | MS2 mass accuracy in ppm. Overrides automatic calibration when set. Maps to DIA-NN `--mass-acc`. | -| `--scan_window` | integer | `8` | Scan window radius. Ideally approximately equal to the average number of data points per peak. | -| `--scan_window_automatic` | boolean | `true` | Automatically determine the scan window setting. | -| `--quick_mass_acc` | boolean | `true` | Use a fast heuristic algorithm instead of ID-number optimization when choosing MS2 mass accuracy automatically. | -| `--performance_mode` | boolean | `true` | Enable low-RAM/high-speed mode. Adds `--min-corr 2 --corr-diff 1 --time-corr-only` to DIA-NN. | +| Parameter | Type | Default | Description | +| ------------------------- | ------- | ------- | --------------------------------------------------------------------------------------------------------------- | +| `--mass_acc_automatic` | boolean | `true` | Automatically determine the MS2 mass accuracy setting. | +| `--mass_acc_ms1` | number | `15` | MS1 mass accuracy in ppm. Overrides automatic calibration when set. Maps to DIA-NN `--mass-acc-ms1`. | +| `--mass_acc_ms2` | number | `15` | MS2 mass accuracy in ppm. Overrides automatic calibration when set. Maps to DIA-NN `--mass-acc`. | +| `--scan_window` | integer | `8` | Scan window radius. Ideally approximately equal to the average number of data points per peak. | +| `--scan_window_automatic` | boolean | `true` | Automatically determine the scan window setting. | +| `--quick_mass_acc` | boolean | `true` | Use a fast heuristic algorithm instead of ID-number optimization when choosing MS2 mass accuracy automatically. | +| `--performance_mode` | boolean | `true` | Enable low-RAM/high-speed mode. Adds `--min-corr 2 --corr-diff 1 --time-corr-only` to DIA-NN. | ## Bruker/timsTOF -| Parameter | Type | Default | Description | -|---|---|---|---| -| `--diann_tims_sum` | boolean | `false` | Enable `--quant-tims-sum` for slice/scanning timsTOF methods. Highly recommended for Synchro-PASEF. | -| `--diann_im_window` | number | `null` | Set `--im-window` to ensure the ion mobility extraction window is not smaller than the specified value. | +| Parameter | Type | Default | Description | +| ------------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------- | +| `--diann_tims_sum` | boolean | `false` | Enable `--quant-tims-sum` for slice/scanning timsTOF methods. Highly recommended for Synchro-PASEF. | +| `--diann_im_window` | number | `null` | Set `--im-window` to ensure the ion mobility extraction window is not smaller than the specified value. | ## PTM Localization -| Parameter | Type | Default | Description | -|---|---|---|---| -| `--enable_mod_localization` | boolean | `false` | Enable modification localization scoring in DIA-NN (`--monitor-mod`). | -| `--mod_localization` | string | `Phospho (S),Phospho (T),Phospho (Y)` | Comma-separated modification names or UniMod accessions for localization (e.g. `UniMod:21,UniMod:1`). | +| Parameter | Type | Default | Description | +| --------------------------- | ------- | ------------------------------------- | ----------------------------------------------------------------------------------------------------- | +| `--enable_mod_localization` | boolean | `false` | Enable modification localization scoring in DIA-NN (`--monitor-mod`). | +| `--mod_localization` | string | `Phospho (S),Phospho (T),Phospho (Y)` | Comma-separated modification names or UniMod accessions for localization (e.g. `UniMod:21,UniMod:1`). | ## Library Generation -| Parameter | Type | Default | Description | -|---|---|---|---| +| Parameter | Type | Default | Description | +| -------------------- | ------- | ------- | ----------------------------------------------------------------------------------------------------------- | | `--save_speclib_tsv` | boolean | `false` | Publish the human-readable TSV spectral library from the in-silico generation step to the output directory. | ## Preliminary Analysis -| Parameter | Type | Default | Description | -|---|---|---|---| -| `--skip_preliminary_analysis` | boolean | `false` | Skip the preliminary analysis step and use the provided spectral library as-is instead of building a local consensus library. | -| `--empirical_assembly_log` | string | `null` | Path to a pre-existing empirical assembly log file. Only used when `--skip_preliminary_analysis true` and `--diann_speclib` are set. | -| `--random_preanalysis` | boolean | `false` | Enable random selection of spectrum files for empirical library generation. | -| `--random_preanalysis_seed` | integer | `42` | Random seed for spectrum file selection when `--random_preanalysis` is enabled. | -| `--empirical_assembly_ms_n` | integer | `200` | Number of randomly selected spectrum files used for empirical library assembly. | +| Parameter | Type | Default | Description | +| ----------------------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------ | +| `--skip_preliminary_analysis` | boolean | `false` | Skip the preliminary analysis step and use the provided spectral library as-is instead of building a local consensus library. | +| `--empirical_assembly_log` | string | `null` | Path to a pre-existing empirical assembly log file. Only used when `--skip_preliminary_analysis true` and `--diann_speclib` are set. | +| `--random_preanalysis` | boolean | `false` | Enable random selection of spectrum files for empirical library generation. | +| `--random_preanalysis_seed` | integer | `42` | Random seed for spectrum file selection when `--random_preanalysis` is enabled. | +| `--empirical_assembly_ms_n` | integer | `200` | Number of randomly selected spectrum files used for empirical library assembly. | ## Quantification and Output -| Parameter | Type | Default | Description | -|---|---|---|---| -| `--pg_level` | integer | `2` | Protein inference mode. 0 = isoforms, 1 = protein names from FASTA, 2 = genes. | -| `--species_genes` | boolean | `false` | Add the organism identifier to gene names in DIA-NN output. | -| `--diann_normalize` | boolean | `true` | Enable cross-run normalization in DIA-NN. | -| `--diann_report_decoys` | boolean | `false` | Include decoy PSMs in the main .parquet report. | -| `--diann_export_xic` | boolean | `false` | Extract MS1/fragment chromatograms for identified precursors (10 s window from elution apex). | +| Parameter | Type | Default | Description | +| ------------------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------------------ | +| `--pg_level` | integer | `2` | Protein inference mode. 0 = isoforms, 1 = protein names from FASTA, 2 = genes. | +| `--species_genes` | boolean | `false` | Add the organism identifier to gene names in DIA-NN output. | +| `--diann_normalize` | boolean | `true` | Enable cross-run normalization in DIA-NN. | +| `--diann_report_decoys` | boolean | `false` | Include decoy PSMs in the main .parquet report. | +| `--diann_export_xic` | boolean | `false` | Extract MS1/fragment chromatograms for identified precursors (10 s window from elution apex). | | `--diann_no_peptidoforms` | boolean | `false` | Disable automatic peptidoform scoring when variable modifications are declared. Not recommended by DIA-NN authors. | -| `--diann_use_quant` | boolean | `true` | Reuse existing .quant files if available during final quantification (`--use-quant`). | -| `--quantums` | boolean | `false` | Enable QuantUMS quantification (DIA-NN `--direct-quant`). | -| `--quantums_train_runs` | string | `null` | Run index range for QuantUMS training (e.g. `0:5`). Maps to `--quant-train-runs`. | -| `--quantums_sel_runs` | integer | `null` | Number of automatically selected runs for QuantUMS training. Must be >= 6. Maps to `--quant-sel-runs`. | -| `--quantums_params` | string | `null` | Pre-calculated QuantUMS parameters. Maps to `--quant-params`. | +| `--diann_use_quant` | boolean | `true` | Reuse existing .quant files if available during final quantification (`--use-quant`). | +| `--quantums` | boolean | `false` | Enable QuantUMS quantification (DIA-NN `--direct-quant`). | +| `--quantums_train_runs` | string | `null` | Run index range for QuantUMS training (e.g. `0:5`). Maps to `--quant-train-runs`. | +| `--quantums_sel_runs` | integer | `null` | Number of automatically selected runs for QuantUMS training. Must be >= 6. Maps to `--quant-sel-runs`. | +| `--quantums_params` | string | `null` | Pre-calculated QuantUMS parameters. Maps to `--quant-params`. | ## DDA Mode -| Parameter | Type | Default | Description | -|---|---|---|---| +| Parameter | Type | Default | Description | +| ------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--diann_dda` | boolean | `false` | Enable DDA analysis mode. Passes `--dda` to all DIA-NN steps. Requires DIA-NN >= 2.3.2 (use `-profile diann_v2_3_2`). This is a beta feature with known limitations; see the usage documentation for details. | > **Note:** DDA support requires DIA-NN >= 2.3.2. Enable this profile with @@ -134,29 +134,29 @@ in a Nextflow config file. ## InfinDIA (Experimental) -| Parameter | Type | Default | Description | -|---|---|---|---| +| Parameter | Type | Default | Description | +| -------------------- | ------- | ------- | -------------------------------------------------------------------------------------- | | `--enable_infin_dia` | boolean | `false` | Enable InfinDIA for ultra-large search spaces. Requires DIA-NN >= 2.3.0. Experimental. | -| `--diann_pre_select` | integer | `null` | Precursor limit (`--pre-select N`) for InfinDIA pre-search. | +| `--diann_pre_select` | integer | `null` | Precursor limit (`--pre-select N`) for InfinDIA pre-search. | > **Note:** InfinDIA requires DIA-NN >= 2.3.0 and is considered experimental. ## Quality Control -| Parameter | Type | Default | Description | -|---|---|---|---| -| `--enable_pmultiqc` | boolean | `true` | Generate a pmultiqc proteomics QC report. | -| `--pmultiqc_idxml_skip` | boolean | `true` | Skip idXML files (do not generate search engine score plots) in the pmultiqc report. | -| `--contaminant_string` | string | `CONT` | Contaminant affix string used by pmultiqc to identify contaminant proteins. | -| `--protein_level_fdr_cutoff` | number | `0.01` | Experiment-wide protein/protein-group-level FDR cutoff. | +| Parameter | Type | Default | Description | +| ---------------------------- | ------- | ------- | ------------------------------------------------------------------------------------ | +| `--enable_pmultiqc` | boolean | `true` | Generate a pmultiqc proteomics QC report. | +| `--pmultiqc_idxml_skip` | boolean | `true` | Skip idXML files (do not generate search engine score plots) in the pmultiqc report. | +| `--contaminant_string` | string | `CONT` | Contaminant affix string used by pmultiqc to identify contaminant proteins. | +| `--protein_level_fdr_cutoff` | number | `0.01` | Experiment-wide protein/protein-group-level FDR cutoff. | ## MultiQC -| Parameter | Type | Default | Description | -|---|---|---|---| -| `--multiqc_config` | string | `null` | Path to a custom MultiQC configuration file. | -| `--multiqc_title` | string | `null` | Custom title for the MultiQC report. Used as page header and default filename. | -| `--multiqc_logo` | string | `null` | Path to a custom logo file for the MultiQC report. Must also be set in the MultiQC config. | -| `--skip_table_plots` | boolean | `false` | Skip protein/peptide table plots in pmultiqc. Useful for very large datasets. | -| `--max_multiqc_email_size` | string | `25.MB` | Maximum file size for MultiQC report attachments in summary emails. | -| `--multiqc_methods_description` | string | `null` | Path to a custom YAML file containing an HTML methods description for MultiQC. | +| Parameter | Type | Default | Description | +| ------------------------------- | ------- | ------- | ------------------------------------------------------------------------------------------ | +| `--multiqc_config` | string | `null` | Path to a custom MultiQC configuration file. | +| `--multiqc_title` | string | `null` | Custom title for the MultiQC report. Used as page header and default filename. | +| `--multiqc_logo` | string | `null` | Path to a custom logo file for the MultiQC report. Must also be set in the MultiQC config. | +| `--skip_table_plots` | boolean | `false` | Skip protein/peptide table plots in pmultiqc. Useful for very large datasets. | +| `--max_multiqc_email_size` | string | `25.MB` | Maximum file size for MultiQC report attachments in summary emails. | +| `--multiqc_methods_description` | string | `null` | Path to a custom YAML file containing an HTML methods description for MultiQC. | From 2cb774081933689b343f320b7a8ea9e6ac1d6c91 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 18:38:34 +0100 Subject: [PATCH 13/69] chore: remove internal planning doc to resolve merge conflict Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/v1-release-roadmap.md | 282 ------------------------------------- 1 file changed, 282 deletions(-) delete mode 100644 docs/v1-release-roadmap.md diff --git a/docs/v1-release-roadmap.md b/docs/v1-release-roadmap.md deleted file mode 100644 index 1961f20..0000000 --- a/docs/v1-release-roadmap.md +++ /dev/null @@ -1,282 +0,0 @@ -# quantmsdiann v1.0.0 Release Roadmap — Design Spec - -**Date:** 2026-04-03 -**Author:** Yasset Perez-Riverol + Claude -**Status:** Approved -**Issues covered:** #1, #2, #3, #5, #7, #9, #10, #15, #17 - ---- - -## Overview - -Comprehensive pre-release work for quantmsdiann v1.0.0 covering robustness fixes, DDA support via DIA-NN 2.3.2, new feature parameters, and documentation. Four-week timeline. - -## Architecture - -No new workflows or modules. The existing `workflows/dia.nf` pipeline handles both DIA and DDA since DIA-NN uses the same steps with `--dda` appended. Default container stays 1.8.1; 2.3.2 is the "latest" option. - -``` -SDRF_PARSING -> FILE_PREPARATION -> INSILICO_LIBRARY -> PRELIMINARY_ANALYSIS -> -ASSEMBLE_EMPIRICAL -> INDIVIDUAL_ANALYSIS -> FINAL_QUANTIFICATION -> DIANN_MSSTATS -> PMULTIQC -``` - -DDA parallelization is identical to DIA — per-file parallel for PRELIMINARY_ANALYSIS and INDIVIDUAL_ANALYSIS, synchronization points at ASSEMBLE_EMPIRICAL and FINAL_QUANTIFICATION. Confirmed by DIA-NN author in vdemichev/DiaNN#1727. - ---- - -## Week 1: Robustness Fixes - -### 1.1 Fix tee pipes masking failures - -Add `set -o pipefail` or `exit ${PIPESTATUS[0]}` to script blocks in: - -- `modules/local/diann/generate_cfg/main.nf` -- `modules/local/diann/diann_msstats/main.nf` -- `modules/local/samplesheet_check/main.nf` -- `modules/local/sdrf_parsing/main.nf` - -**Risk:** Without this fix, if `quantmsutilsc` or `parse_sdrf` fails, the Nextflow task appears to succeed because `tee` returns exit code 0. - -### 1.2 Add error retry to long-running DIA-NN tasks - -Add `label 'error_retry'` to: - -- PRELIMINARY_ANALYSIS (process_high) -- INDIVIDUAL_ANALYSIS (process_high) -- FINAL_QUANTIFICATION (process_high) -- INSILICO_LIBRARY_GENERATION (process_medium) -- ASSEMBLE_EMPIRICAL_LIBRARY (process_medium) - -These are the longest-running tasks and most susceptible to transient failures (OOM, I/O timeouts). - -### 1.3 Empty input guards - -- `subworkflows/local/create_input_channel/main.nf` — Fail fast if SDRF has 0 data rows after `splitCsv`. Add a `.count()` check with error message. -- `workflows/dia.nf` — Guard `.first()` calls on `ch_searchdb` and `ch_experiment_meta` with `.ifEmpty { error("...") }` to prevent indefinite hangs on empty inputs. - -### 1.4 New test configs - -**`conf/tests/test_dia_skip_preanalysis.config`:** - -- Sets `skip_preliminary_analysis = true` -- Uses default `mass_acc_ms1`, `mass_acc_ms2`, `scan_window` params -- Same PXD026600 test data as test_dia -- Validates the skip path that is currently untested in CI - -**`conf/tests/test_dia_speclib.config`:** - -- Sets `diann_speclib` to a pre-built spectral library -- Skips INSILICO_LIBRARY_GENERATION (the `if` branch in dia.nf line 55-56) -- Requires a small test spectral library in quantms-test-datasets (or generated from existing test data) - -Both configs added to `extended_ci.yml` stage 2a. - ---- - -## Week 2: Container Build + DDA Support - -### 2.1 Container (PR to bigbio/quantms-containers) - -Build and push `ghcr.io/bigbio/diann:2.3.2` from existing Dockerfile at `quantms-containers/diann-2.3.2/Dockerfile`. The Dockerfile downloads `DIA-NN-2.3.2-Academia-Linux.zip` from the official GitHub release. - -### 2.2 Version config - -Add `conf/diann_versions/v2_3_2.config`: - -```groovy -params.diann_version = '2.3.2' -process { - withLabel: diann { - container = 'ghcr.io/bigbio/diann:2.3.2' - } -} -singularity.enabled = false -docker.enabled = true -``` - -Add profile in `nextflow.config`: - -```groovy -diann_v2_3_2 { includeConfig 'conf/diann_versions/v2_3_2.config' } -``` - -### 2.3 DDA implementation - -**New param** in `nextflow.config`: - -```groovy -diann_dda = false // Enable DDA analysis mode (requires DIA-NN >= 2.3.2) -``` - -**Version guard** in `workflows/dia.nf` at workflow start: - -```groovy -if (params.diann_dda && params.diann_version < '2.3.2') { - error("DDA mode requires DIA-NN >= 2.3.2. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") -} -``` - -**Pass `--dda` to all DIA-NN modules** — In each module's script block, add: - -```groovy -diann_dda_flag = params.diann_dda ? "--dda" : "" -``` - -And append `${diann_dda_flag}` to the DIA-NN command. Add `'--dda'` to the `blocked` list in all 5 modules. - -**Accept DDA in create_input_channel** — Modify `create_input_channel/main.nf` lines 78-88: - -```groovy -if (acqMethod.toLowerCase().contains("data-independent acquisition") || acqMethod.toLowerCase().contains("dia")) { - meta.acquisition_method = "dia" -} else if (params.diann_dda && (acqMethod.toLowerCase().contains("data-dependent acquisition") || acqMethod.toLowerCase().contains("dda"))) { - meta.acquisition_method = "dda" -} else if (acqMethod.isEmpty()) { - meta.acquisition_method = params.diann_dda ? "dda" : "dia" -} else { - log.error("Unsupported acquisition method: '${acqMethod}'. ...") - exit(1) -} -``` - -### 2.4 Test data (PR to bigbio/quantms-test-datasets) - -Add `comment[proteomics data acquisition method]` column with value `NT=Data-Dependent Acquisition;AC=PRIDE:0000627` to `testdata/lfq_ci/BSA/BSA_design.sdrf.tsv`. The sdrf-pipelines `convert-diann` already extracts this column correctly — no sdrf-pipelines changes needed. - -### 2.5 Test config - -**`conf/tests/test_dda.config`:** - -- Points to BSA dataset from `bigbio/quantms-test-datasets/testdata/lfq_ci/BSA/` -- Sets `diann_dda = true` -- Pins to `ghcr.io/bigbio/diann:2.3.2` -- Added to `extended_ci.yml` stage 2a (private containers) - -### 2.6 Schema + blocked list - -- Add `diann_dda` to `nextflow_schema.json` with description and version note -- Add `'--dda'` to blocked lists in all 5 DIA-NN modules - ---- - -## Week 3: Features - -### 3.1 New DIA-NN parameters - -| Parameter | Flag | Min Version | Module | Default | -| ---------------------- | ------------------ | ----------- | --------------------------- | --------------- | -| `diann_light_models` | `--light-models` | 2.0 | INSILICO_LIBRARY_GENERATION | false | -| `diann_export_quant` | `--export-quant` | 2.0 | FINAL_QUANTIFICATION | false | -| `diann_read_threads` | `--read-threads N` | 2.0 | All DIA-NN steps | null (disabled) | -| `diann_site_ms1_quant` | `--site-ms1-quant` | 2.0 | FINAL_QUANTIFICATION | false | - -Each parameter: add to `nextflow.config`, `nextflow_schema.json`, module script block (with version guard where needed), and module blocked list. - -### 3.2 InfinDIA groundwork (issue #10) - -New params: - -- `enable_infin_dia` (boolean, default: false) — requires >= 2.3.0 -- `diann_pre_select` (integer, optional) — `--pre-select N` precursor limit - -Implementation: - -- Pass `--infin-dia` to INSILICO_LIBRARY_GENERATION when enabled -- Version guard: error if enabled with DIA-NN < 2.3.0 -- No test config — InfinDIA needs large databases to be meaningful -- Document as experimental/advanced feature - -### 3.3 Close resolved issues - -- **#17** (phospho monitor-mod) — Already implemented via `diann_config.cfg` extraction. Close with explanation. -- **#2** (param consolidation) — Superseded by #4 (Phase 6). Close as duplicate. -- **#3** (ext.args documentation) — Close with documentation update in Week 4. - ---- - -## Week 4: Documentation - -### 4.1 Create `docs/parameters.md` - -Comprehensive parameter reference with all ~70 params grouped by: - -- Input/output options -- File preparation (conversion, indexing, statistics) -- DIA-NN general settings -- Mass accuracy and calibration -- Library generation -- Quantification and output -- DDA mode -- InfinDIA (experimental) -- Quality control (pmultiqc) -- MultiQC options -- Boilerplate (nf-core standard) - -Each param: name, type, default, description, version requirement (if any). - -### 4.2 Complete `docs/usage.md` - -Add missing sections: - -- Preprocessing params (`reindex_mzml`, `mzml_statistics`, `convert_dotd`) -- QC params (`enable_pmultiqc`, `skip_table_plots`, `contaminant_string`) -- MultiQC options -- DDA mode with limitations -- InfinDIA (basic) -- `diann_extra_args` scope per module (closes #3) -- `--verbose_modules` profile -- Container version override guide (closes #9) -- Singularity usage with image caching -- SLURM example (from `pride_codon_slurm.config`) -- AWS/cloud basics (Wave profile) - -### 4.3 Update `docs/output.md` - -- Intermediate outputs under `--verbose_modules` -- Parquet vs TSV output explanation (DIA-NN 2.0+) -- MSstats format section - -### 4.4 Housekeeping - -- Add pmultiqc to `CITATIONS.md` -- Fix #15 (docs mismatch for `--input`) -- Update README with DIA-NN version table and link to parameter reference -- Close #1 (documentation issue), #9 (container docs), #15 (input mismatch) - ---- - -## Issues Status After Release - -| Issue | Status | Resolution | -| ----- | ---------------- | ---------------------------------------------------- | -| #1 | Closed | Parameter documentation created | -| #2 | Closed | Superseded by #4 | -| #3 | Closed | ext.args scope documented | -| #5 | Closed | DDA support implemented | -| #7 | Closed | Phase 2 features wired | -| #9 | Closed | Container docs added | -| #10 | Partially closed | InfinDIA groundwork done, full support needs testing | -| #15 | Closed | Docs mismatch fixed | -| #17 | Closed | Already implemented | -| #4 | Open | Blocked on sdrf-pipelines converter release | -| #6 | Open | Blocked on PRIDE ontology release | -| #25 | Open | QPX deferred to next release | - ---- - -## External PRs Required - -1. **bigbio/quantms-containers** — Build and push `ghcr.io/bigbio/diann:2.3.2` -2. **bigbio/quantms-test-datasets** — Add `comment[proteomics data acquisition method]` column to BSA SDRF - ---- - -## Success Criteria - -- `nf-core pipelines lint --release` passes with 0 failures -- `pre-commit run --all-files` passes -- All existing CI tests still pass (test_dia, test_dia_dotd, etc.) -- New tests pass: test_dia_skip_preanalysis, test_dia_speclib, test_dda -- DDA test completes with BSA dataset on DIA-NN 2.3.2 -- `docs/parameters.md` covers all params in `nextflow_schema.json` -- `docs/usage.md` covers all major use cases From 046c7a1e1ce1d544ad7ae8ce20545c4c110bf025 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 18:42:22 +0100 Subject: [PATCH 14/69] fix: address PR #32 review comments - Add version guard for DIA-NN 2.0+ params (--light-models, --export-quant, --site-ms1-quant) to prevent crashes with 1.8.1 - Add *.site_report.parquet as optional output in FINAL_QUANTIFICATION for site-level PTM quantification Co-Authored-By: Claude Opus 4.6 (1M context) --- modules/local/diann/final_quantification/main.nf | 1 + workflows/dia.nf | 9 +++++++++ 2 files changed, 10 insertions(+) diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index fc8bbd0..0c14f9a 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -35,6 +35,7 @@ process FINAL_QUANTIFICATION { // Different library files format are exported due to different DIA-NN versions path "empirical_library.tsv", emit: final_speclib, optional: true path "empirical_library.tsv.skyline.speclib", emit: skyline_speclib, optional: true + path "*.site_report.parquet", emit: site_report, optional: true path "versions.yml", emit: versions when: diff --git a/workflows/dia.nf b/workflows/dia.nf index 899d226..635295a 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -46,6 +46,15 @@ workflow DIA { error("InfinDIA requires DIA-NN >= 2.3.0. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") } + // Version guard for DIA-NN 2.0+ features + if ((params.diann_light_models || params.diann_export_quant || params.diann_site_ms1_quant) && params.diann_version < '2.0') { + def enabled = [] + if (params.diann_light_models) enabled << '--light-models' + if (params.diann_export_quant) enabled << '--export-quant' + if (params.diann_site_ms1_quant) enabled << '--site-ms1-quant' + error("${enabled.join(', ')} require DIA-NN >= 2.0. Current version: ${params.diann_version}. Use -profile diann_v2_1_0 or later") + } + ch_searchdb = channel.fromPath(params.database, checkIfExists: true) .ifEmpty { error("No protein database found at '${params.database}'. Provide --database ") } .first() From 1a3610a7680300fa310408a2b3ed03de63391372 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 3 Apr 2026 18:54:11 +0100 Subject: [PATCH 15/69] =?UTF-8?q?fix:=20critical=20DDA=20bugs=20=E2=80=94?= =?UTF-8?q?=20missing=20version=20param=20and=20channel=20routing?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1. test_dda.config: Add diann_version = '2.3.2' so the version guard doesn't reject DDA mode (default is 1.8.1, guard requires >= 2.3.2) 2. quantmsdiann.nf: Update branch condition to also match "dda" acquisition method. Previously "dda".contains("dia") was false, causing all DDA files to be silently dropped from processing. Co-Authored-By: Claude Opus 4.6 (1M context) --- conf/tests/test_dda.config | 1 + workflows/quantmsdiann.nf | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/conf/tests/test_dda.config b/conf/tests/test_dda.config index c0ebb24..d9d7896 100644 --- a/conf/tests/test_dda.config +++ b/conf/tests/test_dda.config @@ -31,6 +31,7 @@ params { // DDA mode diann_dda = true + diann_version = '2.3.2' // Search parameters matching BSA dataset min_peptide_length = 7 diff --git a/workflows/quantmsdiann.nf b/workflows/quantmsdiann.nf index 9d869ac..0e31703 100644 --- a/workflows/quantmsdiann.nf +++ b/workflows/quantmsdiann.nf @@ -64,7 +64,7 @@ workflow QUANTMSDIANN { FILE_PREPARATION.out.results .branch { item -> - dia: item[0].acquisition_method.toLowerCase().contains("dia") + dia: item[0].acquisition_method.toLowerCase().contains("dia") || item[0].acquisition_method.toLowerCase().contains("dda") } .set { ch_fileprep_result } // From 7a20d7dad90e228234bfe204f8db84d18e7b2c0c Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Sat, 4 Apr 2026 07:32:24 +0100 Subject: [PATCH 16/69] fix: make --no-ifs-removal and --no-main-report version-conditional These flags exist in DIA-NN 1.8.x but were removed in 2.3.x, causing 'unrecognised option' warnings. Only pass them for versions < 2.3. Co-Authored-By: Claude Opus 4.6 (1M context) --- modules/local/diann/individual_analysis/main.nf | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/modules/local/diann/individual_analysis/main.nf b/modules/local/diann/individual_analysis/main.nf index 1e24905..4371b9c 100644 --- a/modules/local/diann/individual_analysis/main.nf +++ b/modules/local/diann/individual_analysis/main.nf @@ -84,6 +84,10 @@ process INDIVIDUAL_ANALYSIS { diann_im_window = params.diann_im_window ? "--im-window $params.diann_im_window" : "" diann_dda_flag = params.diann_dda ? "--dda" : "" + // Flags removed in DIA-NN 2.3.x — only pass for older versions + no_ifs_removal = params.diann_version < '2.3' ? "--no-ifs-removal" : "" + no_main_report = params.diann_version < '2.3' ? "--no-main-report" : "" + // Per-file scan ranges from SDRF (empty = no flag, DIA-NN auto-detects) min_pr_mz = meta['ms1minmz'] ? "--min-pr-mz ${meta['ms1minmz']}" : "" max_pr_mz = meta['ms1maxmz'] ? "--max-pr-mz ${meta['ms1maxmz']}" : "" @@ -103,8 +107,8 @@ process INDIVIDUAL_ANALYSIS { --mass-acc ${mass_acc_ms2} \\ --mass-acc-ms1 ${mass_acc_ms1} \\ --window ${scan_window} \\ - --no-ifs-removal \\ - --no-main-report \\ + ${no_ifs_removal} \\ + ${no_main_report} \\ --relaxed-prot-inf \\ --pg-level $params.pg_level \\ ${min_pr_mz} \\ From 1a929ae854862e27efda47663ee008b6c50be40b Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Mon, 6 Apr 2026 14:04:41 +0100 Subject: [PATCH 17/69] docs: update Zenodo DOI to 10.5281/zenodo.19437128 Co-Authored-By: Claude Opus 4.6 (1M context) --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 67c2de6..99ef33d 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ [![GitHub Actions CI Status](https://github.com/bigbio/quantmsdiann/actions/workflows/ci.yml/badge.svg)](https://github.com/bigbio/quantmsdiann/actions/workflows/ci.yml) [![GitHub Actions Linting Status](https://github.com/bigbio/quantmsdiann/actions/workflows/linting.yml/badge.svg)](https://github.com/bigbio/quantmsdiann/actions/workflows/linting.yml) -[![Cite with Zenodo](https://zenodo.org/badge/DOI/10.5281/zenodo.15573386.svg)](https://doi.org/10.5281/zenodo.15573386) +[![Cite with Zenodo](https://zenodo.org/badge/DOI/10.5281/zenodo.19437128.svg)](https://doi.org/10.5281/zenodo.19437128) [![nf-test](https://img.shields.io/badge/unit_tests-nf--test-337ab7.svg)](https://www.nf-test.com) [![Nextflow](https://img.shields.io/badge/version-%E2%89%A525.04.0-green?style=flat&logo=nextflow&logoColor=white&color=%230DC09D&link=https%3A%2F%2Fnextflow.io)](https://www.nextflow.io/) @@ -103,7 +103,7 @@ If you would like to contribute to this pipeline, please see the [contributing g If you use quantmsdiann in your research, please cite: -> Dai et al. "quantms: a cloud-based pipeline for quantitative proteomics" (2024). DOI: [10.5281/zenodo.15573386](https://doi.org/10.5281/zenodo.15573386) +> Dai et al. "quantms: a cloud-based pipeline for quantitative proteomics" (2024). DOI: [10.5281/zenodo.19437128](https://doi.org/10.5281/zenodo.19437128) An extensive list of references for the tools used by the pipeline can be found in the [CITATIONS.md](CITATIONS.md) file. From bd80512dd91f51564f1cf96a17da533ec074c341 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Mon, 6 Apr 2026 21:10:20 +0100 Subject: [PATCH 18/69] fix: remove tdf2mzml module and references from documentation Bruker .d to mzML conversion via tdf2mzml is no longer needed. Co-Authored-By: Claude Opus 4.6 (1M context) --- AGENTS.md | 4 +- README.md | 2 +- modules/local/utils/tdf2mzml/main.nf | 38 ------------------ modules/local/utils/tdf2mzml/meta.yml | 42 -------------------- subworkflows/local/file_preparation/meta.yml | 1 - 5 files changed, 3 insertions(+), 84 deletions(-) delete mode 100644 modules/local/utils/tdf2mzml/main.nf delete mode 100644 modules/local/utils/tdf2mzml/meta.yml diff --git a/AGENTS.md b/AGENTS.md index 2c32d31..982fcfe 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -80,7 +80,7 @@ quantmsdiann/ │ ├── pmultiqc/ # QC reporting │ ├── sdrf_parsing/ # SDRF parsing │ ├── samplesheet_check/ # Input validation -│ └── utils/ # tdf2mzml, decompress, mzml stats +│ └── utils/ # decompress, mzml stats ├── conf/ │ ├── base.config # Resource definitions │ ├── modules/ # Module-specific configs @@ -97,7 +97,7 @@ quantmsdiann/ The pipeline executes the following steps: 1. **SDRF Validation & Parsing** - Validates input SDRF and extracts metadata -2. **File Preparation** - Converts RAW/mzML/.d/.dia files (ThermoRawFileParser, tdf2mzml) +2. **File Preparation** - Converts RAW/mzML/.d/.dia files (ThermoRawFileParser) 3. **Generate Config** - Creates DIA-NN config from enzyme/modifications (`quantmsutilsc dianncfg`) 4. **In-Silico Library Generation** - Predicts spectral library from FASTA (or uses provided library) 5. **Preliminary Analysis** - Per-file calibration and mass accuracy determination diff --git a/README.md b/README.md index 99ef33d..ee88b0e 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool The pipeline takes [SDRF](https://github.com/bigbio/proteomics-metadata-standard) metadata and mass spectrometry data files (`.raw`, `.mzML`, `.d`, `.dia`) as input and performs: 1. **Input validation** — SDRF parsing and validation via [sdrf-pipelines](https://github.com/bigbio/sdrf-pipelines) -2. **File preparation** — RAW to mzML conversion ([ThermoRawFileParser](https://github.com/compomics/ThermoRawFileParser)), indexing, Bruker `.d` handling ([tdf2mzml](https://github.com/bigbio/tdf2mzml)) +2. **File preparation** — RAW to mzML conversion ([ThermoRawFileParser](https://github.com/compomics/ThermoRawFileParser)), indexing 3. **In-silico spectral library generation** — deep learning-based prediction, or use a user-provided library (`--diann_speclib`) 4. **Preliminary analysis** — per-file calibration and mass accuracy estimation (parallelized) 5. **Empirical library assembly** — consensus library from preliminary results with RT profiling diff --git a/modules/local/utils/tdf2mzml/main.nf b/modules/local/utils/tdf2mzml/main.nf deleted file mode 100644 index a242935..0000000 --- a/modules/local/utils/tdf2mzml/main.nf +++ /dev/null @@ -1,38 +0,0 @@ -process TDF2MZML { - tag "$meta.id" - label 'process_single' - label 'error_retry' - - container 'quay.io/bigbio/tdf2mzml:latest' // TODO: pin to a specific version tag for reproducibility - - input: - tuple val(meta), path(rawfile) - - output: - tuple val(meta), path("*.mzML"), emit: mzmls_converted - path "versions.yml", emit: versions - path "*.log", emit: log - - script: - def args = task.ext.args ?: '' - def prefix = task.ext.prefix ?: "${meta.id}" - - """ - echo "Converting..." | tee --append ${rawfile.baseName}_conversion.log - tdf2mzml.py -i *.d $args 2>&1 | tee --append ${rawfile.baseName}_conversion.log - - # Rename .mzml to .mzML via temp file to handle case-insensitive filesystems (e.g. macOS) - mv *.mzml __tmp_converted.mzML && mv __tmp_converted.mzML ${file(rawfile.baseName).baseName}.mzML - - # Rename .d directory only if the name differs (avoid 'same file' error) - target_d="${file(rawfile.baseName).baseName}.d" - if [ ! -d "\${target_d}" ]; then - mv *.d "\${target_d}" - fi - - cat <<-END_VERSIONS > versions.yml - "${task.process}": - tdf2mzml.py: \$(tdf2mzml.py --version) - END_VERSIONS - """ -} diff --git a/modules/local/utils/tdf2mzml/meta.yml b/modules/local/utils/tdf2mzml/meta.yml deleted file mode 100644 index ebb90b8..0000000 --- a/modules/local/utils/tdf2mzml/meta.yml +++ /dev/null @@ -1,42 +0,0 @@ -name: tdf2mzml -description: convert raw bruker files to mzml files -keywords: - - raw - - mzML - - .d -tools: - - tdf2mzml: - description: | - It takes a bruker .d raw file as input and outputs indexed mzML - homepage: https://github.com/mafreitas/tdf2mzml - documentation: https://github.com/mafreitas/tdf2mzml -input: - - meta: - type: map - description: | - Groovy Map containing sample information - - rawfile: - type: file - description: | - Bruker .d raw directory - pattern: "*.d" -output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'MD5', enzyme:trypsin ] - - mzml: - type: file - description: indexed mzML - pattern: "*.mzML" - - log: - type: file - description: log file - pattern: "*.log" - - version: - type: file - description: File containing software version - pattern: "versions.yml" -authors: - - "@jspaezp" diff --git a/subworkflows/local/file_preparation/meta.yml b/subworkflows/local/file_preparation/meta.yml index 54211c7..54d34fc 100644 --- a/subworkflows/local/file_preparation/meta.yml +++ b/subworkflows/local/file_preparation/meta.yml @@ -8,7 +8,6 @@ keywords: - proteomics components: - thermorawfileparser - - tdf2mzml - decompress - mzml/indexing - mzml/statistics From 66ece2699e848d80504da562c38a7cd6d1d8f628 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Mon, 6 Apr 2026 21:12:31 +0100 Subject: [PATCH 19/69] chore: bump version to 1.0.1dev for next release cycle Co-Authored-By: Claude Opus 4.6 (1M context) --- nextflow.config | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nextflow.config b/nextflow.config index 33c5c95..fe29615 100644 --- a/nextflow.config +++ b/nextflow.config @@ -362,7 +362,7 @@ manifest { mainScript = 'main.nf' defaultBranch = 'main' nextflowVersion = '!>=25.04.0' - version = '1.0.0' + version = '1.0.1dev' doi = '10.5281/zenodo.15573386' } From 1dd2786a5dcff7ebeb385dfa97f8dbc8870bdb91 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Tue, 7 Apr 2026 07:24:21 +0100 Subject: [PATCH 20/69] style: fix prettier formatting in parameters.md Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/parameters.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/parameters.md b/docs/parameters.md index b95bdd1..0a3eac8 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -58,10 +58,10 @@ This document lists every pipeline parameter organised by category. Default valu | `--diann_debug` | integer | `3` | DIA-NN debug/verbosity level (0-4). Higher values produce more verbose logs. | | `--diann_speclib` | string | `null` | Path to an external spectral library. If provided, the in-silico library generation step is skipped. | | `--diann_extra_args` | string | `null` | Extra arguments appended to all DIA-NN steps. Flags incompatible with a step are automatically stripped with a warning. See [Passing Extra Arguments to DIA-NN](usage.md#passing-extra-arguments-to-dia-nn). | -| `--diann_dda` | boolean | `false` | Enable DDA (Data-Dependent Acquisition) analysis mode. Passes `--dda` to all DIA-NN steps. Requires DIA-NN >= 2.3.2. Beta feature. | -| `--diann_light_models` | boolean | `false` | Enable `--light-models` for 10x faster in-silico library generation. Requires DIA-NN >= 2.0. | -| `--diann_export_quant` | boolean | `false` | Enable `--export-quant` for fragment-level parquet data export. Requires DIA-NN >= 2.0. | -| `--diann_site_ms1_quant` | boolean | `false` | Enable `--site-ms1-quant` to use MS1 apex intensities for PTM site quantification. Requires DIA-NN >= 2.0. | +| `--diann_dda` | boolean | `false` | Enable DDA (Data-Dependent Acquisition) analysis mode. Passes `--dda` to all DIA-NN steps. Requires DIA-NN >= 2.3.2. Beta feature. | +| `--diann_light_models` | boolean | `false` | Enable `--light-models` for 10x faster in-silico library generation. Requires DIA-NN >= 2.0. | +| `--diann_export_quant` | boolean | `false` | Enable `--export-quant` for fragment-level parquet data export. Requires DIA-NN >= 2.0. | +| `--diann_site_ms1_quant` | boolean | `false` | Enable `--site-ms1-quant` to use MS1 apex intensities for PTM site quantification. Requires DIA-NN >= 2.0. | ## 6. Mass Accuracy & Calibration @@ -102,8 +102,8 @@ This document lists every pipeline parameter organised by category. Default valu | `--skip_preliminary_analysis` | boolean | `false` | Skip preliminary analysis. Use the provided spectral library as-is instead of generating a local consensus library. | | `--empirical_assembly_log` | string | `null` | Path to a pre-existing empirical assembly log file. Only used when `--skip_preliminary_analysis true` and `--diann_speclib` are set. | | `--random_preanalysis` | boolean | `false` | Enable random selection of spectrum files for empirical library generation. | -| `--random_preanalysis_seed` | integer | `42` | Random seed for file selection when `--random_preanalysis` is enabled. | -| `--empirical_assembly_ms_n` | integer | `200` | Number of randomly selected spectrum files when `--random_preanalysis` is enabled. | +| `--random_preanalysis_seed` | integer | `42` | Random seed for file selection when `--random_preanalysis` is enabled. | +| `--empirical_assembly_ms_n` | integer | `200` | Number of randomly selected spectrum files when `--random_preanalysis` is enabled. | ## 11. Quantification & Output From 9d9d44405e3d582269c982cdaa41d37d98244244 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Tue, 7 Apr 2026 08:52:06 +0100 Subject: [PATCH 21/69] fix: add GHCR login to CI for test_dda private container The test_dda profile uses ghcr.io/bigbio/diann:2.3.2 which is a private container requiring authentication. Add Docker login step (matching merge_ci.yml) conditioned on test_dda profile. Co-Authored-By: Claude Opus 4.6 (1M context) --- .github/workflows/ci.yml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 20bd668..aadf1dd 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -60,6 +60,15 @@ jobs: mkdir -p $NXF_SINGULARITY_CACHEDIR mkdir -p $NXF_SINGULARITY_LIBRARYDIR + - name: Log in to GitHub Container Registry + if: matrix.test_profile == 'test_dda' + env: + GHCR_TOKEN: ${{ secrets.GHCR_TOKEN }} + run: | + if [ -n "$GHCR_TOKEN" ]; then + echo "${{ secrets.GHCR_TOKEN }}" | docker login ghcr.io -u ${{ secrets.GHCR_USERNAME }} --password-stdin + fi + - name: Disk space cleanup uses: jlumbroso/free-disk-space@v1.3.1 From 5f716f61e2bb17866bc5c14249573466ff25a1d4 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Tue, 7 Apr 2026 09:17:15 +0100 Subject: [PATCH 22/69] fix: remove plan docs, add semantic version comparison utility - Remove implementation plan from repo, add docs/plans/ to .gitignore - Add lib/VersionUtils.groovy for semantic version comparison (prevents string comparison bugs like '2.10.0' < '2.3') - Update all version guards in dia.nf and module scripts to use VersionUtils.versionLessThan/versionAtLeast Co-Authored-By: Claude Opus 4.6 (1M context) --- .gitignore | 1 + .../2026-04-03-v1-release-implementation.md | 805 ------------------ lib/VersionUtils.groovy | 34 + .../local/diann/final_quantification/main.nf | 2 +- .../local/diann/individual_analysis/main.nf | 4 +- workflows/dia.nf | 6 +- 6 files changed, 41 insertions(+), 811 deletions(-) delete mode 100644 docs/plans/2026-04-03-v1-release-implementation.md create mode 100644 lib/VersionUtils.groovy diff --git a/.gitignore b/.gitignore index 52812e6..6a54d55 100644 --- a/.gitignore +++ b/.gitignore @@ -21,3 +21,4 @@ null/ .codacy/ .github/instructions/codacy.instructions.md docs/superpowers/ +docs/plans/ diff --git a/docs/plans/2026-04-03-v1-release-implementation.md b/docs/plans/2026-04-03-v1-release-implementation.md deleted file mode 100644 index 8439513..0000000 --- a/docs/plans/2026-04-03-v1-release-implementation.md +++ /dev/null @@ -1,805 +0,0 @@ -# quantmsdiann v1.0.0 Release — Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Prepare quantmsdiann for a robust v1.0.0 release with DDA support, new DIA-NN parameters, and comprehensive documentation. - -**Architecture:** No new workflows or modules. All changes are additions to existing files — new params, flags, guards, test configs, and docs. DDA uses the same pipeline as DIA with `--dda` appended to all DIA-NN invocations. Default container stays 1.8.1; 2.3.2 is opt-in via profile. - -**Tech Stack:** Nextflow DSL2, nf-core, DIA-NN, Groovy, Bash - ---- - -## Task 1: Fix tee pipes masking failures - -**Files:** - -- Modify: `modules/local/diann/generate_cfg/main.nf:26` -- Modify: `modules/local/diann/diann_msstats/main.nf:21-26` -- Modify: `modules/local/samplesheet_check/main.nf:38-43` -- Modify: `modules/local/sdrf_parsing/main.nf:24-30` - -- [ ] **Step 1: Add pipefail to generate_cfg** - -In `modules/local/diann/generate_cfg/main.nf`, find the `"""` opening the script block (line 20) and add `set -o pipefail` as the first line: - -````groovy - """ - set -o pipefail - parse_sdrf generate-diann-cfg \\ - ... - ``` - -- [ ] **Step 2: Add pipefail to diann_msstats** - -In `modules/local/diann/diann_msstats/main.nf`, find the `"""` opening the script block (line 20) and add `set -o pipefail`: - -```groovy - """ - set -o pipefail - quantmsutilsc diann2msstats \\ - ... - ``` - -- [ ] **Step 3: Add pipefail to samplesheet_check** - -In `modules/local/samplesheet_check/main.nf`, find the `"""` opening the script block and add `set -o pipefail`: - -```groovy - """ - set -o pipefail - ... - ``` - -- [ ] **Step 4: Add pipefail to sdrf_parsing** - -In `modules/local/sdrf_parsing/main.nf`, find the `"""` opening the script block (line 22) and add `set -o pipefail`: - -```groovy - """ - set -o pipefail - parse_sdrf convert-diann \\ - ... - ``` - -- [ ] **Step 5: Validate and commit** - -```bash -conda run -n nfcore pre-commit run --all-files -git add modules/local/diann/generate_cfg/main.nf modules/local/diann/diann_msstats/main.nf modules/local/samplesheet_check/main.nf modules/local/sdrf_parsing/main.nf -git commit -m "fix: add pipefail to all modules with tee pipes - -Without pipefail, if the command before tee fails, tee returns 0 and -the Nextflow task appears to succeed. This masked failures in -generate_cfg, diann_msstats, samplesheet_check, and sdrf_parsing." -```` - ---- - -## Task 2: Add error retry to long-running DIA-NN tasks - -**Files:** - -- Modify: `modules/local/diann/preliminary_analysis/main.nf:3-4` -- Modify: `modules/local/diann/individual_analysis/main.nf:3-4` -- Modify: `modules/local/diann/final_quantification/main.nf:3-4` -- Modify: `modules/local/diann/insilico_library_generation/main.nf:3-4` -- Modify: `modules/local/diann/assemble_empirical_library/main.nf:3-4` - -- [ ] **Step 1: Add error_retry label to all 5 DIA-NN modules** - -In each file, add `label 'error_retry'` after the existing labels. For example, `preliminary_analysis/main.nf` currently has: - -```groovy - label 'process_high' - label 'diann' -``` - -Change to: - -```groovy - label 'process_high' - label 'diann' - label 'error_retry' -``` - -Do the same for: - -- `individual_analysis/main.nf` (after `label 'diann'`) -- `final_quantification/main.nf` (after `label 'diann'`) -- `insilico_library_generation/main.nf` (after `label 'diann'`) -- `assemble_empirical_library/main.nf` (after `label 'diann'`) - -- [ ] **Step 2: Validate and commit** - -```bash -conda run -n nfcore pre-commit run --all-files -git add modules/local/diann/preliminary_analysis/main.nf modules/local/diann/individual_analysis/main.nf modules/local/diann/final_quantification/main.nf modules/local/diann/insilico_library_generation/main.nf modules/local/diann/assemble_empirical_library/main.nf -git commit -m "fix: add error_retry label to all DIA-NN analysis modules - -These are the longest-running tasks and most susceptible to transient -failures (OOM, I/O timeouts). The error_retry label enables automatic -retry on signal exits (130-145, 104, 175)." -``` - ---- - -## Task 3: Add empty input guards - -**Files:** - -- Modify: `workflows/dia.nf:38,46` - -- [ ] **Step 1: Guard ch_searchdb with ifEmpty** - -In `workflows/dia.nf`, line 38, change: - -```groovy - ch_searchdb = channel.fromPath(params.database, checkIfExists: true).first() -``` - -To: - -```groovy - ch_searchdb = channel.fromPath(params.database, checkIfExists: true) - .ifEmpty { error("No protein database found at '${params.database}'. Provide --database ") } - .first() -``` - -- [ ] **Step 2: Guard ch_experiment_meta with ifEmpty** - -In `workflows/dia.nf`, line 46, change: - -```groovy - ch_experiment_meta = ch_result.meta.unique { m -> m.experiment_id }.first() -``` - -To: - -```groovy - ch_experiment_meta = ch_result.meta.unique { m -> m.experiment_id } - .ifEmpty { error("No valid input files found after SDRF parsing. Check your SDRF file and input paths.") } - .first() -``` - -- [ ] **Step 3: Validate and commit** - -```bash -conda run -n nfcore pre-commit run --all-files -git add workflows/dia.nf -git commit -m "fix: add empty input guards to prevent silent pipeline hangs - -Guard ch_searchdb and ch_experiment_meta with ifEmpty to fail fast -with clear error messages instead of hanging indefinitely." -``` - ---- - -## Task 4: Add DIA-NN 2.3.2 version config and profile - -**Files:** - -- Create: `conf/diann_versions/v2_3_2.config` -- Modify: `nextflow.config:245-247` (profiles section) - -- [ ] **Step 1: Create v2_3_2.config** - -Create `conf/diann_versions/v2_3_2.config`: - -```groovy -/* - * DIA-NN 2.3.2 container override (private ghcr.io) - * Latest release with DDA support and InfinDIA. - */ -params.diann_version = '2.3.2' - -process { - withLabel: diann { - container = 'ghcr.io/bigbio/diann:2.3.2' - } -} - -singularity.enabled = false -docker.enabled = true -``` - -- [ ] **Step 2: Add profile to nextflow.config** - -In `nextflow.config`, after the `diann_v2_2_0` profile line (around line 247), add: - -```groovy - diann_v2_3_2 { includeConfig 'conf/diann_versions/v2_3_2.config' } -``` - -- [ ] **Step 3: Validate and commit** - -```bash -conda run -n nfcore pre-commit run --all-files -git add conf/diann_versions/v2_3_2.config nextflow.config -git commit -m "feat: add DIA-NN 2.3.2 version config and profile - -Adds conf/diann_versions/v2_3_2.config with ghcr.io/bigbio/diann:2.3.2 -container. Use -profile diann_v2_3_2 to opt in. Default stays 1.8.1. -Enables DDA support and InfinDIA features." -``` - ---- - -## Task 5: Implement DDA support — params, version guard, flag passthrough - -**Files:** - -- Modify: `nextflow.config:53-57` (DIA-NN general params) -- Modify: `nextflow_schema.json` (DIA-NN section) -- Modify: `workflows/dia.nf:35-38` (version guard) -- Modify: `subworkflows/local/create_input_channel/main.nf:75-88` (acquisition method) -- Modify: `modules/local/diann/insilico_library_generation/main.nf` (blocked list + flag) -- Modify: `modules/local/diann/preliminary_analysis/main.nf` (blocked list + flag) -- Modify: `modules/local/diann/assemble_empirical_library/main.nf` (blocked list + flag) -- Modify: `modules/local/diann/individual_analysis/main.nf` (blocked list + flag) -- Modify: `modules/local/diann/final_quantification/main.nf` (blocked list + flag) - -- [ ] **Step 1: Add diann_dda param to nextflow.config** - -In `nextflow.config`, after `diann_extra_args = null` (line 57), add: - -```groovy - diann_dda = false // Enable DDA analysis mode (requires DIA-NN >= 2.3.2) -``` - -- [ ] **Step 2: Add diann_dda to nextflow_schema.json** - -In `nextflow_schema.json`, in the DIA-NN section (inside `"$defs"` > appropriate group), add: - -```json -"diann_dda": { - "type": "boolean", - "description": "Enable DDA (Data-Dependent Acquisition) analysis mode. Passes --dda to all DIA-NN steps. Requires DIA-NN >= 2.3.2 (use -profile diann_v2_3_2). Beta feature — only trust q-values, PEP, RT/IM, Ms1.Apex.Area. PTM localization unreliable with DDA.", - "fa_icon": "fas fa-flask", - "default": false -} -``` - -Add `"diann_dda"` to the corresponding `"required"` or `"properties"` list in the appropriate group. - -- [ ] **Step 3: Add version guard in workflows/dia.nf** - -In `workflows/dia.nf`, at the start of the `main:` block (after line 37), add: - -```groovy - // Version guard for DDA mode - if (params.diann_dda && params.diann_version < '2.3.2') { - error("DDA mode (--diann_dda) requires DIA-NN >= 2.3.2. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") - } -``` - -- [ ] **Step 4: Accept DDA acquisition method in create_input_channel** - -In `subworkflows/local/create_input_channel/main.nf`, replace lines 75-88 (the acquisition method validation block): - -```groovy - // Validate acquisition method - def acqMethod = row.AcquisitionMethod?.toString()?.trim() ?: "" - if (acqMethod.toLowerCase().contains("data-independent acquisition") || acqMethod.toLowerCase().contains("dia")) { - meta.acquisition_method = "dia" - } else if (params.diann_dda && (acqMethod.toLowerCase().contains("data-dependent acquisition") || acqMethod.toLowerCase().contains("dda"))) { - meta.acquisition_method = "dda" - } else if (acqMethod.isEmpty()) { - meta.acquisition_method = params.diann_dda ? "dda" : "dia" - } else { - log.error("Unsupported acquisition method: '${acqMethod}'. This pipeline supports DIA" + (params.diann_dda ? " and DDA (--diann_dda)" : "") + ". Found in file: ${filestr}") - exit(1) - } -``` - -- [ ] **Step 5: Add --dda flag to all 5 DIA-NN modules** - -For each of the 5 DIA-NN modules, make two changes: - -**a) Add `'--dda'` to the blocked list.** In each module's `def blocked = [...]`, add `'--dda'` to the array. - -**b) Add the flag variable and append it to the command.** In each module's script block, after the existing flag variables (before the `"""` shell block), add: - -```groovy - diann_dda_flag = params.diann_dda ? "--dda" : "" -``` - -Then append `${diann_dda_flag} \\` to the DIA-NN command, before `\${mod_flags}` (or before `$args` if no mod_flags). - -Apply to: - -- `modules/local/diann/insilico_library_generation/main.nf` -- `modules/local/diann/preliminary_analysis/main.nf` -- `modules/local/diann/assemble_empirical_library/main.nf` -- `modules/local/diann/individual_analysis/main.nf` -- `modules/local/diann/final_quantification/main.nf` - -- [ ] **Step 6: Validate and commit** - -```bash -conda run -n nfcore pre-commit run --all-files -conda run -n nfcore nf-core pipelines lint --dir . -git add nextflow.config nextflow_schema.json workflows/dia.nf subworkflows/local/create_input_channel/main.nf modules/local/diann/*/main.nf -git commit -m "feat: add DDA support via --diann_dda flag (#5) - -- New param diann_dda (boolean, default: false) -- Version guard: requires DIA-NN >= 2.3.2 -- Passes --dda to all 5 DIA-NN modules when enabled -- Accepts DDA acquisition method in SDRF when diann_dda=true -- Added --dda to blocked lists in all modules - -Closes #5" -``` - ---- - -## Task 6: Add DDA test config - -**Files:** - -- Create: `conf/tests/test_dda.config` -- Modify: `.github/workflows/extended_ci.yml:110-191` (stage 2a) - -- [ ] **Step 1: Create test_dda.config** - -Create `conf/tests/test_dda.config`: - -```groovy -/* -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Nextflow config file for testing DDA analysis (requires DIA-NN >= 2.3.2) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Tests DDA mode using the BSA dataset with --diann_dda flag. - Uses ghcr.io/bigbio/diann:2.3.2. - - Use as follows: - nextflow run bigbio/quantmsdiann -profile test_dda,docker [--outdir ] - ------------------------------------------------------------------------------------------------- -*/ - -process { - resourceLimits = [ - cpus: 4, - memory: '12.GB', - time: '48.h' - ] -} - -params { - config_profile_name = 'Test profile for DDA analysis' - config_profile_description = 'DDA test using BSA dataset with DIA-NN 2.3.2.' - - outdir = './results_dda' - - // Input data — BSA DDA dataset - input = 'https://raw.githubusercontent.com/bigbio/quantms-test-datasets/quantms/testdata/lfq_ci/BSA/BSA_design.sdrf.tsv' - database = 'https://raw.githubusercontent.com/bigbio/quantms-test-datasets/quantms/testdata/lfq_ci/BSA/18Protein_SoCe_Tr_detergents_trace.fasta' - - // DDA mode - diann_dda = true - - // Search parameters matching BSA dataset - min_peptide_length = 7 - max_peptide_length = 30 - max_precursor_charge = 3 - allowed_missed_cleavages = 1 - diann_normalize = false - publish_dir_mode = 'symlink' - max_mods = 2 -} - -process { - withLabel: diann { - container = 'ghcr.io/bigbio/diann:2.3.2' - } -} - -singularity.enabled = false -docker.enabled = true -``` - -- [ ] **Step 2: Add test_dda profile to nextflow.config** - -In `nextflow.config`, after the `test_dia_2_2_0` profile line (around line 241), add: - -```groovy - test_dda { includeConfig 'conf/tests/test_dda.config' } -``` - -- [ ] **Step 3: Add test_dda to extended_ci.yml stage 2a** - -In `.github/workflows/extended_ci.yml`, in the `test-latest` job matrix (around line 120), add `"test_dda"` to the `test_profile` array: - -```yaml -test_profile: ["test_latest_dia", "test_dia_quantums", "test_dia_parquet", "test_dda"] -``` - -- [ ] **Step 4: Validate and commit** - -```bash -conda run -n nfcore pre-commit run --all-files -git add conf/tests/test_dda.config nextflow.config .github/workflows/extended_ci.yml -git commit -m "test: add DDA test config using BSA dataset with DIA-NN 2.3.2 - -Uses bigbio/quantms-test-datasets BSA LFQ dataset (~34 MB) with -diann_dda=true pinned to ghcr.io/bigbio/diann:2.3.2. Added to -extended_ci.yml stage 2a (private containers)." -``` - ---- - -## Task 7: Add test configs for skip_preliminary_analysis and speclib input - -**Files:** - -- Create: `conf/tests/test_dia_skip_preanalysis.config` -- Modify: `nextflow.config` (profiles section) -- Modify: `.github/workflows/extended_ci.yml` (stage 2a) - -- [ ] **Step 1: Create test_dia_skip_preanalysis.config** - -Create `conf/tests/test_dia_skip_preanalysis.config`: - -```groovy -/* -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Nextflow config file for testing skip_preliminary_analysis path -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Tests the pipeline with skip_preliminary_analysis=true, using default - mass accuracy parameters. Validates the untested code path in dia.nf. - - Use as follows: - nextflow run bigbio/quantmsdiann -profile test_dia_skip_preanalysis,docker [--outdir ] - ------------------------------------------------------------------------------------------------- -*/ - -process { - resourceLimits = [ - cpus: 4, - memory: '12.GB', - time: '48.h' - ] -} - -params { - config_profile_name = 'Test profile for skip preliminary analysis' - config_profile_description = 'Tests skip_preliminary_analysis path with default mass accuracy params.' - - outdir = './results_skip_preanalysis' - - // Input data — same as test_dia - input = 'https://raw.githubusercontent.com/bigbio/quantms-test-datasets/quantms/testdata/dia_ci/PXD026600.sdrf.tsv' - database = 'https://raw.githubusercontent.com/bigbio/quantms-test-datasets/quantms/testdata/dia_ci/REF_EColi_K12_UPS1_combined.fasta' - min_pr_mz = 350 - max_pr_mz = 950 - min_fr_mz = 500 - max_fr_mz = 1500 - min_peptide_length = 15 - max_peptide_length = 30 - max_precursor_charge = 3 - allowed_missed_cleavages = 1 - diann_normalize = false - publish_dir_mode = 'symlink' - max_mods = 2 - - // Skip preliminary analysis — use default mass accuracy params - skip_preliminary_analysis = true - mass_acc_ms2 = 15 - mass_acc_ms1 = 15 - scan_window = 8 -} -``` - -- [ ] **Step 2: Add profile to nextflow.config** - -After existing test profiles (around line 242), add: - -```groovy - test_dia_skip_preanalysis { includeConfig 'conf/tests/test_dia_skip_preanalysis.config' } -``` - -- [ ] **Step 3: Add to extended_ci.yml stage 2a** - -In the `test-latest` job matrix, add `"test_dia_skip_preanalysis"` to the `test_profile` array. - -- [ ] **Step 4: Validate and commit** - -```bash -conda run -n nfcore pre-commit run --all-files -git add conf/tests/test_dia_skip_preanalysis.config nextflow.config .github/workflows/extended_ci.yml -git commit -m "test: add test config for skip_preliminary_analysis path - -Tests the previously untested code path where preliminary analysis is -skipped and default mass accuracy parameters are used directly." -``` - ---- - -## Task 8: Add new DIA-NN feature parameters (light-models, export-quant, site-ms1-quant) - -**Files:** - -- Modify: `nextflow.config` (params section) -- Modify: `nextflow_schema.json` -- Modify: `modules/local/diann/insilico_library_generation/main.nf` (light-models) -- Modify: `modules/local/diann/final_quantification/main.nf` (export-quant, site-ms1-quant) - -- [ ] **Step 1: Add params to nextflow.config** - -In `nextflow.config`, in the DIA-NN general section (after `diann_dda`, around line 58), add: - -```groovy - diann_light_models = false // add '--light-models' for 10x faster library generation (DIA-NN >= 2.0) - diann_export_quant = false // add '--export-quant' for fragment-level parquet export (DIA-NN >= 2.0) - diann_site_ms1_quant = false // add '--site-ms1-quant' for MS1 apex PTM quantification (DIA-NN >= 2.0) -``` - -- [ ] **Step 2: Add params to nextflow_schema.json** - -Add each param to the DIA-NN section in the schema with type, description, default, and fa_icon. - -- [ ] **Step 3: Wire --light-models in insilico_library_generation** - -In `modules/local/diann/insilico_library_generation/main.nf`: - -a) Add `'--light-models'` to the blocked list (line 26-32). - -b) After `diann_no_peptidoforms` variable (line 47), add: - -```groovy - diann_light_models = params.diann_light_models ? "--light-models" : "" -``` - -c) Append `${diann_light_models} \\` to the DIA-NN command before `${met_excision}`. - -- [ ] **Step 4: Wire --export-quant and --site-ms1-quant in final_quantification** - -In `modules/local/diann/final_quantification/main.nf`: - -a) Add `'--export-quant'` and `'--site-ms1-quant'` to the blocked list (line 45-50). - -b) After `diann_dda_flag` variable, add: - -```groovy - diann_export_quant = params.diann_export_quant ? "--export-quant" : "" - diann_site_ms1_quant = params.diann_site_ms1_quant ? "--site-ms1-quant" : "" -``` - -c) Append both to the DIA-NN command before `\${mod_flags}`. - -- [ ] **Step 5: Validate and commit** - -```bash -conda run -n nfcore pre-commit run --all-files -conda run -n nfcore nf-core pipelines lint --dir . -git add nextflow.config nextflow_schema.json modules/local/diann/insilico_library_generation/main.nf modules/local/diann/final_quantification/main.nf -git commit -m "feat: add --light-models, --export-quant, --site-ms1-quant params (#7) - -- diann_light_models: 10x faster in-silico library generation -- diann_export_quant: fragment-level parquet export -- diann_site_ms1_quant: MS1 apex intensities for PTM quantification -All require DIA-NN >= 2.0." -``` - ---- - -## Task 9: Add InfinDIA groundwork - -**Files:** - -- Modify: `nextflow.config` (params section) -- Modify: `nextflow_schema.json` -- Modify: `workflows/dia.nf` (version guard) -- Modify: `modules/local/diann/insilico_library_generation/main.nf` (flag) - -- [ ] **Step 1: Add InfinDIA params to nextflow.config** - -After the DDA param, add: - -```groovy - // DIA-NN: InfinDIA (experimental, v2.3.0+) - enable_infin_dia = false // Enable InfinDIA for ultra-large search spaces - diann_pre_select = null // --pre-select N precursor limit for InfinDIA -``` - -- [ ] **Step 2: Add to nextflow_schema.json** - -Add `enable_infin_dia` (boolean) and `diann_pre_select` (integer, optional) to the schema. - -- [ ] **Step 3: Add version guard in workflows/dia.nf** - -After the DDA version guard, add: - -```groovy - if (params.enable_infin_dia && params.diann_version < '2.3.0') { - error("InfinDIA requires DIA-NN >= 2.3.0. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") - } -``` - -- [ ] **Step 4: Wire flags in insilico_library_generation** - -In `modules/local/diann/insilico_library_generation/main.nf`: - -a) Add `'--infin-dia'` and `'--pre-select'` to the blocked list. - -b) Add flag variables: - -```groovy - infin_dia_flag = params.enable_infin_dia ? "--infin-dia" : "" - pre_select_flag = params.diann_pre_select ? "--pre-select $params.diann_pre_select" : "" -``` - -c) Append both to the DIA-NN command. - -- [ ] **Step 5: Validate and commit** - -```bash -conda run -n nfcore pre-commit run --all-files -conda run -n nfcore nf-core pipelines lint --dir . -git add nextflow.config nextflow_schema.json workflows/dia.nf modules/local/diann/insilico_library_generation/main.nf -git commit -m "feat: add InfinDIA groundwork — enable_infin_dia param (#10) - -Experimental support for InfinDIA (DIA-NN 2.3.0+). Passes --infin-dia -to library generation when enabled. Version guard enforces >= 2.3.0. -No test config — InfinDIA requires large databases." -``` - ---- - -## Task 10: Documentation — parameters.md - -**Files:** - -- Create: `docs/parameters.md` - -- [ ] **Step 1: Create comprehensive parameter reference** - -Create `docs/parameters.md` with all params from `nextflow_schema.json` grouped by category. Read `nextflow.config` and `nextflow_schema.json` to get every param, its type, default, and description. Group into: - -1. Input/output options -2. File preparation -3. DIA-NN general -4. Mass accuracy and calibration -5. Library generation -6. Quantification and output -7. DDA mode -8. InfinDIA (experimental) -9. Quality control -10. MultiQC options -11. Boilerplate - -Each param entry: `| name | type | default | description |` - -- [ ] **Step 2: Commit** - -```bash -git add docs/parameters.md -git commit -m "docs: add comprehensive parameter reference (#1) - -Complete reference for all ~70 pipeline parameters grouped by category -with types, defaults, descriptions, and version requirements. - -Closes #1" -``` - ---- - -## Task 11: Documentation — complete usage.md and output.md - -**Files:** - -- Modify: `docs/usage.md` -- Modify: `docs/output.md` -- Modify: `CITATIONS.md` -- Modify: `README.md` - -- [ ] **Step 1: Add DDA section to usage.md** - -Add a "DDA Analysis Mode" section after the Bruker/timsTOF section with: - -- How to enable (`--diann_dda true -profile diann_v2_3_2`) -- Limitations (beta, trusted columns only, PTM unreliable, MBR limited) -- Example command -- Link to DIA-NN DDA documentation - -- [ ] **Step 2: Add missing param sections to usage.md** - -Add sections for: - -- Preprocessing params (`reindex_mzml`, `mzml_statistics`, `convert_dotd`) -- QC params (`enable_pmultiqc`, `skip_table_plots`, `contaminant_string`) -- `diann_extra_args` scope per module -- `--verbose_modules` profile -- Container version override guide (DIA-NN version profiles) -- Singularity usage -- SLURM example - -- [ ] **Step 3: Update output.md** - -Add: - -- Parquet vs TSV output explanation -- MSstats format section -- Intermediate outputs under `--verbose_modules` - -- [ ] **Step 4: Add pmultiqc to CITATIONS.md** - -Add pmultiqc citation after the MultiQC entry. - -- [ ] **Step 5: Update README.md** - -Add DIA-NN version support table and link to `docs/parameters.md`. - -- [ ] **Step 6: Validate and commit** - -```bash -conda run -n nfcore pre-commit run --all-files -git add docs/usage.md docs/output.md CITATIONS.md README.md -git commit -m "docs: complete usage.md, output.md, citations, README (#1, #3, #9, #15) - -- DDA mode documentation with limitations -- Missing param sections (preprocessing, QC, extra_args scope) -- Container version override and Singularity guides -- Parquet vs TSV output explanation -- pmultiqc citation added -- README updated with version table - -Closes #3, #9, #15" -``` - ---- - -## Task 12: Close resolved issues - -- [ ] **Step 1: Close issues via GitHub CLI** - -```bash -gh issue close 17 --repo bigbio/quantmsdiann --comment "Already implemented — --monitor-mod is extracted from diann_config.cfg (generated by sdrf-pipelines convert-diann) and passed to all DIA-NN steps via mod_flags." -gh issue close 2 --repo bigbio/quantmsdiann --comment "Superseded by #4 (Phase 6: consolidate param generation to sdrf-pipelines)." -gh issue close 1 --repo bigbio/quantmsdiann --comment "Resolved — docs/parameters.md created with comprehensive parameter reference." -gh issue close 3 --repo bigbio/quantmsdiann --comment "Resolved — diann_extra_args scope documented in docs/usage.md." -gh issue close 9 --repo bigbio/quantmsdiann --comment "Resolved — container version override guide and Singularity usage added to docs/usage.md." -gh issue close 15 --repo bigbio/quantmsdiann --comment "Resolved — docs/usage.md input documentation updated." -``` - ---- - -## Task 13: Final validation and push - -- [ ] **Step 1: Run full validation suite** - -```bash -conda run -n nfcore pre-commit run --all-files -conda run -n nfcore nf-core pipelines lint --release --dir . -``` - -Expected: 0 failures on both. - -- [ ] **Step 2: Push dda branch and create PR** - -```bash -git push -u origin dda -gh pr create --title "feat: v1.0.0 release — robustness, DDA support, features, docs" --body "$(cat <<'PREOF' -## Summary -- Robustness fixes: pipefail, error_retry, empty input guards -- DDA support via --diann_dda flag (DIA-NN >= 2.3.2) -- New params: --light-models, --export-quant, --site-ms1-quant -- InfinDIA groundwork (experimental) -- DIA-NN 2.3.2 version config -- New test configs: test_dda, test_dia_skip_preanalysis -- Comprehensive docs: parameters.md, complete usage.md, output.md - -## Issues -Closes #1, #3, #5, #7, #9, #10, #15, #17 - -## Test plan -- [ ] Existing CI tests pass (test_dia, test_dia_dotd) -- [ ] New test_dda passes with BSA dataset on DIA-NN 2.3.2 -- [ ] test_dia_skip_preanalysis passes -- [ ] nf-core lint --release: 0 failures -- [ ] pre-commit: all passing -PREOF -)" --base dev -``` diff --git a/lib/VersionUtils.groovy b/lib/VersionUtils.groovy new file mode 100644 index 0000000..f340a61 --- /dev/null +++ b/lib/VersionUtils.groovy @@ -0,0 +1,34 @@ +/** + * Semantic version comparison utility for DIA-NN version guards. + * + * Nextflow auto-loads all classes in lib/, so these are available + * in workflows and module scripts without explicit imports. + */ +class VersionUtils { + + /** + * Compare two version strings semantically (e.g. '2.10.0' > '2.3.2'). + * Returns negative if a < b, zero if equal, positive if a > b. + */ + static int compare(String a, String b) { + def partsA = a.tokenize('.').collect { it.isInteger() ? it.toInteger() : 0 } + def partsB = b.tokenize('.').collect { it.isInteger() ? it.toInteger() : 0 } + def maxLen = Math.max(partsA.size(), partsB.size()) + for (int i = 0; i < maxLen; i++) { + int va = i < partsA.size() ? partsA[i] : 0 + int vb = i < partsB.size() ? partsB[i] : 0 + if (va != vb) return va <=> vb + } + return 0 + } + + /** True if version is strictly less than required. */ + static boolean versionLessThan(String version, String required) { + return compare(version, required) < 0 + } + + /** True if version is greater than or equal to required. */ + static boolean versionAtLeast(String version, String required) { + return compare(version, required) >= 0 + } +} diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index a16f551..00e5659 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -66,7 +66,7 @@ process FINAL_QUANTIFICATION { report_decoys = params.diann_report_decoys ? "--report-decoys": "" diann_export_xic = params.diann_export_xic ? "--xic": "" // --direct-quant only exists in DIA-NN >= 1.9.2 (QuantUMS counterpart); skip for older versions - quantums = params.quantums ? "" : (params.diann_version >= '1.9' ? "--direct-quant" : "") + quantums = params.quantums ? "" : (VersionUtils.versionAtLeast(params.diann_version, '1.9') ? "--direct-quant" : "") quantums_train_runs = params.quantums_train_runs ? "--quant-train-runs $params.quantums_train_runs": "" quantums_sel_runs = params.quantums_sel_runs ? "--quant-sel-runs $params.quantums_sel_runs": "" quantums_params = params.quantums_params ? "--quant-params $params.quantums_params": "" diff --git a/modules/local/diann/individual_analysis/main.nf b/modules/local/diann/individual_analysis/main.nf index b4e8507..a17af6a 100644 --- a/modules/local/diann/individual_analysis/main.nf +++ b/modules/local/diann/individual_analysis/main.nf @@ -86,8 +86,8 @@ process INDIVIDUAL_ANALYSIS { diann_dda_flag = params.diann_dda ? "--dda" : "" // Flags removed in DIA-NN 2.3.x — only pass for older versions - no_ifs_removal = params.diann_version < '2.3' ? "--no-ifs-removal" : "" - no_main_report = params.diann_version < '2.3' ? "--no-main-report" : "" + no_ifs_removal = VersionUtils.versionLessThan(params.diann_version, '2.3') ? "--no-ifs-removal" : "" + no_main_report = VersionUtils.versionLessThan(params.diann_version, '2.3') ? "--no-main-report" : "" // Per-file scan ranges from SDRF (empty = no flag, DIA-NN auto-detects) min_pr_mz = meta['ms1minmz'] ? "--min-pr-mz ${meta['ms1minmz']}" : "" diff --git a/workflows/dia.nf b/workflows/dia.nf index 351f4f2..be3f850 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -36,17 +36,17 @@ workflow DIA { ch_software_versions = channel.empty() // Version guard for DDA mode - if (params.diann_dda && params.diann_version < '2.3.2') { + if (params.diann_dda && VersionUtils.versionLessThan(params.diann_version, '2.3.2')) { error("DDA mode (--diann_dda) requires DIA-NN >= 2.3.2. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") } // Version guard for InfinDIA - if (params.enable_infin_dia && params.diann_version < '2.3.0') { + if (params.enable_infin_dia && VersionUtils.versionLessThan(params.diann_version, '2.3.0')) { error("InfinDIA requires DIA-NN >= 2.3.0. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") } // Version guard for DIA-NN 2.0+ features - if ((params.diann_light_models || params.diann_export_quant || params.diann_site_ms1_quant) && params.diann_version < '2.0') { + if ((params.diann_light_models || params.diann_export_quant || params.diann_site_ms1_quant) && VersionUtils.versionLessThan(params.diann_version, '2.0')) { def enabled = [] if (params.diann_light_models) enabled << '--light-models' if (params.diann_export_quant) enabled << '--export-quant' From 7df29e2663ae58b2730a31bc69d187edacce1468 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Tue, 7 Apr 2026 09:22:37 +0100 Subject: [PATCH 23/69] chore: bump version to 2.0.0dev for DDA support release DDA analysis support is a major feature warranting a major version bump. Co-Authored-By: Claude Opus 4.6 (1M context) --- nextflow.config | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nextflow.config b/nextflow.config index 8d8677c..dd305a0 100644 --- a/nextflow.config +++ b/nextflow.config @@ -373,7 +373,7 @@ manifest { mainScript = 'main.nf' defaultBranch = 'main' nextflowVersion = '!>=25.04.0' - version = '1.0.1dev' + version = '2.0.0dev' doi = '10.5281/zenodo.15573386' } From a3f4e2563f9c10f649cc298035d01f6d2735b2f2 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Tue, 7 Apr 2026 10:02:20 +0100 Subject: [PATCH 24/69] fix: standardize meta.yml naming, fix descriptions and minor issues MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Rename output version→versions in sdrf_parsing/meta.yml - Add ch_ prefix to input_file→ch_input_file in input_check/meta.yml - Fix grammar in pmultiqc and diann_msstats meta.yml descriptions - Fix glob pattern in decompress_dotd/meta.yml (double-dot expansion) - Update CITATIONS.md to link published Nature Methods article - Fix schema_input.json error messages (source name, whitespace) - Standardize quantmsdiann keyword in utils meta.yml Co-Authored-By: Claude Opus 4.6 (1M context) --- CITATIONS.md | 2 +- assets/schema_input.json | 4 ++-- modules/local/diann/diann_msstats/meta.yml | 2 +- modules/local/pmultiqc/meta.yml | 2 +- modules/local/sdrf_parsing/meta.yml | 2 +- modules/local/utils/decompress_dotd/meta.yml | 2 +- subworkflows/local/input_check/meta.yml | 2 +- subworkflows/local/utils_nfcore_quantms_pipeline/meta.yml | 2 +- 8 files changed, 9 insertions(+), 9 deletions(-) diff --git a/CITATIONS.md b/CITATIONS.md index d74e0f9..8985a13 100644 --- a/CITATIONS.md +++ b/CITATIONS.md @@ -1,6 +1,6 @@ # bigbio/quantmsdiann: Citations -## [Pipeline](https://www.researchsquare.com/article/rs-3002027/v1) +## [Pipeline](https://doi.org/10.1038/s41592-024-02343-1) > Dai C, Pfeuffer J, Wang H, Zheng P, Käll L, Sachsenberg T, Demichev V, Bai M, Kohlbacher O, Perez-Riverol Y. quantms: a cloud-based pipeline for quantitative proteomics enables the reanalysis of public proteomics data. Nat Methods. 2024 Jul 4. doi: 10.1038/s41592-024-02343-1. Epub ahead of print. PMID: 38965444. diff --git a/assets/schema_input.json b/assets/schema_input.json index 7b15010..1699aad 100644 --- a/assets/schema_input.json +++ b/assets/schema_input.json @@ -11,7 +11,7 @@ "source name": { "type": "string", "pattern": "^\\S+$", - "errorMessage": "Sample name must be provided and cannot contain spaces" + "errorMessage": "Source name must be provided and cannot contain spaces" }, "comment[data file]": { "type": "string", @@ -22,7 +22,7 @@ "assay name": { "type": "string", "pattern": "^\\S+$", - "errorMessage": "Assay name must be provided and cannot contain spaces", + "errorMessage": "Assay name must be provided and cannot contain whitespace", "meta": ["assay"] } } diff --git a/modules/local/diann/diann_msstats/meta.yml b/modules/local/diann/diann_msstats/meta.yml index ac1f147..a440d61 100644 --- a/modules/local/diann/diann_msstats/meta.yml +++ b/modules/local/diann/diann_msstats/meta.yml @@ -17,7 +17,7 @@ input: pattern: "*.tsv" - exp_design: type: file - description: An experimental design file including Sample and replicates column et al. + description: An experimental design file including Sample and replicates column etc. pattern: "*.tsv" - report_pr: type: file diff --git a/modules/local/pmultiqc/meta.yml b/modules/local/pmultiqc/meta.yml index adf63f2..fcca33e 100644 --- a/modules/local/pmultiqc/meta.yml +++ b/modules/local/pmultiqc/meta.yml @@ -23,7 +23,7 @@ output: pattern: "*.html" - quantmsdb: type: file - description: Sqlite3 database file stored protein psm and quantification information + description: SQLite3 database file that stores protein, PSM, and quantification information pattern: "*.db" - data: type: dir diff --git a/modules/local/sdrf_parsing/meta.yml b/modules/local/sdrf_parsing/meta.yml index 7c311f4..846cfaa 100644 --- a/modules/local/sdrf_parsing/meta.yml +++ b/modules/local/sdrf_parsing/meta.yml @@ -28,7 +28,7 @@ output: type: file description: log file pattern: "*.log" - - version: + - versions: type: file description: File containing software version pattern: "versions.yml" diff --git a/modules/local/utils/decompress_dotd/meta.yml b/modules/local/utils/decompress_dotd/meta.yml index bbc7c58..55330d3 100644 --- a/modules/local/utils/decompress_dotd/meta.yml +++ b/modules/local/utils/decompress_dotd/meta.yml @@ -22,7 +22,7 @@ input: type: file description: | Bruker Raw file archived using tar - pattern: "*.{d.tar,.tar,.gz,.d.tar.gz}" + pattern: "*.{d.tar,tar,gz,d.tar.gz}" output: - meta: type: map diff --git a/subworkflows/local/input_check/meta.yml b/subworkflows/local/input_check/meta.yml index abe2c7f..3c88724 100644 --- a/subworkflows/local/input_check/meta.yml +++ b/subworkflows/local/input_check/meta.yml @@ -9,7 +9,7 @@ keywords: components: - samplesheet/check input: - - input_file: + - ch_input_file: type: file description: | Input file to be validated diff --git a/subworkflows/local/utils_nfcore_quantms_pipeline/meta.yml b/subworkflows/local/utils_nfcore_quantms_pipeline/meta.yml index 06365ae..cf1fd6d 100644 --- a/subworkflows/local/utils_nfcore_quantms_pipeline/meta.yml +++ b/subworkflows/local/utils_nfcore_quantms_pipeline/meta.yml @@ -4,7 +4,7 @@ description: Pipeline completion utilities for the nf-core quantmsdiann pipeline keywords: - utils - nf-core - - quantms + - quantmsdiann components: - completionemail - completionsummary From fc96eeb95e608f797e4bb904bd42b59616412f58 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Tue, 7 Apr 2026 13:10:10 +0100 Subject: [PATCH 25/69] feat: auto-detect DDA mode from SDRF acquisition method column DDA mode is now automatically detected from the SDRF `comment[proteomics data acquisition method]` column when it contains `data-dependent acquisition`. The `--diann_dda` flag is kept as a fallback for SDRFs that lack this column. - Modules now read acquisition method from meta instead of params - INSILICO_LIBRARY_GENERATION receives is_dda as input from workflow - Version guard triggers for both param and SDRF-detected DDA - Updated docs, parameters, and schema to reflect auto-detection Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/parameters.md | 33 ++++++++++--------- docs/usage.md | 13 ++++++-- .../diann/assemble_empirical_library/main.nf | 2 +- .../local/diann/final_quantification/main.nf | 2 +- .../local/diann/individual_analysis/main.nf | 2 +- .../diann/insilico_library_generation/main.nf | 3 +- .../insilico_library_generation/meta.yml | 3 ++ .../local/diann/preliminary_analysis/main.nf | 2 +- nextflow.config | 2 +- nextflow_schema.json | 2 +- .../local/create_input_channel/main.nf | 6 ++-- workflows/dia.nf | 13 ++++++-- 12 files changed, 53 insertions(+), 30 deletions(-) diff --git a/docs/parameters.md b/docs/parameters.md index 0a3eac8..f3c76c0 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -52,16 +52,16 @@ This document lists every pipeline parameter organised by category. Default valu ## 5. DIA-NN General -| Parameter | Type | Default | Description | -| ------------------------ | ------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `--diann_version` | string | `1.8.1` | DIA-NN version used by the workflow. Controls version-dependent flags (e.g. `--monitor-mod` for 1.8.x). See [DIA-NN Version Selection](usage.md#dia-nn-version-selection). | -| `--diann_debug` | integer | `3` | DIA-NN debug/verbosity level (0-4). Higher values produce more verbose logs. | -| `--diann_speclib` | string | `null` | Path to an external spectral library. If provided, the in-silico library generation step is skipped. | -| `--diann_extra_args` | string | `null` | Extra arguments appended to all DIA-NN steps. Flags incompatible with a step are automatically stripped with a warning. See [Passing Extra Arguments to DIA-NN](usage.md#passing-extra-arguments-to-dia-nn). | -| `--diann_dda` | boolean | `false` | Enable DDA (Data-Dependent Acquisition) analysis mode. Passes `--dda` to all DIA-NN steps. Requires DIA-NN >= 2.3.2. Beta feature. | -| `--diann_light_models` | boolean | `false` | Enable `--light-models` for 10x faster in-silico library generation. Requires DIA-NN >= 2.0. | -| `--diann_export_quant` | boolean | `false` | Enable `--export-quant` for fragment-level parquet data export. Requires DIA-NN >= 2.0. | -| `--diann_site_ms1_quant` | boolean | `false` | Enable `--site-ms1-quant` to use MS1 apex intensities for PTM site quantification. Requires DIA-NN >= 2.0. | +| Parameter | Type | Default | Description | +| ------------------------ | ------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `--diann_version` | string | `1.8.1` | DIA-NN version used by the workflow. Controls version-dependent flags (e.g. `--monitor-mod` for 1.8.x). See [DIA-NN Version Selection](usage.md#dia-nn-version-selection). | +| `--diann_debug` | integer | `3` | DIA-NN debug/verbosity level (0-4). Higher values produce more verbose logs. | +| `--diann_speclib` | string | `null` | Path to an external spectral library. If provided, the in-silico library generation step is skipped. | +| `--diann_extra_args` | string | `null` | Extra arguments appended to all DIA-NN steps. Flags incompatible with a step are automatically stripped with a warning. See [Passing Extra Arguments to DIA-NN](usage.md#passing-extra-arguments-to-dia-nn). | +| `--diann_dda` | boolean | `false` | Explicitly enable DDA mode. Normally auto-detected from the SDRF `comment[proteomics data acquisition method]` column. Use this flag only when the SDRF lacks the acquisition method. Requires DIA-NN >= 2.3.2. | +| `--diann_light_models` | boolean | `false` | Enable `--light-models` for 10x faster in-silico library generation. Requires DIA-NN >= 2.0. | +| `--diann_export_quant` | boolean | `false` | Enable `--export-quant` for fragment-level parquet data export. Requires DIA-NN >= 2.0. | +| `--diann_site_ms1_quant` | boolean | `false` | Enable `--site-ms1-quant` to use MS1 apex intensities for PTM site quantification. Requires DIA-NN >= 2.0. | ## 6. Mass Accuracy & Calibration @@ -123,13 +123,14 @@ This document lists every pipeline parameter organised by category. Default valu ## 12. DDA Mode -| Parameter | Type | Default | Description | -| ------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `--diann_dda` | boolean | `false` | Enable DDA analysis mode. Passes `--dda` to all DIA-NN steps. Requires DIA-NN >= 2.3.2 (use `-profile diann_v2_3_2`). This is a beta feature with known limitations; see the usage documentation for details. | +| Parameter | Type | Default | Description | +| ------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `--diann_dda` | boolean | `false` | Explicitly enable DDA mode when SDRF lacks the acquisition method column. Normally DDA is auto-detected from the SDRF `comment[proteomics data acquisition method]`. Requires DIA-NN >= 2.3.2 (use `-profile diann_v2_3_2`). Beta feature. | -> **Note:** DDA support requires DIA-NN >= 2.3.2. Enable this profile with -> `-profile diann_v2_3_2`. The DDA mode is experimental and may not support -> all pipeline features available in DIA mode. +> **Note:** DDA mode is auto-detected from the SDRF when the `comment[proteomics data acquisition method]` +> column contains `data-dependent acquisition`. The `--diann_dda` flag is only needed as a +> fallback when the SDRF does not include this column. DDA requires DIA-NN >= 2.3.2 +> (`-profile diann_v2_3_2`). ## 13. InfinDIA (Experimental) diff --git a/docs/usage.md b/docs/usage.md index 4055f47..8b696df 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -66,7 +66,16 @@ For Synchro-PASEF data, enable `--diann_tims_sum` (which adds `--quant-tims-sum` ### DDA Analysis Mode (Beta) -DIA-NN 2.3.2+ supports DDA data analysis via the `--dda` flag. Enable it with: +DIA-NN 2.3.2+ supports DDA data analysis via the `--dda` flag. The pipeline **auto-detects DDA mode** from the SDRF `comment[proteomics data acquisition method]` column — no extra flags needed if your SDRF contains `data-dependent acquisition`: + +```bash +nextflow run bigbio/quantmsdiann \ + --input dda_sdrf.tsv \ + --database proteins.fasta \ + -profile diann_v2_3_2,docker +``` + +If your SDRF does not include the acquisition method column, you can explicitly enable DDA mode with `--diann_dda true`: ```bash nextflow run bigbio/quantmsdiann \ @@ -84,7 +93,7 @@ nextflow run bigbio/quantmsdiann \ - No isobaric labeling or reporter-tag quantification - Primary use cases: legacy DDA reanalysis, spectral library creation, immunopeptidomics -The pipeline uses the same workflow for DDA as DIA — the `--dda` flag is passed to all DIA-NN steps automatically. +The pipeline uses the same workflow for DDA as DIA — the `--dda` flag is passed to all DIA-NN steps automatically when DDA is detected from the SDRF or enabled via `--diann_dda`. ### Preprocessing Options diff --git a/modules/local/diann/assemble_empirical_library/main.nf b/modules/local/diann/assemble_empirical_library/main.nf index b3f0181..2bfb67e 100644 --- a/modules/local/diann/assemble_empirical_library/main.nf +++ b/modules/local/diann/assemble_empirical_library/main.nf @@ -54,7 +54,7 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" diann_tims_sum = params.diann_tims_sum ? "--quant-tims-sum" : "" diann_im_window = params.diann_im_window ? "--im-window $params.diann_im_window" : "" - diann_dda_flag = params.diann_dda ? "--dda" : "" + diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" """ # Precursor Tolerance value was: ${meta['precursormasstolerance']} diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index 00e5659..a1ba609 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -72,7 +72,7 @@ process FINAL_QUANTIFICATION { quantums_params = params.quantums_params ? "--quant-params $params.quantums_params": "" diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" diann_use_quant = params.diann_use_quant ? "--use-quant" : "" - diann_dda_flag = params.diann_dda ? "--dda" : "" + diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" diann_export_quant = params.diann_export_quant ? "--export-quant" : "" diann_site_ms1_quant = params.diann_site_ms1_quant ? "--site-ms1-quant" : "" diff --git a/modules/local/diann/individual_analysis/main.nf b/modules/local/diann/individual_analysis/main.nf index a17af6a..975ceaf 100644 --- a/modules/local/diann/individual_analysis/main.nf +++ b/modules/local/diann/individual_analysis/main.nf @@ -83,7 +83,7 @@ process INDIVIDUAL_ANALYSIS { diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" diann_tims_sum = params.diann_tims_sum ? "--quant-tims-sum" : "" diann_im_window = params.diann_im_window ? "--im-window $params.diann_im_window" : "" - diann_dda_flag = params.diann_dda ? "--dda" : "" + diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" // Flags removed in DIA-NN 2.3.x — only pass for older versions no_ifs_removal = VersionUtils.versionLessThan(params.diann_version, '2.3') ? "--no-ifs-removal" : "" diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index 7de4665..8b05859 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -11,6 +11,7 @@ process INSILICO_LIBRARY_GENERATION { input: path(fasta) path(diann_config) + val(is_dda) output: path "versions.yml", emit: versions @@ -47,7 +48,7 @@ process INSILICO_LIBRARY_GENERATION { max_fr_mz = params.max_fr_mz ? "--max-fr-mz $params.max_fr_mz":"" met_excision = params.met_excision ? "--met-excision" : "" diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" - diann_dda_flag = params.diann_dda ? "--dda" : "" + diann_dda_flag = is_dda ? "--dda" : "" diann_light_models = params.diann_light_models ? "--light-models" : "" infin_dia_flag = params.enable_infin_dia ? "--infin-dia" : "" pre_select_flag = params.diann_pre_select ? "--pre-select $params.diann_pre_select" : "" diff --git a/modules/local/diann/insilico_library_generation/meta.yml b/modules/local/diann/insilico_library_generation/meta.yml index 5f9d68b..a6185d7 100644 --- a/modules/local/diann/insilico_library_generation/meta.yml +++ b/modules/local/diann/insilico_library_generation/meta.yml @@ -19,6 +19,9 @@ input: type: file description: specifies a configuration file to load options/commands from. pattern: "*.cfg" + - is_dda: + type: boolean + description: Whether DDA mode is enabled (auto-detected from SDRF or set via --diann_dda) output: - predict_speclib: type: file diff --git a/modules/local/diann/preliminary_analysis/main.nf b/modules/local/diann/preliminary_analysis/main.nf index 7e24280..f9a3508 100644 --- a/modules/local/diann/preliminary_analysis/main.nf +++ b/modules/local/diann/preliminary_analysis/main.nf @@ -68,7 +68,7 @@ process PRELIMINARY_ANALYSIS { scan_window = params.scan_window_automatic ? '' : "--window $params.scan_window" diann_tims_sum = params.diann_tims_sum ? "--quant-tims-sum" : "" diann_im_window = params.diann_im_window ? "--im-window $params.diann_im_window" : "" - diann_dda_flag = params.diann_dda ? "--dda" : "" + diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" // Per-file scan ranges from SDRF (empty = no flag, DIA-NN auto-detects) min_pr_mz = meta['ms1minmz'] ? "--min-pr-mz ${meta['ms1minmz']}" : "" diff --git a/nextflow.config b/nextflow.config index dd305a0..696ebc4 100644 --- a/nextflow.config +++ b/nextflow.config @@ -52,7 +52,7 @@ params { diann_debug = 3 diann_speclib = null diann_extra_args = null - diann_dda = false // Enable DDA analysis mode (requires DIA-NN >= 2.3.2) + diann_dda = false // Fallback: explicitly enable DDA when SDRF lacks acquisition method (requires DIA-NN >= 2.3.2) diann_light_models = false // add '--light-models' for 10x faster library generation (DIA-NN >= 2.0) diann_export_quant = false // add '--export-quant' for fragment-level parquet export (DIA-NN >= 2.0) diann_site_ms1_quant = false // add '--site-ms1-quant' for MS1 apex PTM quantification (DIA-NN >= 2.0) diff --git a/nextflow_schema.json b/nextflow_schema.json index ebe3a45..08d564b 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -458,7 +458,7 @@ }, "diann_dda": { "type": "boolean", - "description": "Enable DDA (Data-Dependent Acquisition) analysis mode. Passes --dda to all DIA-NN steps. Requires DIA-NN >= 2.3.2 (use -profile diann_v2_3_2). Beta feature.", + "description": "Explicitly enable DDA mode. Normally auto-detected from the SDRF acquisition method column. Use only when SDRF lacks this column. Requires DIA-NN >= 2.3.2.", "fa_icon": "fas fa-flask", "default": false }, diff --git a/subworkflows/local/create_input_channel/main.nf b/subworkflows/local/create_input_channel/main.nf index c0249f1..a52b9ee 100644 --- a/subworkflows/local/create_input_channel/main.nf +++ b/subworkflows/local/create_input_channel/main.nf @@ -72,16 +72,16 @@ def create_meta_channel(LinkedHashMap row, enzymes, files, wrapper) { exit(1, "ERROR: Please check input file -> File Uri does not exist!\n${filestr}") } - // Validate acquisition method + // Detect acquisition method from SDRF or fallback to --diann_dda param def acqMethod = row.AcquisitionMethod?.toString()?.trim() ?: "" if (acqMethod.toLowerCase().contains("data-independent acquisition") || acqMethod.toLowerCase().contains("dia")) { meta.acquisition_method = "dia" - } else if (params.diann_dda && (acqMethod.toLowerCase().contains("data-dependent acquisition") || acqMethod.toLowerCase().contains("dda"))) { + } else if (acqMethod.toLowerCase().contains("data-dependent acquisition") || acqMethod.toLowerCase().contains("dda")) { meta.acquisition_method = "dda" } else if (acqMethod.isEmpty()) { meta.acquisition_method = params.diann_dda ? "dda" : "dia" } else { - log.error("Unsupported acquisition method: '${acqMethod}'. This pipeline supports DIA" + (params.diann_dda ? " and DDA (--diann_dda)" : "") + ". Found in file: ${filestr}") + log.error("Unsupported acquisition method: '${acqMethod}'. This pipeline supports DIA and DDA. Found in file: ${filestr}") exit(1) } diff --git a/workflows/dia.nf b/workflows/dia.nf index be3f850..69dd78d 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -35,7 +35,7 @@ workflow DIA { ch_software_versions = channel.empty() - // Version guard for DDA mode + // Version guard for DDA mode (when explicitly set via param) if (params.diann_dda && VersionUtils.versionLessThan(params.diann_version, '2.3.2')) { error("DDA mode (--diann_dda) requires DIA-NN >= 2.3.2. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") } @@ -68,6 +68,15 @@ workflow DIA { .ifEmpty { error("No valid input files found after SDRF parsing. Check your SDRF file and input paths.") } .first() + // Determine DDA mode: true if explicitly set via param OR auto-detected from SDRF + ch_is_dda = ch_experiment_meta.map { meta -> + def dda = params.diann_dda || meta.acquisition_method == 'dda' + if (dda && VersionUtils.versionLessThan(params.diann_version, '2.3.2')) { + error("DDA mode (detected from SDRF) requires DIA-NN >= 2.3.2. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") + } + return dda + } + // diann_config.cfg comes directly from SDRF_PARSING (convert-diann) // Use as value channel so it can be consumed by all per-file processes ch_diann_cfg_val = ch_diann_cfg @@ -78,7 +87,7 @@ workflow DIA { if (params.diann_speclib != null && params.diann_speclib.toString() != "") { speclib = channel.from(file(params.diann_speclib, checkIfExists: true)) } else { - INSILICO_LIBRARY_GENERATION(ch_searchdb, ch_diann_cfg_val) + INSILICO_LIBRARY_GENERATION(ch_searchdb, ch_diann_cfg_val, ch_is_dda) speclib = INSILICO_LIBRARY_GENERATION.out.predict_speclib } From 9f4ee58d98a90a9eaec8ec03381519da7748f709 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Tue, 7 Apr 2026 13:14:07 +0100 Subject: [PATCH 26/69] fix: address Copilot review comments - Revert meta.yml input_file naming to match workflow interface - Gate --pre-select behind enable_infin_dia (InfinDIA-only flag) - Tighten --direct-quant version check from 1.9 to 1.9.2 - Improve GHCR login: guard both token and username, skip test_dda gracefully on fork PRs where credentials are unavailable Co-Authored-By: Claude Opus 4.6 (1M context) --- .github/workflows/ci.yml | 10 +++++++--- modules/local/diann/final_quantification/main.nf | 4 ++-- .../local/diann/insilico_library_generation/main.nf | 2 +- subworkflows/local/input_check/meta.yml | 4 ++-- 4 files changed, 12 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index aadf1dd..e80a477 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -64,16 +64,20 @@ jobs: if: matrix.test_profile == 'test_dda' env: GHCR_TOKEN: ${{ secrets.GHCR_TOKEN }} + GHCR_USERNAME: ${{ secrets.GHCR_USERNAME }} run: | - if [ -n "$GHCR_TOKEN" ]; then - echo "${{ secrets.GHCR_TOKEN }}" | docker login ghcr.io -u ${{ secrets.GHCR_USERNAME }} --password-stdin + if [ -z "$GHCR_TOKEN" ] || [ -z "$GHCR_USERNAME" ]; then + echo "::warning::Skipping test_dda: GHCR credentials not available (expected for fork PRs)" + echo "SKIP_DDA=true" >> $GITHUB_ENV + exit 0 fi + echo "$GHCR_TOKEN" | docker login ghcr.io -u "$GHCR_USERNAME" --password-stdin - name: Disk space cleanup uses: jlumbroso/free-disk-space@v1.3.1 - name: Run pipeline with test data in docker/singularity profile - if: github.event.pull_request.base.ref != 'master' + if: github.event.pull_request.base.ref != 'master' && env.SKIP_DDA != 'true' run: | nextflow run ${GITHUB_WORKSPACE} -profile $TEST_PROFILE,$EXEC_PROFILE,dev --outdir ${TEST_PROFILE}_${EXEC_PROFILE}_results diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index a1ba609..64aa5cc 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -65,8 +65,8 @@ process FINAL_QUANTIFICATION { no_norm = params.diann_normalize ? "" : "--no-norm" report_decoys = params.diann_report_decoys ? "--report-decoys": "" diann_export_xic = params.diann_export_xic ? "--xic": "" - // --direct-quant only exists in DIA-NN >= 1.9.2 (QuantUMS counterpart); skip for older versions - quantums = params.quantums ? "" : (VersionUtils.versionAtLeast(params.diann_version, '1.9') ? "--direct-quant" : "") + // --direct-quant exists in DIA-NN >= 1.9.2 (QuantUMS counterpart); skip for older versions + quantums = params.quantums ? "" : (VersionUtils.versionAtLeast(params.diann_version, '1.9.2') ? "--direct-quant" : "") quantums_train_runs = params.quantums_train_runs ? "--quant-train-runs $params.quantums_train_runs": "" quantums_sel_runs = params.quantums_sel_runs ? "--quant-sel-runs $params.quantums_sel_runs": "" quantums_params = params.quantums_params ? "--quant-params $params.quantums_params": "" diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index 8b05859..66bca5e 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -51,7 +51,7 @@ process INSILICO_LIBRARY_GENERATION { diann_dda_flag = is_dda ? "--dda" : "" diann_light_models = params.diann_light_models ? "--light-models" : "" infin_dia_flag = params.enable_infin_dia ? "--infin-dia" : "" - pre_select_flag = params.diann_pre_select ? "--pre-select $params.diann_pre_select" : "" + pre_select_flag = (params.enable_infin_dia && params.diann_pre_select) ? "--pre-select $params.diann_pre_select" : "" """ diann `cat ${diann_config}` \\ diff --git a/subworkflows/local/input_check/meta.yml b/subworkflows/local/input_check/meta.yml index 3c88724..1f2cefb 100644 --- a/subworkflows/local/input_check/meta.yml +++ b/subworkflows/local/input_check/meta.yml @@ -9,12 +9,12 @@ keywords: components: - samplesheet/check input: - - ch_input_file: + - input_file: type: file description: | Input file to be validated output: - - ch_input_file: + - input_file: type: file description: | Channel containing validated input files From f04e754d0d17361686748c79369fa9e8ab5e2ddb Mon Sep 17 00:00:00 2001 From: yueqixuan Date: Wed, 8 Apr 2026 10:59:01 +0800 Subject: [PATCH 27/69] Support multiplexing --- .../local/diann/final_quantification/main.nf | 14 +- nextflow.config | 2 + nextflow_schema.json | 13 ++ .../local/create_input_channel/main.nf | 136 ++++++++---------- 4 files changed, 86 insertions(+), 79 deletions(-) diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index 64aa5cc..f83af13 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -25,11 +25,11 @@ process FINAL_QUANTIFICATION { path "diann_report.{tsv,parquet}", emit: main_report, optional: true path "diann_report.manifest.txt", emit: report_manifest, optional: true path "diann_report.protein_description.tsv", emit: protein_description, optional: true - path "diann_report.stats.tsv", emit: report_stats - path "diann_report.pr_matrix.tsv", emit: pr_matrix - path "diann_report.pg_matrix.tsv", emit: pg_matrix - path "diann_report.gg_matrix.tsv", emit: gg_matrix - path "diann_report.unique_genes_matrix.tsv", emit: unique_gene_matrix + path "diann_report.stats.tsv", emit: report_stats, optional: true + path "diann_report.pr_matrix.tsv", emit: pr_matrix, optional: true + path "diann_report.pg_matrix.tsv", emit: pg_matrix, optional: true + path "diann_report.gg_matrix.tsv", emit: gg_matrix, optional: true + path "diann_report.unique_genes_matrix.tsv", emit: unique_gene_matrix, optional: true path "diannsummary.log", emit: log // Different library files format are exported due to different DIA-NN versions @@ -75,6 +75,8 @@ process FINAL_QUANTIFICATION { diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" diann_export_quant = params.diann_export_quant ? "--export-quant" : "" diann_site_ms1_quant = params.diann_site_ms1_quant ? "--site-ms1-quant" : "" + diann_channel_run_norm = params.diann_channel_run_norm ? "--channel-run-norm" : "" + diann_channel_spec_norm = params.diann_channel_spec_norm ? "--channel-spec-norm" : "" """ # Notes: if .quant files are passed, mzml/.d files are not accessed, so the name needs to be passed but files @@ -107,6 +109,8 @@ process FINAL_QUANTIFICATION { ${diann_dda_flag} \\ ${diann_export_quant} \\ ${diann_site_ms1_quant} \\ + ${diann_channel_run_norm} \\ + ${diann_channel_spec_norm} \\ \${mod_flags} \\ $args diff --git a/nextflow.config b/nextflow.config index 696ebc4..a8fbb0d 100644 --- a/nextflow.config +++ b/nextflow.config @@ -101,6 +101,8 @@ params { quantums_params = null diann_no_peptidoforms = false // add '--no-peptidoforms' diann_use_quant = true // add '--use-quant' to FINAL_QUANTIFICATION + diann_channel_run_norm = false // add '--channel-run-norm' to FINAL_QUANTIFICATION + diann_channel_spec_norm = false // add '--channel-spec-norm' to FINAL_QUANTIFICATION // pmultiqc options enable_pmultiqc = true diff --git a/nextflow_schema.json b/nextflow_schema.json index 08d564b..9fe8f21 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -411,6 +411,19 @@ "fa_icon": "far fa-check-square", "default": true }, + + "diann_channel_run_norm": { + "type": "boolean", + "description": "Set '--channel-run-norm'. Run-specific normalisation for multiplexing (e.g., Protein Turnover SILAC)", + "fa_icon": "far fa-check-square", + "default": false + }, + "diann_channel_spec_norm": { + "type": "boolean", + "description": "Set '--channel-spec-norm'. Channel-specific normalisation for multiplexing (Independent Samples)", + "fa_icon": "far fa-check-square", + "default": false + }, "skip_preliminary_analysis": { "type": "boolean", "description": "Skip the preliminary analysis step, thus use the passed spectral library as-is instead of generating a local consensus library.", diff --git a/subworkflows/local/create_input_channel/main.nf b/subworkflows/local/create_input_channel/main.nf index a52b9ee..577441b 100644 --- a/subworkflows/local/create_input_channel/main.nf +++ b/subworkflows/local/create_input_channel/main.nf @@ -29,8 +29,23 @@ workflow CREATE_INPUT_CHANNEL { .combine(ch_expdesign) .splitCsv(header: true, sep: '\t') .map { experiment_id, row -> + def filestr + if (!params.root_folder) { + filestr = row.URI?.toString()?.trim() ? row.URI.toString() : row.Filename.toString() + } else { + filestr = row.Filename.toString() + filestr = params.root_folder + File.separator + filestr + filestr = (params.local_input_type + ? filestr.take(filestr.lastIndexOf('.')) + '.' + params.local_input_type + : filestr) + } + return [filestr, experiment_id, row] + } + .groupTuple(by: 0) + .map { filestr, experiment_ids, rows -> + def experiment_id = experiment_ids[0] def wrapper = [acquisition_method: "", experiment_id: experiment_id] - create_meta_channel(row, enzymes, files, wrapper) + return create_meta_channel_grouped(filestr, rows, wrapper) } .set { ch_meta_config_dia } @@ -42,38 +57,23 @@ workflow CREATE_INPUT_CHANNEL { } // Function to get list of [meta, [ spectra_files ]] -def create_meta_channel(LinkedHashMap row, enzymes, files, wrapper) { +def create_meta_channel_grouped(String filestr, List rows, Map wrapper) { def meta = [:] - def filestr - // Always use SDRF format - if (!params.root_folder) { - filestr = row.URI?.toString()?.trim() ? row.URI.toString() : row.Filename.toString() - } - else { - filestr = row.Filename.toString() - } + def base_row = rows[0] def fileName = file(filestr).name def dotIndex = fileName.lastIndexOf('.') meta.id = dotIndex > 0 ? fileName.take(dotIndex) : fileName meta.experiment_id = wrapper.experiment_id - // apply transformations given by specified root_folder and type - if (params.root_folder) { - filestr = params.root_folder + File.separator + filestr - filestr = (params.local_input_type - ? filestr.take(filestr.lastIndexOf('.')) + '.' + params.local_input_type - : filestr) - } - // existence check if (!file(filestr).exists()) { exit(1, "ERROR: Please check input file -> File Uri does not exist!\n${filestr}") } // Detect acquisition method from SDRF or fallback to --diann_dda param - def acqMethod = row.AcquisitionMethod?.toString()?.trim() ?: "" + def acqMethod = base_row.AcquisitionMethod?.toString()?.trim() ?: "" if (acqMethod.toLowerCase().contains("data-independent acquisition") || acqMethod.toLowerCase().contains("dia")) { meta.acquisition_method = "dia" } else if (acqMethod.toLowerCase().contains("data-dependent acquisition") || acqMethod.toLowerCase().contains("dda")) { @@ -85,21 +85,35 @@ def create_meta_channel(LinkedHashMap row, enzymes, files, wrapper) { exit(1) } - // DissociationMethod is already normalized by convert-diann (HCD, CID, ETD, ECD) - meta.dissociationmethod = row.DissociationMethod?.toString()?.trim() ?: "" - + meta.dissociationmethod = base_row.DissociationMethod?.toString()?.trim() ?: "" wrapper.acquisition_method = meta.acquisition_method - // Validate required SDRF columns - these parameters are exclusively read from SDRF (no command-line override) + def labels = rows.collect { it.Label?.toString()?.trim() }.findAll { it }.unique() + meta.labelling_type = labels.join(';') + + def is_plexdia = labels.size() > 1 || (labels.size() == 1 && !labels[0].toLowerCase().contains("label free")) + meta.plexdia = is_plexdia + + def enzymes = rows.collect { it.Enzyme?.toString()?.trim() }.findAll { it }.unique() + if (enzymes.size() > 1) { + log.error("Currently only one enzyme is supported per file. Found conflicting enzymes for ${filestr}: '${enzymes}'.") + exit(1) + } + meta.enzyme = enzymes ? enzymes[0] : null + + def fixedMods = rows.collect { it.FixedModifications?.toString()?.trim() }.findAll { it }.unique() + meta.fixedmodifications = fixedMods ? fixedMods[0] : null + + // Validate required SDRF columns def requiredColumns = [ - 'Label': row.Label, - 'Enzyme': row.Enzyme, - 'FixedModifications': row.FixedModifications + 'Label': meta.labelling_type, + 'Enzyme': meta.enzyme, + 'FixedModifications': meta.fixedmodifications ] def missingColumns = [] requiredColumns.each { colName, colValue -> - if (colValue == null || colValue.toString().trim().isEmpty()) { + if (colValue == null || colValue.toString().isEmpty()) { missingColumns.add(colName) } } @@ -110,20 +124,13 @@ def create_meta_channel(LinkedHashMap row, enzymes, files, wrapper) { exit(1) } - // Set values from SDRF (required columns) - meta.labelling_type = row.Label - meta.fixedmodifications = row.FixedModifications - meta.enzyme = row.Enzyme - - // Set tolerance values: use SDRF if available, otherwise fall back to params def validUnits = ['ppm', 'da', 'Da', 'PPM'] - // Precursor mass tolerance - if (row.PrecursorMassTolerance != null && !row.PrecursorMassTolerance.toString().trim().isEmpty()) { + if (base_row.PrecursorMassTolerance != null && !base_row.PrecursorMassTolerance.toString().trim().isEmpty()) { try { - meta.precursormasstolerance = Double.parseDouble(row.PrecursorMassTolerance) + meta.precursormasstolerance = Double.parseDouble(base_row.PrecursorMassTolerance) } catch (NumberFormatException e) { - log.error("ERROR: Invalid PrecursorMassTolerance value '${row.PrecursorMassTolerance}' for file '${filestr}'. Must be a valid number.") + log.error("ERROR: Invalid PrecursorMassTolerance value '${base_row.PrecursorMassTolerance}' for file '${filestr}'. Must be a valid number.") exit(1) } } else { @@ -131,23 +138,21 @@ def create_meta_channel(LinkedHashMap row, enzymes, files, wrapper) { meta.precursormasstolerance = params.precursor_mass_tolerance } - // Precursor mass tolerance unit - if (row.PrecursorMassToleranceUnit != null && !row.PrecursorMassToleranceUnit.toString().trim().isEmpty()) { - if (!validUnits.any { row.PrecursorMassToleranceUnit.toString().equalsIgnoreCase(it) }) { - log.error("ERROR: Invalid PrecursorMassToleranceUnit '${row.PrecursorMassToleranceUnit}' for file '${filestr}'. Must be 'ppm' or 'Da'.") + if (base_row.PrecursorMassToleranceUnit != null && !base_row.PrecursorMassToleranceUnit.toString().trim().isEmpty()) { + if (!validUnits.any { base_row.PrecursorMassToleranceUnit.toString().equalsIgnoreCase(it) }) { + log.error("ERROR: Invalid PrecursorMassToleranceUnit '${base_row.PrecursorMassToleranceUnit}' for file '${filestr}'. Must be 'ppm' or 'Da'.") exit(1) } - meta.precursormasstoleranceunit = row.PrecursorMassToleranceUnit + meta.precursormasstoleranceunit = base_row.PrecursorMassToleranceUnit } else { meta.precursormasstoleranceunit = params.precursor_mass_tolerance_unit } - // Fragment mass tolerance - if (row.FragmentMassTolerance != null && !row.FragmentMassTolerance.toString().trim().isEmpty()) { + if (base_row.FragmentMassTolerance != null && !base_row.FragmentMassTolerance.toString().trim().isEmpty()) { try { - meta.fragmentmasstolerance = Double.parseDouble(row.FragmentMassTolerance) + meta.fragmentmasstolerance = Double.parseDouble(base_row.FragmentMassTolerance) } catch (NumberFormatException e) { - log.error("ERROR: Invalid FragmentMassTolerance value '${row.FragmentMassTolerance}' for file '${filestr}'. Must be a valid number.") + log.error("ERROR: Invalid FragmentMassTolerance value '${base_row.FragmentMassTolerance}' for file '${filestr}'. Must be a valid number.") exit(1) } } else { @@ -155,43 +160,26 @@ def create_meta_channel(LinkedHashMap row, enzymes, files, wrapper) { meta.fragmentmasstolerance = params.fragment_mass_tolerance } - // Fragment mass tolerance unit - if (row.FragmentMassToleranceUnit != null && !row.FragmentMassToleranceUnit.toString().trim().isEmpty()) { - if (!validUnits.any { row.FragmentMassToleranceUnit.toString().equalsIgnoreCase(it) }) { - log.error("ERROR: Invalid FragmentMassToleranceUnit '${row.FragmentMassToleranceUnit}' for file '${filestr}'. Must be 'ppm' or 'Da'.") + if (base_row.FragmentMassToleranceUnit != null && !base_row.FragmentMassToleranceUnit.toString().trim().isEmpty()) { + if (!validUnits.any { base_row.FragmentMassToleranceUnit.toString().equalsIgnoreCase(it) }) { + log.error("ERROR: Invalid FragmentMassToleranceUnit '${base_row.FragmentMassToleranceUnit}' for file '${filestr}'. Must be 'ppm' or 'Da'.") exit(1) } - meta.fragmentmasstoleranceunit = row.FragmentMassToleranceUnit + meta.fragmentmasstoleranceunit = base_row.FragmentMassToleranceUnit } else { meta.fragmentmasstoleranceunit = params.fragment_mass_tolerance_unit } - // Variable modifications: use SDRF if available, otherwise fall back to params - if (row.VariableModifications != null && !row.VariableModifications.toString().trim().isEmpty()) { - meta.variablemodifications = row.VariableModifications + if (base_row.VariableModifications != null && !base_row.VariableModifications.toString().trim().isEmpty()) { + meta.variablemodifications = base_row.VariableModifications } else { meta.variablemodifications = params.variable_mods } - // Per-file scan ranges (empty string = no flags passed, DIA-NN auto-detects) - meta.ms1minmz = row.MS1MinMz?.toString()?.trim() ?: "" - meta.ms1maxmz = row.MS1MaxMz?.toString()?.trim() ?: "" - meta.ms2minmz = row.MS2MinMz?.toString()?.trim() ?: "" - meta.ms2maxmz = row.MS2MaxMz?.toString()?.trim() ?: "" - - enzymes += row.Enzyme - if (enzymes.size() > 1) { - log.error("Currently only one enzyme is supported for the whole experiment. Specified was '${enzymes}'. Check or split your SDRF.") - log.error(filestr) - exit(1) - } - - // Check for duplicate files - if (filestr in files) { - log.error("Currently only one DIA-NN setting per file is supported for the whole experiment. ${filestr} has multiple entries in your SDRF. Consider splitting your design into multiple experiments.") - exit(1) - } - files += filestr + meta.ms1minmz = base_row.MS1MinMz?.toString()?.trim() ?: "" + meta.ms1maxmz = base_row.MS1MaxMz?.toString()?.trim() ?: "" + meta.ms2minmz = base_row.MS2MinMz?.toString()?.trim() ?: "" + meta.ms2maxmz = base_row.MS2MaxMz?.toString()?.trim() ?: "" return [meta, filestr] -} +} \ No newline at end of file From af505677ffb95a56f2cae4909920cba70cb036da Mon Sep 17 00:00:00 2001 From: yueqixuan Date: Wed, 8 Apr 2026 11:02:28 +0800 Subject: [PATCH 28/69] Support multiplexing --- subworkflows/local/create_input_channel/main.nf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/subworkflows/local/create_input_channel/main.nf b/subworkflows/local/create_input_channel/main.nf index 577441b..2208eca 100644 --- a/subworkflows/local/create_input_channel/main.nf +++ b/subworkflows/local/create_input_channel/main.nf @@ -104,7 +104,7 @@ def create_meta_channel_grouped(String filestr, List rows, Map wrapper) { def fixedMods = rows.collect { it.FixedModifications?.toString()?.trim() }.findAll { it }.unique() meta.fixedmodifications = fixedMods ? fixedMods[0] : null - // Validate required SDRF columns + // Validate required SDRF columns def requiredColumns = [ 'Label': meta.labelling_type, 'Enzyme': meta.enzyme, @@ -182,4 +182,4 @@ def create_meta_channel_grouped(String filestr, List rows, Map wrapper) { meta.ms2maxmz = base_row.MS2MaxMz?.toString()?.trim() ?: "" return [meta, filestr] -} \ No newline at end of file +} From 512f285e89fb87ae0732694d932b536fd40b6dc1 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Thu, 9 Apr 2026 09:50:02 +0100 Subject: [PATCH 29/69] fix: handle missing log file in insilico library generation for DIA-NN >= 2.x DIA-NN 2.3.2 no longer produces a *lib.log.txt file during in-silico library generation. The hard-coded `cp *lib.log.txt` caused a fatal error. Now gracefully handles the missing log file with a fallback message. Co-Authored-By: Claude Opus 4.6 (1M context) --- modules/local/diann/insilico_library_generation/main.nf | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index 66bca5e..2e3b456 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -79,7 +79,11 @@ process INSILICO_LIBRARY_GENERATION { ${diann_dda_flag} \\ ${args} - cp *lib.log.txt silicolibrarygeneration.log + if ls *.log.txt 1>/dev/null 2>&1; then + cp *.log.txt silicolibrarygeneration.log + else + echo "No DIA-NN log file produced (expected with DIA-NN >= 2.x)" > silicolibrarygeneration.log + fi cat <<-END_VERSIONS > versions.yml "${task.process}": From d4a624355a71475c2149067e886f7ee5c617adf6 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Thu, 9 Apr 2026 09:51:35 +0100 Subject: [PATCH 30/69] fix: handle missing log file in insilico library generation for DIA-NN >= 2.x DIA-NN 2.3.2 no longer produces a *lib.log.txt file during in-silico library generation. Instead of using a shell cp workaround, use the proper Nextflow approach: glob pattern with optional: true in the output declaration. Co-Authored-By: Claude Opus 4.6 (1M context) --- modules/local/diann/insilico_library_generation/main.nf | 8 +------- modules/local/diann/insilico_library_generation/meta.yml | 4 ++-- 2 files changed, 3 insertions(+), 9 deletions(-) diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index 2e3b456..622c32a 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -17,7 +17,7 @@ process INSILICO_LIBRARY_GENERATION { path "versions.yml", emit: versions path "*.predicted.speclib", emit: predict_speclib path "*.tsv", emit: speclib_tsv, optional: true - path "silicolibrarygeneration.log", emit: log + path "*.log.txt", emit: log, optional: true when: task.ext.when == null || task.ext.when @@ -79,12 +79,6 @@ process INSILICO_LIBRARY_GENERATION { ${diann_dda_flag} \\ ${args} - if ls *.log.txt 1>/dev/null 2>&1; then - cp *.log.txt silicolibrarygeneration.log - else - echo "No DIA-NN log file produced (expected with DIA-NN >= 2.x)" > silicolibrarygeneration.log - fi - cat <<-END_VERSIONS > versions.yml "${task.process}": DIA-NN: \$(diann 2>&1 | grep "DIA-NN" | grep -oP "\\d+\\.\\d+(\\.\\w+)*(\\.[\\d]+)?") diff --git a/modules/local/diann/insilico_library_generation/meta.yml b/modules/local/diann/insilico_library_generation/meta.yml index a6185d7..e00bae2 100644 --- a/modules/local/diann/insilico_library_generation/meta.yml +++ b/modules/local/diann/insilico_library_generation/meta.yml @@ -29,8 +29,8 @@ output: pattern: "*.predicted.speclib" - log: type: file - description: DIA-NN log file - pattern: "silicolibrarygeneration.log" + description: DIA-NN log file (optional, not produced by DIA-NN >= 2.x) + pattern: "*.log.txt" - versions: type: file description: File containing software version From d46b4617c68d8223e1f2f8a15ecaa09823bb2c8e Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Thu, 9 Apr 2026 10:03:43 +0100 Subject: [PATCH 31/69] fix: remove cp log workarounds across all DIA-NN modules Replace shell `cp` log file renaming with native Nextflow output declarations using the actual DIA-NN filenames and optional: true. This handles DIA-NN >= 2.x which may not produce log files. Modules updated: - preliminary_analysis: report.log.txt (optional) - assemble_empirical_library: report.log.txt (optional) - final_quantification: diann_report.log.txt (optional) - individual_analysis: report.log.txt (optional) The downstream log parser in dia.nf now uses .ifEmpty() to fall back to user params when no log file is produced. Co-Authored-By: Claude Opus 4.6 (1M context) --- modules/local/diann/assemble_empirical_library/main.nf | 4 +--- modules/local/diann/assemble_empirical_library/meta.yml | 4 ++-- modules/local/diann/final_quantification/main.nf | 4 +--- modules/local/diann/final_quantification/meta.yml | 4 ++-- modules/local/diann/individual_analysis/main.nf | 4 +--- modules/local/diann/individual_analysis/meta.yml | 4 ++-- modules/local/diann/preliminary_analysis/main.nf | 4 +--- modules/local/diann/preliminary_analysis/meta.yml | 4 ++-- workflows/dia.nf | 2 ++ 9 files changed, 14 insertions(+), 20 deletions(-) diff --git a/modules/local/diann/assemble_empirical_library/main.nf b/modules/local/diann/assemble_empirical_library/main.nf index 2bfb67e..dddbef4 100644 --- a/modules/local/diann/assemble_empirical_library/main.nf +++ b/modules/local/diann/assemble_empirical_library/main.nf @@ -18,7 +18,7 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { output: path "empirical_library.*", emit: empirical_library - path "assemble_empirical_library.log", emit: log + path "report.log.txt", emit: log, optional: true path "versions.yml", emit: versions when: @@ -85,8 +85,6 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { \${mod_flags} \\ $args - cp report.log.txt assemble_empirical_library.log - cat <<-END_VERSIONS > versions.yml "${task.process}": DIA-NN: \$(diann 2>&1 | grep "DIA-NN" | grep -oP "\\d+\\.\\d+(\\.\\w+)*(\\.[\\d]+)?") diff --git a/modules/local/diann/assemble_empirical_library/meta.yml b/modules/local/diann/assemble_empirical_library/meta.yml index 7fa6585..6de6590 100644 --- a/modules/local/diann/assemble_empirical_library/meta.yml +++ b/modules/local/diann/assemble_empirical_library/meta.yml @@ -29,8 +29,8 @@ output: pattern: "empirical_library.tsv" - log: type: file - description: DIA-NN log file - pattern: "assemble_empirical_library.log" + description: DIA-NN log file (optional, not produced by DIA-NN >= 2.x) + pattern: "report.log.txt" - versions: type: file description: File containing software version diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index f83af13..9a53a2a 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -30,7 +30,7 @@ process FINAL_QUANTIFICATION { path "diann_report.pg_matrix.tsv", emit: pg_matrix, optional: true path "diann_report.gg_matrix.tsv", emit: gg_matrix, optional: true path "diann_report.unique_genes_matrix.tsv", emit: unique_gene_matrix, optional: true - path "diannsummary.log", emit: log + path "diannsummary.log", emit: log, optional: true // Different library files format are exported due to different DIA-NN versions path "empirical_library.tsv", emit: final_speclib, optional: true @@ -114,8 +114,6 @@ process FINAL_QUANTIFICATION { \${mod_flags} \\ $args - cp diann_report.log.txt diannsummary.log - cat <<-END_VERSIONS > versions.yml "${task.process}": DIA-NN: \$(diann 2>&1 | grep "DIA-NN" | grep -oP "\\d+\\.\\d+(\\.\\w+)*(\\.[\\d]+)?") diff --git a/modules/local/diann/final_quantification/meta.yml b/modules/local/diann/final_quantification/meta.yml index 23f64c7..0c9742b 100644 --- a/modules/local/diann/final_quantification/meta.yml +++ b/modules/local/diann/final_quantification/meta.yml @@ -45,8 +45,8 @@ output: pattern: "*.tsv" - log: type: file - description: DIA-NN log file - pattern: "diannsummary.log" + description: DIA-NN log file (optional, not produced by DIA-NN >= 2.x) + pattern: "diann_report.log.txt" - versions: type: file description: File containing software version diff --git a/modules/local/diann/individual_analysis/main.nf b/modules/local/diann/individual_analysis/main.nf index 975ceaf..a1100c6 100644 --- a/modules/local/diann/individual_analysis/main.nf +++ b/modules/local/diann/individual_analysis/main.nf @@ -14,7 +14,7 @@ process INDIVIDUAL_ANALYSIS { output: path "*.quant", emit: diann_quant - path "*_final_diann.log", emit: log + path "report.log.txt", emit: log, optional: true path "versions.yml", emit: versions when: @@ -123,8 +123,6 @@ process INDIVIDUAL_ANALYSIS { \${mod_flags} \\ $args - cp report.log.txt ${ms_file.baseName}_final_diann.log - cat <<-END_VERSIONS > versions.yml "${task.process}": DIA-NN: \$(diann 2>&1 | grep "DIA-NN" | grep -oP "\\d+\\.\\d+(\\.\\w+)*(\\.[\\d]+)?") diff --git a/modules/local/diann/individual_analysis/meta.yml b/modules/local/diann/individual_analysis/meta.yml index 8c0c677..164ae16 100644 --- a/modules/local/diann/individual_analysis/meta.yml +++ b/modules/local/diann/individual_analysis/meta.yml @@ -33,8 +33,8 @@ output: pattern: "*.tsv" - log: type: file - description: DIA-NN log file - pattern: "*_diann.log" + description: DIA-NN log file (optional, not produced by DIA-NN >= 2.x) + pattern: "report.log.txt" - versions: type: file description: File containing software version diff --git a/modules/local/diann/preliminary_analysis/main.nf b/modules/local/diann/preliminary_analysis/main.nf index f9a3508..058ab52 100644 --- a/modules/local/diann/preliminary_analysis/main.nf +++ b/modules/local/diann/preliminary_analysis/main.nf @@ -14,7 +14,7 @@ process PRELIMINARY_ANALYSIS { output: path "*.quant", emit: diann_quant - tuple val(meta), path("*_diann.log"), emit: log + tuple val(meta), path("report.log.txt"), emit: log, optional: true path "versions.yml", emit: versions when: @@ -108,8 +108,6 @@ process PRELIMINARY_ANALYSIS { \${mod_flags} \\ $args - cp report.log.txt ${ms_file.baseName}_diann.log - cat <<-END_VERSIONS > versions.yml "${task.process}": DIA-NN: \$(diann 2>&1 | grep "DIA-NN" | grep -oP "\\d+\\.\\d+(\\.\\w+)*(\\.[\\d]+)?") diff --git a/modules/local/diann/preliminary_analysis/meta.yml b/modules/local/diann/preliminary_analysis/meta.yml index 5eb752a..8e421a3 100644 --- a/modules/local/diann/preliminary_analysis/meta.yml +++ b/modules/local/diann/preliminary_analysis/meta.yml @@ -28,8 +28,8 @@ output: pattern: "*.quant" - log: type: file - description: DIA-NN log file - pattern: "*_diann.log" + description: DIA-NN log file (optional, not produced by DIA-NN >= 2.x) + pattern: "report.log.txt" - version: type: file description: File containing software version diff --git a/workflows/dia.nf b/workflows/dia.nf index 69dd78d..39d6d09 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -144,6 +144,7 @@ workflow DIA { ch_software_versions = ch_software_versions .mix(ASSEMBLE_EMPIRICAL_LIBRARY.out.versions) // Parse calibrated params from the assembly log on the head node + // The log file is optional (not produced by DIA-NN >= 2.x); fall back to user params ch_parsed_vals = ASSEMBLE_EMPIRICAL_LIBRARY.out.log .map { log_file -> def match = log_file.text.readLines().find { it.contains("Averaged recommended settings") } @@ -156,6 +157,7 @@ workflow DIA { } return "${params.mass_acc_ms2},${params.mass_acc_ms1},${params.scan_window}" } + .ifEmpty("${params.mass_acc_ms2},${params.mass_acc_ms1},${params.scan_window}") indiv_fin_analysis_in = ch_file_preparation_results .combine(ch_searchdb) .combine(ASSEMBLE_EMPIRICAL_LIBRARY.out.empirical_library) From ce4497979ec644023d718406f56214aacaae80bb Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Thu, 9 Apr 2026 10:07:56 +0100 Subject: [PATCH 32/69] fix: keep meaningful log names with guarded cp for DIA-NN >= 2.x Restore the cp-based renaming (e.g., report.log.txt -> *_diann.log) to preserve sample-specific log names, but guard each cp with `if [ -f ... ]` so it doesn't fail when DIA-NN >= 2.x doesn't produce the log file. All log outputs are now marked optional: true. Co-Authored-By: Claude Opus 4.6 (1M context) --- modules/local/diann/assemble_empirical_library/main.nf | 6 +++++- modules/local/diann/assemble_empirical_library/meta.yml | 2 +- modules/local/diann/final_quantification/main.nf | 4 ++++ modules/local/diann/final_quantification/meta.yml | 2 +- modules/local/diann/individual_analysis/main.nf | 6 +++++- modules/local/diann/individual_analysis/meta.yml | 2 +- modules/local/diann/preliminary_analysis/main.nf | 6 +++++- modules/local/diann/preliminary_analysis/meta.yml | 2 +- 8 files changed, 23 insertions(+), 7 deletions(-) diff --git a/modules/local/diann/assemble_empirical_library/main.nf b/modules/local/diann/assemble_empirical_library/main.nf index dddbef4..d39f107 100644 --- a/modules/local/diann/assemble_empirical_library/main.nf +++ b/modules/local/diann/assemble_empirical_library/main.nf @@ -18,7 +18,7 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { output: path "empirical_library.*", emit: empirical_library - path "report.log.txt", emit: log, optional: true + path "assemble_empirical_library.log", emit: log, optional: true path "versions.yml", emit: versions when: @@ -85,6 +85,10 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { \${mod_flags} \\ $args + if [ -f report.log.txt ]; then + cp report.log.txt assemble_empirical_library.log + fi + cat <<-END_VERSIONS > versions.yml "${task.process}": DIA-NN: \$(diann 2>&1 | grep "DIA-NN" | grep -oP "\\d+\\.\\d+(\\.\\w+)*(\\.[\\d]+)?") diff --git a/modules/local/diann/assemble_empirical_library/meta.yml b/modules/local/diann/assemble_empirical_library/meta.yml index 6de6590..93c16b3 100644 --- a/modules/local/diann/assemble_empirical_library/meta.yml +++ b/modules/local/diann/assemble_empirical_library/meta.yml @@ -30,7 +30,7 @@ output: - log: type: file description: DIA-NN log file (optional, not produced by DIA-NN >= 2.x) - pattern: "report.log.txt" + pattern: "assemble_empirical_library.log" - versions: type: file description: File containing software version diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index 9a53a2a..9a721a9 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -114,6 +114,10 @@ process FINAL_QUANTIFICATION { \${mod_flags} \\ $args + if [ -f diann_report.log.txt ]; then + cp diann_report.log.txt diannsummary.log + fi + cat <<-END_VERSIONS > versions.yml "${task.process}": DIA-NN: \$(diann 2>&1 | grep "DIA-NN" | grep -oP "\\d+\\.\\d+(\\.\\w+)*(\\.[\\d]+)?") diff --git a/modules/local/diann/final_quantification/meta.yml b/modules/local/diann/final_quantification/meta.yml index 0c9742b..fa07309 100644 --- a/modules/local/diann/final_quantification/meta.yml +++ b/modules/local/diann/final_quantification/meta.yml @@ -46,7 +46,7 @@ output: - log: type: file description: DIA-NN log file (optional, not produced by DIA-NN >= 2.x) - pattern: "diann_report.log.txt" + pattern: "diannsummary.log" - versions: type: file description: File containing software version diff --git a/modules/local/diann/individual_analysis/main.nf b/modules/local/diann/individual_analysis/main.nf index a1100c6..4ca6731 100644 --- a/modules/local/diann/individual_analysis/main.nf +++ b/modules/local/diann/individual_analysis/main.nf @@ -14,7 +14,7 @@ process INDIVIDUAL_ANALYSIS { output: path "*.quant", emit: diann_quant - path "report.log.txt", emit: log, optional: true + path "*_final_diann.log", emit: log, optional: true path "versions.yml", emit: versions when: @@ -123,6 +123,10 @@ process INDIVIDUAL_ANALYSIS { \${mod_flags} \\ $args + if [ -f report.log.txt ]; then + cp report.log.txt ${ms_file.baseName}_final_diann.log + fi + cat <<-END_VERSIONS > versions.yml "${task.process}": DIA-NN: \$(diann 2>&1 | grep "DIA-NN" | grep -oP "\\d+\\.\\d+(\\.\\w+)*(\\.[\\d]+)?") diff --git a/modules/local/diann/individual_analysis/meta.yml b/modules/local/diann/individual_analysis/meta.yml index 164ae16..e52cbb2 100644 --- a/modules/local/diann/individual_analysis/meta.yml +++ b/modules/local/diann/individual_analysis/meta.yml @@ -34,7 +34,7 @@ output: - log: type: file description: DIA-NN log file (optional, not produced by DIA-NN >= 2.x) - pattern: "report.log.txt" + pattern: "*_final_diann.log" - versions: type: file description: File containing software version diff --git a/modules/local/diann/preliminary_analysis/main.nf b/modules/local/diann/preliminary_analysis/main.nf index 058ab52..c60db52 100644 --- a/modules/local/diann/preliminary_analysis/main.nf +++ b/modules/local/diann/preliminary_analysis/main.nf @@ -14,7 +14,7 @@ process PRELIMINARY_ANALYSIS { output: path "*.quant", emit: diann_quant - tuple val(meta), path("report.log.txt"), emit: log, optional: true + tuple val(meta), path("*_diann.log"), emit: log, optional: true path "versions.yml", emit: versions when: @@ -108,6 +108,10 @@ process PRELIMINARY_ANALYSIS { \${mod_flags} \\ $args + if [ -f report.log.txt ]; then + cp report.log.txt ${ms_file.baseName}_diann.log + fi + cat <<-END_VERSIONS > versions.yml "${task.process}": DIA-NN: \$(diann 2>&1 | grep "DIA-NN" | grep -oP "\\d+\\.\\d+(\\.\\w+)*(\\.[\\d]+)?") diff --git a/modules/local/diann/preliminary_analysis/meta.yml b/modules/local/diann/preliminary_analysis/meta.yml index 8e421a3..db0685d 100644 --- a/modules/local/diann/preliminary_analysis/meta.yml +++ b/modules/local/diann/preliminary_analysis/meta.yml @@ -29,7 +29,7 @@ output: - log: type: file description: DIA-NN log file (optional, not produced by DIA-NN >= 2.x) - pattern: "report.log.txt" + pattern: "*_diann.log" - version: type: file description: File containing software version From 02a3efc5e52152c6fe2bf1127f627222a8197d5d Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Thu, 9 Apr 2026 10:11:51 +0100 Subject: [PATCH 33/69] fix: add --out to insilico library generation to produce log file DIA-NN names the log file based on the --out parameter (e.g., --out report.tsv -> report.log.txt). The in-silico library generation step was missing --out entirely, so DIA-NN 2.3.2 didn't produce any log file. Adding --out silicolibrarygeneration.tsv tells DIA-NN to write silicolibrarygeneration.log.txt. Still marked optional: true in case some DIA-NN versions skip the log for library-only runs. Co-Authored-By: Claude Opus 4.6 (1M context) --- modules/local/diann/insilico_library_generation/main.nf | 3 ++- modules/local/diann/insilico_library_generation/meta.yml | 4 ++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index 622c32a..e358900 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -17,7 +17,7 @@ process INSILICO_LIBRARY_GENERATION { path "versions.yml", emit: versions path "*.predicted.speclib", emit: predict_speclib path "*.tsv", emit: speclib_tsv, optional: true - path "*.log.txt", emit: log, optional: true + path "silicolibrarygeneration.log.txt", emit: log, optional: true when: task.ext.when == null || task.ext.when @@ -57,6 +57,7 @@ process INSILICO_LIBRARY_GENERATION { diann `cat ${diann_config}` \\ --fasta ${fasta} \\ --fasta-search \\ + --out silicolibrarygeneration.tsv \\ ${min_pr_mz} \\ ${max_pr_mz} \\ ${min_fr_mz} \\ diff --git a/modules/local/diann/insilico_library_generation/meta.yml b/modules/local/diann/insilico_library_generation/meta.yml index e00bae2..d6aa49d 100644 --- a/modules/local/diann/insilico_library_generation/meta.yml +++ b/modules/local/diann/insilico_library_generation/meta.yml @@ -29,8 +29,8 @@ output: pattern: "*.predicted.speclib" - log: type: file - description: DIA-NN log file (optional, not produced by DIA-NN >= 2.x) - pattern: "*.log.txt" + description: DIA-NN log file (optional, may not be produced by some DIA-NN versions) + pattern: "silicolibrarygeneration.log.txt" - versions: type: file description: File containing software version From a74b20ed0635a58e1b3a844f1dfe419d66415e03 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Thu, 9 Apr 2026 10:27:26 +0100 Subject: [PATCH 34/69] fix: use tee as fallback log capture, prefer native DIA-NN log All DIA-NN modules now capture stdout/stderr via tee to guarantee a log file exists regardless of DIA-NN version. If DIA-NN also produces its native .log.txt file, it overwrites the tee output (native log may contain additional structured information). This two-layer approach ensures: - Log files are always produced (tee fallback) - Native DIA-NN logs are preferred when available (cp override) - Meaningful per-step names are preserved - No optional: true needed on log outputs - .ifEmpty() workaround in dia.nf no longer needed Co-Authored-By: Claude Opus 4.6 (1M context) --- modules/local/diann/assemble_empirical_library/main.nf | 5 +++-- modules/local/diann/assemble_empirical_library/meta.yml | 2 +- modules/local/diann/final_quantification/main.nf | 5 +++-- modules/local/diann/final_quantification/meta.yml | 2 +- modules/local/diann/individual_analysis/main.nf | 5 +++-- modules/local/diann/individual_analysis/meta.yml | 2 +- modules/local/diann/insilico_library_generation/main.nf | 9 +++++++-- modules/local/diann/insilico_library_generation/meta.yml | 4 ++-- modules/local/diann/preliminary_analysis/main.nf | 5 +++-- modules/local/diann/preliminary_analysis/meta.yml | 2 +- workflows/dia.nf | 2 -- 11 files changed, 25 insertions(+), 18 deletions(-) diff --git a/modules/local/diann/assemble_empirical_library/main.nf b/modules/local/diann/assemble_empirical_library/main.nf index d39f107..b11b257 100644 --- a/modules/local/diann/assemble_empirical_library/main.nf +++ b/modules/local/diann/assemble_empirical_library/main.nf @@ -18,7 +18,7 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { output: path "empirical_library.*", emit: empirical_library - path "assemble_empirical_library.log", emit: log, optional: true + path "assemble_empirical_library.log", emit: log path "versions.yml", emit: versions when: @@ -83,7 +83,8 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { ${diann_im_window} \\ ${diann_dda_flag} \\ \${mod_flags} \\ - $args + $args \\ + 2>&1 | tee assemble_empirical_library.log if [ -f report.log.txt ]; then cp report.log.txt assemble_empirical_library.log diff --git a/modules/local/diann/assemble_empirical_library/meta.yml b/modules/local/diann/assemble_empirical_library/meta.yml index 93c16b3..4355482 100644 --- a/modules/local/diann/assemble_empirical_library/meta.yml +++ b/modules/local/diann/assemble_empirical_library/meta.yml @@ -29,7 +29,7 @@ output: pattern: "empirical_library.tsv" - log: type: file - description: DIA-NN log file (optional, not produced by DIA-NN >= 2.x) + description: DIA-NN log file (captured via tee, overwritten with native log if available) pattern: "assemble_empirical_library.log" - versions: type: file diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index 9a721a9..e2d9195 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -30,7 +30,7 @@ process FINAL_QUANTIFICATION { path "diann_report.pg_matrix.tsv", emit: pg_matrix, optional: true path "diann_report.gg_matrix.tsv", emit: gg_matrix, optional: true path "diann_report.unique_genes_matrix.tsv", emit: unique_gene_matrix, optional: true - path "diannsummary.log", emit: log, optional: true + path "diannsummary.log", emit: log // Different library files format are exported due to different DIA-NN versions path "empirical_library.tsv", emit: final_speclib, optional: true @@ -112,7 +112,8 @@ process FINAL_QUANTIFICATION { ${diann_channel_run_norm} \\ ${diann_channel_spec_norm} \\ \${mod_flags} \\ - $args + $args \\ + 2>&1 | tee diannsummary.log if [ -f diann_report.log.txt ]; then cp diann_report.log.txt diannsummary.log diff --git a/modules/local/diann/final_quantification/meta.yml b/modules/local/diann/final_quantification/meta.yml index fa07309..c220202 100644 --- a/modules/local/diann/final_quantification/meta.yml +++ b/modules/local/diann/final_quantification/meta.yml @@ -45,7 +45,7 @@ output: pattern: "*.tsv" - log: type: file - description: DIA-NN log file (optional, not produced by DIA-NN >= 2.x) + description: DIA-NN log file (captured via tee, overwritten with native log if available) pattern: "diannsummary.log" - versions: type: file diff --git a/modules/local/diann/individual_analysis/main.nf b/modules/local/diann/individual_analysis/main.nf index 4ca6731..11ca424 100644 --- a/modules/local/diann/individual_analysis/main.nf +++ b/modules/local/diann/individual_analysis/main.nf @@ -14,7 +14,7 @@ process INDIVIDUAL_ANALYSIS { output: path "*.quant", emit: diann_quant - path "*_final_diann.log", emit: log, optional: true + path "*_final_diann.log", emit: log path "versions.yml", emit: versions when: @@ -121,7 +121,8 @@ process INDIVIDUAL_ANALYSIS { ${diann_im_window} \\ ${diann_dda_flag} \\ \${mod_flags} \\ - $args + $args \\ + 2>&1 | tee ${ms_file.baseName}_final_diann.log if [ -f report.log.txt ]; then cp report.log.txt ${ms_file.baseName}_final_diann.log diff --git a/modules/local/diann/individual_analysis/meta.yml b/modules/local/diann/individual_analysis/meta.yml index e52cbb2..8550821 100644 --- a/modules/local/diann/individual_analysis/meta.yml +++ b/modules/local/diann/individual_analysis/meta.yml @@ -33,7 +33,7 @@ output: pattern: "*.tsv" - log: type: file - description: DIA-NN log file (optional, not produced by DIA-NN >= 2.x) + description: DIA-NN log file (captured via tee, overwritten with native log if available) pattern: "*_final_diann.log" - versions: type: file diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index e358900..4665ea3 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -17,7 +17,7 @@ process INSILICO_LIBRARY_GENERATION { path "versions.yml", emit: versions path "*.predicted.speclib", emit: predict_speclib path "*.tsv", emit: speclib_tsv, optional: true - path "silicolibrarygeneration.log.txt", emit: log, optional: true + path "silicolibrarygeneration.log", emit: log when: task.ext.when == null || task.ext.when @@ -78,7 +78,12 @@ process INSILICO_LIBRARY_GENERATION { ${pre_select_flag} \\ ${met_excision} \\ ${diann_dda_flag} \\ - ${args} + ${args} \\ + 2>&1 | tee silicolibrarygeneration.log + + if [ -f silicolibrarygeneration.log.txt ]; then + cp silicolibrarygeneration.log.txt silicolibrarygeneration.log + fi cat <<-END_VERSIONS > versions.yml "${task.process}": diff --git a/modules/local/diann/insilico_library_generation/meta.yml b/modules/local/diann/insilico_library_generation/meta.yml index d6aa49d..27fee04 100644 --- a/modules/local/diann/insilico_library_generation/meta.yml +++ b/modules/local/diann/insilico_library_generation/meta.yml @@ -29,8 +29,8 @@ output: pattern: "*.predicted.speclib" - log: type: file - description: DIA-NN log file (optional, may not be produced by some DIA-NN versions) - pattern: "silicolibrarygeneration.log.txt" + description: DIA-NN log file (captured via tee, overwritten with native log if available) + pattern: "silicolibrarygeneration.log" - versions: type: file description: File containing software version diff --git a/modules/local/diann/preliminary_analysis/main.nf b/modules/local/diann/preliminary_analysis/main.nf index c60db52..a51a4d8 100644 --- a/modules/local/diann/preliminary_analysis/main.nf +++ b/modules/local/diann/preliminary_analysis/main.nf @@ -14,7 +14,7 @@ process PRELIMINARY_ANALYSIS { output: path "*.quant", emit: diann_quant - tuple val(meta), path("*_diann.log"), emit: log, optional: true + tuple val(meta), path("*_diann.log"), emit: log path "versions.yml", emit: versions when: @@ -106,7 +106,8 @@ process PRELIMINARY_ANALYSIS { --no-prot-inf \\ ${diann_dda_flag} \\ \${mod_flags} \\ - $args + $args \\ + 2>&1 | tee ${ms_file.baseName}_diann.log if [ -f report.log.txt ]; then cp report.log.txt ${ms_file.baseName}_diann.log diff --git a/modules/local/diann/preliminary_analysis/meta.yml b/modules/local/diann/preliminary_analysis/meta.yml index db0685d..5185c32 100644 --- a/modules/local/diann/preliminary_analysis/meta.yml +++ b/modules/local/diann/preliminary_analysis/meta.yml @@ -28,7 +28,7 @@ output: pattern: "*.quant" - log: type: file - description: DIA-NN log file (optional, not produced by DIA-NN >= 2.x) + description: DIA-NN log file (captured via tee, overwritten with native log if available) pattern: "*_diann.log" - version: type: file diff --git a/workflows/dia.nf b/workflows/dia.nf index 39d6d09..69dd78d 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -144,7 +144,6 @@ workflow DIA { ch_software_versions = ch_software_versions .mix(ASSEMBLE_EMPIRICAL_LIBRARY.out.versions) // Parse calibrated params from the assembly log on the head node - // The log file is optional (not produced by DIA-NN >= 2.x); fall back to user params ch_parsed_vals = ASSEMBLE_EMPIRICAL_LIBRARY.out.log .map { log_file -> def match = log_file.text.readLines().find { it.contains("Averaged recommended settings") } @@ -157,7 +156,6 @@ workflow DIA { } return "${params.mass_acc_ms2},${params.mass_acc_ms1},${params.scan_window}" } - .ifEmpty("${params.mass_acc_ms2},${params.mass_acc_ms1},${params.scan_window}") indiv_fin_analysis_in = ch_file_preparation_results .combine(ch_searchdb) .combine(ASSEMBLE_EMPIRICAL_LIBRARY.out.empirical_library) From f932b77b1b5484af2c8e77ac8b00d41b943e7315 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Thu, 9 Apr 2026 11:02:47 +0100 Subject: [PATCH 35/69] fix: remove unnecessary .first() on ch_sdrf value channel Nextflow warns that .first() is useless on a value channel. The ch_sdrf channel already contains a single value and is multicast in DSL2, so .first() was redundant. Removed it and map directly. Co-Authored-By: Claude Opus 4.6 (1M context) --- subworkflows/local/create_input_channel/main.nf | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/subworkflows/local/create_input_channel/main.nf b/subworkflows/local/create_input_channel/main.nf index 2208eca..9a819a3 100644 --- a/subworkflows/local/create_input_channel/main.nf +++ b/subworkflows/local/create_input_channel/main.nf @@ -21,9 +21,8 @@ workflow CREATE_INPUT_CHANNEL { def enzymes = new HashSet() def files = new HashSet() - // Extract experiment_id from the SDRF filename via the value channel - ch_sdrf_val = ch_sdrf.first() - ch_experiment_id = ch_sdrf_val.map { sdrf_file -> file(sdrf_file).baseName } + // Extract experiment_id from the SDRF filename + ch_experiment_id = ch_sdrf.map { sdrf_file -> file(sdrf_file).baseName } ch_experiment_id .combine(ch_expdesign) From c6dab976369f48ecb7bfba0c3c03ab9597b75760 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Thu, 9 Apr 2026 11:45:09 +0100 Subject: [PATCH 36/69] fix: handle multiple report files in DIANN_MSSTATS input FINAL_QUANTIFICATION emits both diann_report.parquet and diann_report.tsv when both exist (DIA-NN 2.x). The --report flag only accepts one file. Now selects parquet preferentially over TSV when both are staged. Co-Authored-By: Claude Opus 4.6 (1M context) --- modules/local/diann/diann_msstats/main.nf | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/modules/local/diann/diann_msstats/main.nf b/modules/local/diann/diann_msstats/main.nf index 1844fdc..708809a 100644 --- a/modules/local/diann/diann_msstats/main.nf +++ b/modules/local/diann/diann_msstats/main.nf @@ -17,10 +17,12 @@ process DIANN_MSSTATS { script: def args = task.ext.args ?: '' + // When both parquet and tsv are staged, prefer parquet (DIA-NN >= 2.x output) + def report_file = report instanceof List ? report.find { it.name.endsWith('.parquet') } ?: report[0] : report """ set -o pipefail quantmsutilsc diann2msstats \\ - --report ${report} \\ + --report ${report_file} \\ --exp_design ${exp_design} \\ --qvalue_threshold $params.protein_level_fdr_cutoff \\ $args \\ From 3fc0717fe9537f530d4abc45e118be3feca9bfbe Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Thu, 9 Apr 2026 11:51:52 +0100 Subject: [PATCH 37/69] fix: split FINAL_QUANTIFICATION report output by format DIA-NN < 1.9 produces TSV, >= 1.9 produces parquet (ignoring the .tsv extension in --out). The previous glob {tsv,parquet} matched both files when intermediate versions produced both, breaking DIANN_MSSTATS which only accepts one file. Now emits separate channels (main_report_parquet, main_report_tsv) each marked optional, merged via .mix() in the workflow so only the version-appropriate format reaches downstream consumers. Co-Authored-By: Claude Opus 4.6 (1M context) --- modules/local/diann/diann_msstats/main.nf | 4 +--- modules/local/diann/final_quantification/main.nf | 6 ++++-- modules/local/diann/final_quantification/meta.yml | 10 +++++++--- workflows/dia.nf | 4 +++- 4 files changed, 15 insertions(+), 9 deletions(-) diff --git a/modules/local/diann/diann_msstats/main.nf b/modules/local/diann/diann_msstats/main.nf index 708809a..1844fdc 100644 --- a/modules/local/diann/diann_msstats/main.nf +++ b/modules/local/diann/diann_msstats/main.nf @@ -17,12 +17,10 @@ process DIANN_MSSTATS { script: def args = task.ext.args ?: '' - // When both parquet and tsv are staged, prefer parquet (DIA-NN >= 2.x output) - def report_file = report instanceof List ? report.find { it.name.endsWith('.parquet') } ?: report[0] : report """ set -o pipefail quantmsutilsc diann2msstats \\ - --report ${report_file} \\ + --report ${report} \\ --exp_design ${exp_design} \\ --qvalue_threshold $params.protein_level_fdr_cutoff \\ $args \\ diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index e2d9195..3eba85f 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -21,8 +21,10 @@ process FINAL_QUANTIFICATION { path(diann_config) output: - // DIA-NN 2.0 don't return report in tsv format - path "diann_report.{tsv,parquet}", emit: main_report, optional: true + // DIA-NN < 1.9 produces TSV; >= 1.9 produces parquet (ignores .tsv in --out) + // Both are optional since only one format will be produced per version + path "diann_report.parquet", emit: main_report_parquet, optional: true + path "diann_report.tsv", emit: main_report_tsv, optional: true path "diann_report.manifest.txt", emit: report_manifest, optional: true path "diann_report.protein_description.tsv", emit: protein_description, optional: true path "diann_report.stats.tsv", emit: report_stats, optional: true diff --git a/modules/local/diann/final_quantification/meta.yml b/modules/local/diann/final_quantification/meta.yml index c220202..9db0e12 100644 --- a/modules/local/diann/final_quantification/meta.yml +++ b/modules/local/diann/final_quantification/meta.yml @@ -27,10 +27,14 @@ input: description: Identification and Quantification file from DIA-NN. pattern: "*.quant" output: - - main_report: + - main_report_parquet: type: file - description: A text table containing precursor and protein IDs, as well as plenty of associated information. - pattern: "*.tsv" + description: DIA-NN main report in parquet format (DIA-NN >= 1.9) + pattern: "diann_report.parquet" + - main_report_tsv: + type: file + description: DIA-NN main report in TSV format (DIA-NN < 1.9) + pattern: "diann_report.tsv" - pr_matrix: type: file description: A text table containing normalized quantities for precursors. They are filtered at 1% FDR, using both global and run-specific q-values for precursors diff --git a/workflows/dia.nf b/workflows/dia.nf index 69dd78d..77ce8c3 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -204,7 +204,9 @@ workflow DIA { FINAL_QUANTIFICATION.out.versions ) - diann_main_report = FINAL_QUANTIFICATION.out.main_report + // Only one format is produced per DIA-NN version: parquet (>= 1.9) or TSV (< 1.9) + diann_main_report = FINAL_QUANTIFICATION.out.main_report_parquet + .mix(FINAL_QUANTIFICATION.out.main_report_tsv) // // MODULE: DIANN_MSSTATS — Convert DIA-NN report to MSstats-compatible format From fca00931c49067a2713aacefb918db927cbad6d8 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Thu, 9 Apr 2026 11:57:31 +0100 Subject: [PATCH 38/69] fix: decouple container engine from DIA-NN version configs - Remove hardcoded DIA-NN 1.9.2 container from pride_codon_slurm.config (cluster config should not dictate DIA-NN version) - Remove docker.enabled/singularity.enabled from version configs (container engine is selected via -profile, not version config) Users select DIA-NN version via -profile diann_v2_3_2 (or similar) and container engine via -profile singularity/docker independently. Co-Authored-By: Claude Opus 4.6 (1M context) --- conf/diann_versions/v2_1_0.config | 3 +-- conf/diann_versions/v2_2_0.config | 3 +-- conf/diann_versions/v2_3_2.config | 3 +-- conf/pride_codon_slurm.config | 5 ++--- 4 files changed, 5 insertions(+), 9 deletions(-) diff --git a/conf/diann_versions/v2_1_0.config b/conf/diann_versions/v2_1_0.config index 9915726..9bd5208 100644 --- a/conf/diann_versions/v2_1_0.config +++ b/conf/diann_versions/v2_1_0.config @@ -10,5 +10,4 @@ process { } } -singularity.enabled = false -docker.enabled = true +// Container engine is selected via -profile (docker/singularity), not here diff --git a/conf/diann_versions/v2_2_0.config b/conf/diann_versions/v2_2_0.config index 93ea4ee..e72fcba 100644 --- a/conf/diann_versions/v2_2_0.config +++ b/conf/diann_versions/v2_2_0.config @@ -10,5 +10,4 @@ process { } } -singularity.enabled = false -docker.enabled = true +// Container engine is selected via -profile (docker/singularity), not here diff --git a/conf/diann_versions/v2_3_2.config b/conf/diann_versions/v2_3_2.config index 2912f15..1a1dd97 100644 --- a/conf/diann_versions/v2_3_2.config +++ b/conf/diann_versions/v2_3_2.config @@ -10,5 +10,4 @@ process { } } -singularity.enabled = false -docker.enabled = true +// Container engine is selected via -profile (docker/singularity), not here diff --git a/conf/pride_codon_slurm.config b/conf/pride_codon_slurm.config index 0848be1..e38b8b6 100644 --- a/conf/pride_codon_slurm.config +++ b/conf/pride_codon_slurm.config @@ -42,9 +42,8 @@ process { cpus = {Math.min((ms_files as List).size() < 200 ? 12 * task.attempt : 24 * task.attempt, 48)} } - withLabel: diann { - container = '/hps/nobackup/juan/pride/reanalysis/singularity/ghcr.io-bigbio-diann-1.9.2.sif' - } + // DIA-NN container: use -c conf/diann_versions/v.config or + // --diann_container to select the version. No default hardcoded here. } executor { From e3928ce5516382a682f5ec2a7067b8f647a9626d Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 10 Apr 2026 08:03:14 +0100 Subject: [PATCH 39/69] feat: rename FDR params and expose matrix-level q-value controls Rename protein_level_fdr_cutoff -> precursor_qvalue to accurately reflect that DIA-NN's --qvalue controls precursor-level filtering, not protein-level FDR. Expose two new matrix-level FDR parameters: - matrix_qvalue (default 0.01): maps to --matrix-qvalue - matrix_spec_q (default 0.05): maps to --matrix-spec-q This enables proteogenomics workflows to independently control matrix filtering (e.g., matrix_spec_q=1.0 to retain variant proteins that lack sufficient unique peptides). Closes #42 Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/parameters.md | 4 +- modules/local/diann/diann_msstats/main.nf | 2 +- .../local/diann/final_quantification/main.nf | 6 ++- nextflow.config | 4 +- nextflow_schema.json | 40 ++++++++++--------- 5 files changed, 32 insertions(+), 24 deletions(-) diff --git a/docs/parameters.md b/docs/parameters.md index f3c76c0..af2fe1f 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -148,7 +148,9 @@ This document lists every pipeline parameter organised by category. Default valu | `--enable_pmultiqc` | boolean | `true` | Generate the pmultiqc QC report. | | `--pmultiqc_idxml_skip` | boolean | `true` | Skip idXML files (do not generate search engine score plots) in the pmultiqc report. | | `--contaminant_string` | string | `CONT` | Contaminant affix string for pmultiqc. Maps to `--contaminant_affix` in pmultiqc. | -| `--protein_level_fdr_cutoff` | number | `0.01` | Experiment-wide protein (group)-level FDR cutoff. | +| `--precursor_qvalue` | number | `0.01` | Precursor-level q-value threshold for the DIA-NN main report. Maps to `--qvalue`. | +| `--matrix_qvalue` | number | `0.01` | Q-value threshold for DIA-NN output matrices (pr_matrix, pg_matrix). Maps to `--matrix-qvalue`. | +| `--matrix_spec_q` | number | `0.05` | Run-specific protein q-value for protein/gene matrices. Maps to `--matrix-spec-q`. | ## 15. MultiQC & Reporting diff --git a/modules/local/diann/diann_msstats/main.nf b/modules/local/diann/diann_msstats/main.nf index 1844fdc..a7b448c 100644 --- a/modules/local/diann/diann_msstats/main.nf +++ b/modules/local/diann/diann_msstats/main.nf @@ -22,7 +22,7 @@ process DIANN_MSSTATS { quantmsutilsc diann2msstats \\ --report ${report} \\ --exp_design ${exp_design} \\ - --qvalue_threshold $params.protein_level_fdr_cutoff \\ + --qvalue_threshold $params.precursor_qvalue \\ $args \\ 2>&1 | tee convert_report.log diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index f83af13..291166d 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -47,7 +47,7 @@ process FINAL_QUANTIFICATION { def blocked = ['--no-main-report', '--gen-spec-lib', '--out-lib', '--no-ifs-removal', '--temp', '--threads', '--verbose', '--lib', '--f', '--fasta', '--use-quant', '--matrices', '--out', '--relaxed-prot-inf', '--pg-level', - '--qvalue', '--window', '--individual-windows', + '--qvalue', '--matrix-qvalue', '--matrix-spec-q', '--window', '--individual-windows', '--species-genes', '--report-decoys', '--xic', '--no-norm', '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', '--export-quant', '--site-ms1-quant', '--channels', '--lib-fixed-mod', '--original-mods'] @@ -97,7 +97,9 @@ process FINAL_QUANTIFICATION { ${no_norm} \\ --matrices \\ --out diann_report.tsv \\ - --qvalue $params.protein_level_fdr_cutoff \\ + --qvalue $params.precursor_qvalue \\ + --matrix-qvalue $params.matrix_qvalue \\ + --matrix-spec-q $params.matrix_spec_q \\ ${report_decoys} \\ ${diann_export_xic} \\ ${quantums} \\ diff --git a/nextflow.config b/nextflow.config index a8fbb0d..d5066b1 100644 --- a/nextflow.config +++ b/nextflow.config @@ -19,7 +19,9 @@ params { use_ols_cache_only = true // Use only the OLS cache for ontology validation (no network requests) // Tools flags - protein_level_fdr_cutoff = 0.01 + precursor_qvalue = 0.01 // --qvalue: precursor-level q-value threshold for main report + matrix_qvalue = 0.01 // --matrix-qvalue: q-value threshold for output matrices + matrix_spec_q = 0.05 // --matrix-spec-q: run-specific protein q-value for protein/gene matrices // Debug level pp_debug = 0 diff --git a/nextflow_schema.json b/nextflow_schema.json index 9fe8f21..d91a8d8 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -248,22 +248,6 @@ }, "fa_icon": "fas fa-star-half-alt" }, - "protein_inference": { - "title": "Protein inference", - "type": "object", - "description": "To group proteins, calculate scores on the protein (group) level and to potentially modify associations from peptides to proteins.", - "default": "", - "properties": { - "protein_level_fdr_cutoff": { - "type": "number", - "description": "The experiment-wide protein (group)-level FDR cutoff. Default: 0.01", - "default": 0.01, - "fa_icon": "fas fa-filter", - "help_text": "This can be protein level if 'strictly_unique_peptides' are used for protein quantification. See [`--protein_quant`](#protein_quant)" - } - }, - "fa_icon": "fab fa-hubspot" - }, "DIA-NN": { "title": "DIA-NN", "type": "object", @@ -337,6 +321,27 @@ "enum": [0, 1, 2], "default": 2 }, + "precursor_qvalue": { + "type": "number", + "description": "Precursor-level q-value filtering threshold for the DIA-NN main report. Maps to --qvalue.", + "default": 0.01, + "fa_icon": "fas fa-filter", + "help_text": "Controls how strictly precursor identifications are filtered in the DIA-NN main report. For proteogenomics with variant databases, the standard 0.01 (1%) is recommended." + }, + "matrix_qvalue": { + "type": "number", + "description": "Q-value threshold for DIA-NN output matrices (pr_matrix, pg_matrix, etc.). Maps to --matrix-qvalue.", + "default": 0.01, + "fa_icon": "fas fa-filter", + "help_text": "Controls the global q-value filtering applied when generating the quantification matrices. Default matches DIA-NN's built-in default of 1%." + }, + "matrix_spec_q": { + "type": "number", + "description": "Run-specific protein q-value filter for protein/gene matrices. Maps to --matrix-spec-q.", + "default": 0.05, + "fa_icon": "fas fa-filter", + "help_text": "An additional run-specific protein-level FDR filter applied to the protein and gene matrices. Default matches DIA-NN's built-in default of 5%. For proteogenomics/variant detection, consider setting to 1.0 to retain variant proteins that lack sufficient unique peptides for protein-level confidence." + }, "species_genes": { "type": "boolean", "description": "Instructs DIA-NN to add the organism identifier to the gene names", @@ -722,9 +727,6 @@ { "$ref": "#/$defs/psm_re_scoring_general" }, - { - "$ref": "#/$defs/protein_inference" - }, { "$ref": "#/$defs/DIA-NN" }, From 9bbcee7e7b3dec230c8100deff266b169c2926ac Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 10 Apr 2026 08:52:17 +0100 Subject: [PATCH 40/69] style: format parameters.md with prettier Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/parameters.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/parameters.md b/docs/parameters.md index af2fe1f..620fa71 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -143,14 +143,14 @@ This document lists every pipeline parameter organised by category. Default valu ## 14. Quality Control -| Parameter | Type | Default | Description | -| ---------------------------- | ------- | ------- | ------------------------------------------------------------------------------------ | -| `--enable_pmultiqc` | boolean | `true` | Generate the pmultiqc QC report. | -| `--pmultiqc_idxml_skip` | boolean | `true` | Skip idXML files (do not generate search engine score plots) in the pmultiqc report. | -| `--contaminant_string` | string | `CONT` | Contaminant affix string for pmultiqc. Maps to `--contaminant_affix` in pmultiqc. | -| `--precursor_qvalue` | number | `0.01` | Precursor-level q-value threshold for the DIA-NN main report. Maps to `--qvalue`. | -| `--matrix_qvalue` | number | `0.01` | Q-value threshold for DIA-NN output matrices (pr_matrix, pg_matrix). Maps to `--matrix-qvalue`. | -| `--matrix_spec_q` | number | `0.05` | Run-specific protein q-value for protein/gene matrices. Maps to `--matrix-spec-q`. | +| Parameter | Type | Default | Description | +| ----------------------- | ------- | ------- | ----------------------------------------------------------------------------------------------- | +| `--enable_pmultiqc` | boolean | `true` | Generate the pmultiqc QC report. | +| `--pmultiqc_idxml_skip` | boolean | `true` | Skip idXML files (do not generate search engine score plots) in the pmultiqc report. | +| `--contaminant_string` | string | `CONT` | Contaminant affix string for pmultiqc. Maps to `--contaminant_affix` in pmultiqc. | +| `--precursor_qvalue` | number | `0.01` | Precursor-level q-value threshold for the DIA-NN main report. Maps to `--qvalue`. | +| `--matrix_qvalue` | number | `0.01` | Q-value threshold for DIA-NN output matrices (pr_matrix, pg_matrix). Maps to `--matrix-qvalue`. | +| `--matrix_spec_q` | number | `0.05` | Run-specific protein q-value for protein/gene matrices. Maps to `--matrix-spec-q`. | ## 15. MultiQC & Reporting From 1dd8d455ce3009a63192666b35abf405c314d12d Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 10 Apr 2026 09:05:26 +0100 Subject: [PATCH 41/69] feat: add DIA-NN scoring mode parameter for proteogenomics support Add diann_scoring_mode parameter with three options: - 'generic' (default): standard scoring, maximizes protein IDs - 'proteoforms': single-residue mutation decoys, recommended for proteogenomics/variant detection (requires DIA-NN >= 2.0) - 'peptidoforms': extra peptidoform q-values for PTM analysis The flag is passed to all 5 DIA-NN steps and added to each module's blocked flags list. Version validation ensures proteoforms mode is only used with DIA-NN >= 2.0. Closes #43 Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/parameters.md | 27 ++++++++++--------- .../diann/assemble_empirical_library/main.nf | 6 ++++- .../local/diann/final_quantification/main.nf | 6 ++++- .../local/diann/individual_analysis/main.nf | 6 ++++- .../diann/insilico_library_generation/main.nf | 5 +++- .../local/diann/preliminary_analysis/main.nf | 6 ++++- nextflow.config | 1 + nextflow_schema.json | 8 ++++++ workflows/dia.nf | 5 ++++ 9 files changed, 52 insertions(+), 18 deletions(-) diff --git a/docs/parameters.md b/docs/parameters.md index f3c76c0..6cb9746 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -107,19 +107,20 @@ This document lists every pipeline parameter organised by category. Default valu ## 11. Quantification & Output -| Parameter | Type | Default | Description | -| ------------------------- | ------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `--pg_level` | integer | `2` | Protein inference mode. `0` = isoforms, `1` = protein names (from FASTA), `2` = genes (default). | -| `--species_genes` | boolean | `false` | Add the organism identifier to gene names in DIA-NN output. | -| `--diann_normalize` | boolean | `true` | Enable cross-run normalisation in DIA-NN. Set to `false` to add `--no-norm`. | -| `--diann_report_decoys` | boolean | `false` | Include decoy PSMs in the main `.parquet` report (DIA-NN 2.0+ only). | -| `--diann_export_xic` | boolean | `false` | Extract MS1/fragment chromatograms for identified precursors (equivalent to the XICs option in the DIA-NN GUI). | -| `--diann_no_peptidoforms` | boolean | `false` | Disable automatic peptidoform scoring when variable modifications are declared (not recommended by DIA-NN). | -| `--diann_use_quant` | boolean | `true` | Reuse existing `.quant` files if available (`--use-quant`). | -| `--quantums` | boolean | `false` | Enable QuantUMS quantification (requires DIA-NN >= 1.9.2). When `false`, the pipeline passes `--direct-quant` to use legacy quantification (only for DIA-NN >= 1.9.2; silently skipped for 1.8.x where direct quant is the only mode). | -| `--quantums_train_runs` | string | `null` | Run index range for QuantUMS training (e.g. `0:5`). Maps to `--quant-train-runs`. Requires DIA-NN >= 1.9.2. | -| `--quantums_sel_runs` | integer | `null` | Number of automatically selected runs for QuantUMS training. Must be >= 6. Maps to `--quant-sel-runs`. Requires DIA-NN >= 1.9.2. | -| `--quantums_params` | string | `null` | Pre-calculated QuantUMS parameters. Maps to `--quant-params`. Requires DIA-NN >= 1.9.2. | +| Parameter | Type | Default | Description | +| ------------------------- | ------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `--pg_level` | integer | `2` | Protein inference mode. `0` = isoforms, `1` = protein names (from FASTA), `2` = genes (default). | +| `--species_genes` | boolean | `false` | Add the organism identifier to gene names in DIA-NN output. | +| `--diann_normalize` | boolean | `true` | Enable cross-run normalisation in DIA-NN. Set to `false` to add `--no-norm`. | +| `--diann_report_decoys` | boolean | `false` | Include decoy PSMs in the main `.parquet` report (DIA-NN 2.0+ only). | +| `--diann_export_xic` | boolean | `false` | Extract MS1/fragment chromatograms for identified precursors (equivalent to the XICs option in the DIA-NN GUI). | +| `--diann_scoring_mode` | string | `generic` | DIA-NN scoring mode: `generic` (default, maximizes IDs), `proteoforms` (recommended for proteogenomics/variant detection, requires >= 2.0), `peptidoforms` (extra q-values for PTM analysis). | +| `--diann_no_peptidoforms` | boolean | `false` | Disable automatic peptidoform scoring when variable modifications are declared (not recommended by DIA-NN). | +| `--diann_use_quant` | boolean | `true` | Reuse existing `.quant` files if available (`--use-quant`). | +| `--quantums` | boolean | `false` | Enable QuantUMS quantification (requires DIA-NN >= 1.9.2). When `false`, the pipeline passes `--direct-quant` to use legacy quantification (only for DIA-NN >= 1.9.2; silently skipped for 1.8.x where direct quant is the only mode). | +| `--quantums_train_runs` | string | `null` | Run index range for QuantUMS training (e.g. `0:5`). Maps to `--quant-train-runs`. Requires DIA-NN >= 1.9.2. | +| `--quantums_sel_runs` | integer | `null` | Number of automatically selected runs for QuantUMS training. Must be >= 6. Maps to `--quant-sel-runs`. Requires DIA-NN >= 1.9.2. | +| `--quantums_params` | string | `null` | Pre-calculated QuantUMS parameters. Maps to `--quant-params`. Requires DIA-NN >= 1.9.2. | ## 12. DDA Mode diff --git a/modules/local/diann/assemble_empirical_library/main.nf b/modules/local/diann/assemble_empirical_library/main.nf index 2bfb67e..c66c005 100644 --- a/modules/local/diann/assemble_empirical_library/main.nf +++ b/modules/local/diann/assemble_empirical_library/main.nf @@ -33,7 +33,8 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { '--individual-mass-acc', '--individual-windows', '--out-lib', '--use-quant', '--gen-spec-lib', '--rt-profiling', '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', - '--channels', '--lib-fixed-mod', '--original-mods'] + '--channels', '--lib-fixed-mod', '--original-mods', + '--proteoforms', '--peptidoforms'] // Sort by length descending so longer flags (e.g. --mass-acc-ms1) are matched before shorter prefixes (--mass-acc) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -52,6 +53,8 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { } scan_window = params.scan_window_automatic ? '--individual-windows' : "--window $params.scan_window" diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" + diann_scoring_mode = params.diann_scoring_mode == 'proteoforms' ? '--proteoforms' : + params.diann_scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' diann_tims_sum = params.diann_tims_sum ? "--quant-tims-sum" : "" diann_im_window = params.diann_im_window ? "--im-window $params.diann_im_window" : "" diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" @@ -79,6 +82,7 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { ${scan_window} \\ --gen-spec-lib \\ ${diann_no_peptidoforms} \\ + ${diann_scoring_mode} \\ ${diann_tims_sum} \\ ${diann_im_window} \\ ${diann_dda_flag} \\ diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index f83af13..5d509da 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -50,7 +50,8 @@ process FINAL_QUANTIFICATION { '--qvalue', '--window', '--individual-windows', '--species-genes', '--report-decoys', '--xic', '--no-norm', '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', '--export-quant', '--site-ms1-quant', - '--channels', '--lib-fixed-mod', '--original-mods'] + '--channels', '--lib-fixed-mod', '--original-mods', + '--proteoforms', '--peptidoforms'] // Sort by length descending so longer flags (e.g. --individual-windows) are matched before shorter prefixes (--window) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -71,6 +72,8 @@ process FINAL_QUANTIFICATION { quantums_sel_runs = params.quantums_sel_runs ? "--quant-sel-runs $params.quantums_sel_runs": "" quantums_params = params.quantums_params ? "--quant-params $params.quantums_params": "" diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" + diann_scoring_mode = params.diann_scoring_mode == 'proteoforms' ? '--proteoforms' : + params.diann_scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' diann_use_quant = params.diann_use_quant ? "--use-quant" : "" diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" diann_export_quant = params.diann_export_quant ? "--export-quant" : "" @@ -105,6 +108,7 @@ process FINAL_QUANTIFICATION { ${quantums_sel_runs} \\ ${quantums_params} \\ ${diann_no_peptidoforms} \\ + ${diann_scoring_mode} \\ ${diann_use_quant} \\ ${diann_dda_flag} \\ ${diann_export_quant} \\ diff --git a/modules/local/diann/individual_analysis/main.nf b/modules/local/diann/individual_analysis/main.nf index 975ceaf..37cfe69 100644 --- a/modules/local/diann/individual_analysis/main.nf +++ b/modules/local/diann/individual_analysis/main.nf @@ -29,7 +29,8 @@ process INDIVIDUAL_ANALYSIS { '--no-ifs-removal', '--no-main-report', '--relaxed-prot-inf', '--pg-level', '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', - '--channels', '--lib-fixed-mod', '--original-mods'] + '--channels', '--lib-fixed-mod', '--original-mods', + '--proteoforms', '--peptidoforms'] // Sort by length descending so longer flags (e.g. --mass-acc-ms1) are matched before shorter prefixes (--mass-acc) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -81,6 +82,8 @@ process INDIVIDUAL_ANALYSIS { } diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" + diann_scoring_mode = params.diann_scoring_mode == 'proteoforms' ? '--proteoforms' : + params.diann_scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' diann_tims_sum = params.diann_tims_sum ? "--quant-tims-sum" : "" diann_im_window = params.diann_im_window ? "--im-window $params.diann_im_window" : "" diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" @@ -117,6 +120,7 @@ process INDIVIDUAL_ANALYSIS { ${min_fr_mz} \\ ${max_fr_mz} \\ ${diann_no_peptidoforms} \\ + ${diann_scoring_mode} \\ ${diann_tims_sum} \\ ${diann_im_window} \\ ${diann_dda_flag} \\ diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index 66bca5e..42b52d8 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -32,7 +32,7 @@ process INSILICO_LIBRARY_GENERATION { '--min-pr-charge', '--max-pr-charge', '--var-mods', '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', '--met-excision', '--monitor-mod', '--dda', '--light-models', - '--infin-dia', '--pre-select'] + '--infin-dia', '--pre-select', '--proteoforms', '--peptidoforms'] // Sort by length descending so longer flags (e.g. --fasta-search) are matched before shorter prefixes (--fasta, --f) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -48,6 +48,8 @@ process INSILICO_LIBRARY_GENERATION { max_fr_mz = params.max_fr_mz ? "--max-fr-mz $params.max_fr_mz":"" met_excision = params.met_excision ? "--met-excision" : "" diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" + diann_scoring_mode = params.diann_scoring_mode == 'proteoforms' ? '--proteoforms' : + params.diann_scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' diann_dda_flag = is_dda ? "--dda" : "" diann_light_models = params.diann_light_models ? "--light-models" : "" infin_dia_flag = params.enable_infin_dia ? "--infin-dia" : "" @@ -72,6 +74,7 @@ process INSILICO_LIBRARY_GENERATION { --verbose $params.diann_debug \\ --gen-spec-lib \\ ${diann_no_peptidoforms} \\ + ${diann_scoring_mode} \\ ${diann_light_models} \\ ${infin_dia_flag} \\ ${pre_select_flag} \\ diff --git a/modules/local/diann/preliminary_analysis/main.nf b/modules/local/diann/preliminary_analysis/main.nf index f9a3508..29963bb 100644 --- a/modules/local/diann/preliminary_analysis/main.nf +++ b/modules/local/diann/preliminary_analysis/main.nf @@ -29,7 +29,8 @@ process PRELIMINARY_ANALYSIS { '--quick-mass-acc', '--min-corr', '--corr-diff', '--time-corr-only', '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', '--monitor-mod', '--var-mod', '--fixed-mod', '--no-prot-inf', '--dda', - '--channels', '--lib-fixed-mod', '--original-mods'] + '--channels', '--lib-fixed-mod', '--original-mods', + '--proteoforms', '--peptidoforms'] // Sort by length descending so longer flags (e.g. --mass-acc-ms1) are matched before shorter prefixes (--mass-acc) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -43,6 +44,8 @@ process PRELIMINARY_ANALYSIS { quick_mass_acc = params.quick_mass_acc ? "--quick-mass-acc" : "" performance_flags = params.performance_mode ? "--min-corr 2 --corr-diff 1 --time-corr-only" : "" diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" + diann_scoring_mode = params.diann_scoring_mode == 'proteoforms' ? '--proteoforms' : + params.diann_scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' // I am using here the ["key"] syntax, since the preprocessed meta makes // was evaluating to null when using the dot notation. @@ -101,6 +104,7 @@ process PRELIMINARY_ANALYSIS { ${min_fr_mz} \\ ${max_fr_mz} \\ ${diann_no_peptidoforms} \\ + ${diann_scoring_mode} \\ ${diann_tims_sum} \\ ${diann_im_window} \\ --no-prot-inf \\ diff --git a/nextflow.config b/nextflow.config index a8fbb0d..5435e91 100644 --- a/nextflow.config +++ b/nextflow.config @@ -52,6 +52,7 @@ params { diann_debug = 3 diann_speclib = null diann_extra_args = null + diann_scoring_mode = 'generic' // Scoring mode: 'generic' (default), 'proteoforms' (variant detection, >= 2.0), 'peptidoforms' (PTM analysis) diann_dda = false // Fallback: explicitly enable DDA when SDRF lacks acquisition method (requires DIA-NN >= 2.3.2) diann_light_models = false // add '--light-models' for 10x faster library generation (DIA-NN >= 2.0) diann_export_quant = false // add '--export-quant' for fragment-level parquet export (DIA-NN >= 2.0) diff --git a/nextflow_schema.json b/nextflow_schema.json index 9fe8f21..b1e104b 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -388,6 +388,14 @@ "help_text": "Sets the DIA-NN parameter --quant-params. Provide previously obtained QuantUMS parameters to use instead of training new ones.", "hidden": false }, + "diann_scoring_mode": { + "type": "string", + "default": "generic", + "enum": ["generic", "proteoforms", "peptidoforms"], + "description": "DIA-NN scoring mode. 'generic' maximizes IDs (default). 'proteoforms' validates single-residue differences — recommended for proteogenomics/variant detection (requires DIA-NN >= 2.0). 'peptidoforms' provides extra peptidoform q-values for PTM analysis.", + "fa_icon": "fas fa-bullseye", + "help_text": "Controls DIA-NN's decoy generation strategy. In 'generic' mode, decoys are fully shuffled sequences. In 'proteoforms' mode, decoys differ by a single residue — directly modelling the confusion between canonical and variant peptides (e.g., COSMIC missense mutations). In 'peptidoforms' mode, generic q-values are calculated plus extra peptidoform-level q-values for PTM localization." + }, "diann_no_peptidoforms": { "type": "boolean", "description": "Set no-peptidoforms mode: disables automatic peptidoform scoring when variable modifications are declared (DIA-NN: not recommended).", diff --git a/workflows/dia.nf b/workflows/dia.nf index 69dd78d..ceddf37 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -45,6 +45,11 @@ workflow DIA { error("InfinDIA requires DIA-NN >= 2.3.0. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") } + // Version guard for scoring mode + if (params.diann_scoring_mode == 'proteoforms' && VersionUtils.versionLessThan(params.diann_version, '2.0')) { + error("--proteoforms scoring mode requires DIA-NN >= 2.0. Current version: ${params.diann_version}. Use -profile diann_v2_1_0 or later") + } + // Version guard for DIA-NN 2.0+ features if ((params.diann_light_models || params.diann_export_quant || params.diann_site_ms1_quant) && VersionUtils.versionLessThan(params.diann_version, '2.0')) { def enabled = [] From 4ee3a35d850faa93c6fb0e2252c816c1f4a0d26b Mon Sep 17 00:00:00 2001 From: yueqixuan Date: Fri, 10 Apr 2026 16:15:38 +0800 Subject: [PATCH 42/69] Update the quantms-utils and pmultiqc images --- modules/local/diann/diann_msstats/main.nf | 4 ++-- modules/local/diann/generate_cfg/main.nf | 4 ++-- modules/local/pmultiqc/main.nf | 4 ++-- modules/local/samplesheet_check/main.nf | 4 ++-- modules/local/utils/mzml_statistics/main.nf | 4 ++-- 5 files changed, 10 insertions(+), 10 deletions(-) diff --git a/modules/local/diann/diann_msstats/main.nf b/modules/local/diann/diann_msstats/main.nf index 1844fdc..a2e8636 100644 --- a/modules/local/diann/diann_msstats/main.nf +++ b/modules/local/diann/diann_msstats/main.nf @@ -3,8 +3,8 @@ process DIANN_MSSTATS { label 'process_medium' container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/quantms-utils:0.0.28--pyh106432d_0' : - 'biocontainers/quantms-utils:0.0.28--pyh106432d_0' }" + 'https://depot.galaxyproject.org/singularity/quantms-utils:0.0.29--pyhdfd78af_0' : + 'biocontainers/quantms-utils:0.0.29--pyhdfd78af_0' }" input: path(report) diff --git a/modules/local/diann/generate_cfg/main.nf b/modules/local/diann/generate_cfg/main.nf index 8e36641..8d1ce2a 100644 --- a/modules/local/diann/generate_cfg/main.nf +++ b/modules/local/diann/generate_cfg/main.nf @@ -3,8 +3,8 @@ process GENERATE_CFG { label 'process_tiny' container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/quantms-utils:0.0.28--pyh106432d_0' : - 'biocontainers/quantms-utils:0.0.28--pyh106432d_0' }" + 'https://depot.galaxyproject.org/singularity/quantms-utils:0.0.29--pyhdfd78af_0' : + 'biocontainers/quantms-utils:0.0.29--pyhdfd78af_0' }" input: val(meta) diff --git a/modules/local/pmultiqc/main.nf b/modules/local/pmultiqc/main.nf index facf4e8..2218a9f 100644 --- a/modules/local/pmultiqc/main.nf +++ b/modules/local/pmultiqc/main.nf @@ -2,8 +2,8 @@ process PMULTIQC { label 'process_high' container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/pmultiqc:0.0.43--pyhdfd78af_0' : - 'biocontainers/pmultiqc:0.0.43--pyhdfd78af_0' }" + 'https://depot.galaxyproject.org/singularity/pmultiqc:0.0.44--pyhdfd78af_0' : + 'biocontainers/pmultiqc:0.0.44--pyhdfd78af_0' }" input: path multiqc_inputs, stageAs: 'results/*' diff --git a/modules/local/samplesheet_check/main.nf b/modules/local/samplesheet_check/main.nf index 76fae90..129a646 100644 --- a/modules/local/samplesheet_check/main.nf +++ b/modules/local/samplesheet_check/main.nf @@ -4,8 +4,8 @@ process SAMPLESHEET_CHECK { label 'process_tiny' container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/quantms-utils:0.0.28--pyh106432d_0' : - 'biocontainers/quantms-utils:0.0.28--pyh106432d_0' }" + 'https://depot.galaxyproject.org/singularity/quantms-utils:0.0.29--pyhdfd78af_0' : + 'biocontainers/quantms-utils:0.0.29--pyhdfd78af_0' }" input: path input_file diff --git a/modules/local/utils/mzml_statistics/main.nf b/modules/local/utils/mzml_statistics/main.nf index d6eb3e0..f493371 100644 --- a/modules/local/utils/mzml_statistics/main.nf +++ b/modules/local/utils/mzml_statistics/main.nf @@ -4,8 +4,8 @@ process MZML_STATISTICS { label 'process_single' container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/quantms-utils:0.0.28--pyh106432d_0' : - 'biocontainers/quantms-utils:0.0.28--pyh106432d_0' }" + 'https://depot.galaxyproject.org/singularity/quantms-utils:0.0.29--pyhdfd78af_0' : + 'biocontainers/quantms-utils:0.0.29--pyhdfd78af_0' }" input: tuple val(meta), path(ms_file) From 8a3d00cfc14834d12d2c9f9dae402c59c67015a9 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 10 Apr 2026 09:18:20 +0100 Subject: [PATCH 43/69] refactor: rename diann_scoring_mode -> scoring_mode Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/parameters.md | 2 +- modules/local/diann/assemble_empirical_library/main.nf | 6 +++--- modules/local/diann/final_quantification/main.nf | 6 +++--- modules/local/diann/individual_analysis/main.nf | 6 +++--- modules/local/diann/insilico_library_generation/main.nf | 6 +++--- modules/local/diann/preliminary_analysis/main.nf | 6 +++--- nextflow.config | 2 +- nextflow_schema.json | 2 +- workflows/dia.nf | 2 +- 9 files changed, 19 insertions(+), 19 deletions(-) diff --git a/docs/parameters.md b/docs/parameters.md index 6cb9746..109eba0 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -114,7 +114,7 @@ This document lists every pipeline parameter organised by category. Default valu | `--diann_normalize` | boolean | `true` | Enable cross-run normalisation in DIA-NN. Set to `false` to add `--no-norm`. | | `--diann_report_decoys` | boolean | `false` | Include decoy PSMs in the main `.parquet` report (DIA-NN 2.0+ only). | | `--diann_export_xic` | boolean | `false` | Extract MS1/fragment chromatograms for identified precursors (equivalent to the XICs option in the DIA-NN GUI). | -| `--diann_scoring_mode` | string | `generic` | DIA-NN scoring mode: `generic` (default, maximizes IDs), `proteoforms` (recommended for proteogenomics/variant detection, requires >= 2.0), `peptidoforms` (extra q-values for PTM analysis). | +| `--scoring_mode` | string | `generic` | DIA-NN scoring mode: `generic` (default, maximizes IDs), `proteoforms` (recommended for proteogenomics/variant detection, requires >= 2.0), `peptidoforms` (extra q-values for PTM analysis). | | `--diann_no_peptidoforms` | boolean | `false` | Disable automatic peptidoform scoring when variable modifications are declared (not recommended by DIA-NN). | | `--diann_use_quant` | boolean | `true` | Reuse existing `.quant` files if available (`--use-quant`). | | `--quantums` | boolean | `false` | Enable QuantUMS quantification (requires DIA-NN >= 1.9.2). When `false`, the pipeline passes `--direct-quant` to use legacy quantification (only for DIA-NN >= 1.9.2; silently skipped for 1.8.x where direct quant is the only mode). | diff --git a/modules/local/diann/assemble_empirical_library/main.nf b/modules/local/diann/assemble_empirical_library/main.nf index c66c005..450cd4a 100644 --- a/modules/local/diann/assemble_empirical_library/main.nf +++ b/modules/local/diann/assemble_empirical_library/main.nf @@ -53,8 +53,8 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { } scan_window = params.scan_window_automatic ? '--individual-windows' : "--window $params.scan_window" diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" - diann_scoring_mode = params.diann_scoring_mode == 'proteoforms' ? '--proteoforms' : - params.diann_scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' + scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : + params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' diann_tims_sum = params.diann_tims_sum ? "--quant-tims-sum" : "" diann_im_window = params.diann_im_window ? "--im-window $params.diann_im_window" : "" diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" @@ -82,7 +82,7 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { ${scan_window} \\ --gen-spec-lib \\ ${diann_no_peptidoforms} \\ - ${diann_scoring_mode} \\ + ${scoring_mode} \\ ${diann_tims_sum} \\ ${diann_im_window} \\ ${diann_dda_flag} \\ diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index 5d509da..58625a6 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -72,8 +72,8 @@ process FINAL_QUANTIFICATION { quantums_sel_runs = params.quantums_sel_runs ? "--quant-sel-runs $params.quantums_sel_runs": "" quantums_params = params.quantums_params ? "--quant-params $params.quantums_params": "" diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" - diann_scoring_mode = params.diann_scoring_mode == 'proteoforms' ? '--proteoforms' : - params.diann_scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' + scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : + params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' diann_use_quant = params.diann_use_quant ? "--use-quant" : "" diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" diann_export_quant = params.diann_export_quant ? "--export-quant" : "" @@ -108,7 +108,7 @@ process FINAL_QUANTIFICATION { ${quantums_sel_runs} \\ ${quantums_params} \\ ${diann_no_peptidoforms} \\ - ${diann_scoring_mode} \\ + ${scoring_mode} \\ ${diann_use_quant} \\ ${diann_dda_flag} \\ ${diann_export_quant} \\ diff --git a/modules/local/diann/individual_analysis/main.nf b/modules/local/diann/individual_analysis/main.nf index 37cfe69..b6c1cf1 100644 --- a/modules/local/diann/individual_analysis/main.nf +++ b/modules/local/diann/individual_analysis/main.nf @@ -82,8 +82,8 @@ process INDIVIDUAL_ANALYSIS { } diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" - diann_scoring_mode = params.diann_scoring_mode == 'proteoforms' ? '--proteoforms' : - params.diann_scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' + scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : + params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' diann_tims_sum = params.diann_tims_sum ? "--quant-tims-sum" : "" diann_im_window = params.diann_im_window ? "--im-window $params.diann_im_window" : "" diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" @@ -120,7 +120,7 @@ process INDIVIDUAL_ANALYSIS { ${min_fr_mz} \\ ${max_fr_mz} \\ ${diann_no_peptidoforms} \\ - ${diann_scoring_mode} \\ + ${scoring_mode} \\ ${diann_tims_sum} \\ ${diann_im_window} \\ ${diann_dda_flag} \\ diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index 42b52d8..3c64f40 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -48,8 +48,8 @@ process INSILICO_LIBRARY_GENERATION { max_fr_mz = params.max_fr_mz ? "--max-fr-mz $params.max_fr_mz":"" met_excision = params.met_excision ? "--met-excision" : "" diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" - diann_scoring_mode = params.diann_scoring_mode == 'proteoforms' ? '--proteoforms' : - params.diann_scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' + scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : + params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' diann_dda_flag = is_dda ? "--dda" : "" diann_light_models = params.diann_light_models ? "--light-models" : "" infin_dia_flag = params.enable_infin_dia ? "--infin-dia" : "" @@ -74,7 +74,7 @@ process INSILICO_LIBRARY_GENERATION { --verbose $params.diann_debug \\ --gen-spec-lib \\ ${diann_no_peptidoforms} \\ - ${diann_scoring_mode} \\ + ${scoring_mode} \\ ${diann_light_models} \\ ${infin_dia_flag} \\ ${pre_select_flag} \\ diff --git a/modules/local/diann/preliminary_analysis/main.nf b/modules/local/diann/preliminary_analysis/main.nf index 29963bb..50f1aee 100644 --- a/modules/local/diann/preliminary_analysis/main.nf +++ b/modules/local/diann/preliminary_analysis/main.nf @@ -44,8 +44,8 @@ process PRELIMINARY_ANALYSIS { quick_mass_acc = params.quick_mass_acc ? "--quick-mass-acc" : "" performance_flags = params.performance_mode ? "--min-corr 2 --corr-diff 1 --time-corr-only" : "" diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" - diann_scoring_mode = params.diann_scoring_mode == 'proteoforms' ? '--proteoforms' : - params.diann_scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' + scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : + params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' // I am using here the ["key"] syntax, since the preprocessed meta makes // was evaluating to null when using the dot notation. @@ -104,7 +104,7 @@ process PRELIMINARY_ANALYSIS { ${min_fr_mz} \\ ${max_fr_mz} \\ ${diann_no_peptidoforms} \\ - ${diann_scoring_mode} \\ + ${scoring_mode} \\ ${diann_tims_sum} \\ ${diann_im_window} \\ --no-prot-inf \\ diff --git a/nextflow.config b/nextflow.config index 5435e91..cb51d8c 100644 --- a/nextflow.config +++ b/nextflow.config @@ -52,7 +52,7 @@ params { diann_debug = 3 diann_speclib = null diann_extra_args = null - diann_scoring_mode = 'generic' // Scoring mode: 'generic' (default), 'proteoforms' (variant detection, >= 2.0), 'peptidoforms' (PTM analysis) + scoring_mode = 'generic' // Scoring mode: 'generic' (default), 'proteoforms' (variant detection, >= 2.0), 'peptidoforms' (PTM analysis) diann_dda = false // Fallback: explicitly enable DDA when SDRF lacks acquisition method (requires DIA-NN >= 2.3.2) diann_light_models = false // add '--light-models' for 10x faster library generation (DIA-NN >= 2.0) diann_export_quant = false // add '--export-quant' for fragment-level parquet export (DIA-NN >= 2.0) diff --git a/nextflow_schema.json b/nextflow_schema.json index b1e104b..6a358dc 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -388,7 +388,7 @@ "help_text": "Sets the DIA-NN parameter --quant-params. Provide previously obtained QuantUMS parameters to use instead of training new ones.", "hidden": false }, - "diann_scoring_mode": { + "scoring_mode": { "type": "string", "default": "generic", "enum": ["generic", "proteoforms", "peptidoforms"], diff --git a/workflows/dia.nf b/workflows/dia.nf index ceddf37..b4781ec 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -46,7 +46,7 @@ workflow DIA { } // Version guard for scoring mode - if (params.diann_scoring_mode == 'proteoforms' && VersionUtils.versionLessThan(params.diann_version, '2.0')) { + if (params.scoring_mode == 'proteoforms' && VersionUtils.versionLessThan(params.diann_version, '2.0')) { error("--proteoforms scoring mode requires DIA-NN >= 2.0. Current version: ${params.diann_version}. Use -profile diann_v2_1_0 or later") } From 6ef44faa6bfc062933dcfd5d41516326a4376a40 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 10 Apr 2026 09:20:18 +0100 Subject: [PATCH 44/69] style: format parameters.md with prettier Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/parameters.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/parameters.md b/docs/parameters.md index 109eba0..976ca7b 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -114,7 +114,7 @@ This document lists every pipeline parameter organised by category. Default valu | `--diann_normalize` | boolean | `true` | Enable cross-run normalisation in DIA-NN. Set to `false` to add `--no-norm`. | | `--diann_report_decoys` | boolean | `false` | Include decoy PSMs in the main `.parquet` report (DIA-NN 2.0+ only). | | `--diann_export_xic` | boolean | `false` | Extract MS1/fragment chromatograms for identified precursors (equivalent to the XICs option in the DIA-NN GUI). | -| `--scoring_mode` | string | `generic` | DIA-NN scoring mode: `generic` (default, maximizes IDs), `proteoforms` (recommended for proteogenomics/variant detection, requires >= 2.0), `peptidoforms` (extra q-values for PTM analysis). | +| `--scoring_mode` | string | `generic` | DIA-NN scoring mode: `generic` (default, maximizes IDs), `proteoforms` (recommended for proteogenomics/variant detection, requires >= 2.0), `peptidoforms` (extra q-values for PTM analysis). | | `--diann_no_peptidoforms` | boolean | `false` | Disable automatic peptidoform scoring when variable modifications are declared (not recommended by DIA-NN). | | `--diann_use_quant` | boolean | `true` | Reuse existing `.quant` files if available (`--use-quant`). | | `--quantums` | boolean | `false` | Enable QuantUMS quantification (requires DIA-NN >= 1.9.2). When `false`, the pipeline passes `--direct-quant` to use legacy quantification (only for DIA-NN >= 1.9.2; silently skipped for 1.8.x where direct quant is the only mode). | From 77379525b681486768c0defd1dbafad280ce1310 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 10 Apr 2026 13:32:40 +0100 Subject: [PATCH 45/69] refactor: remove diann_ prefix from all params, remove no_peptidoforms MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Since quantmsdiann is a DIA-NN-only pipeline, the diann_ prefix on parameters is redundant. Renamed all user-facing params: diann_debug -> debug_level diann_speclib -> speclib diann_extra_args -> extra_args diann_dda -> dda diann_light_models -> light_models diann_export_quant -> export_quant diann_site_ms1_quant -> site_ms1_quant diann_pre_select -> pre_select diann_report_decoys -> report_decoys diann_export_xic -> export_xic diann_normalize -> normalize diann_use_quant -> use_quant diann_tims_sum -> tims_sum diann_im_window -> im_window diann_channel_run_norm -> channel_run_norm diann_channel_spec_norm -> channel_spec_norm Removed diann_no_peptidoforms entirely — superseded by the new scoring_mode parameter (generic/proteoforms/peptidoforms). --no-peptidoforms remains in blocked flags lists. Note: diann_version is NOT renamed (used in profile names). Co-Authored-By: Claude Opus 4.6 (1M context) --- AGENTS.md | 2 +- README.md | 2 +- conf/modules/dia.config | 12 +-- conf/tests/test_dda.config | 6 +- conf/tests/test_dia.config | 2 +- conf/tests/test_dia_2_2_0.config | 2 +- conf/tests/test_dia_dotd.config | 2 +- conf/tests/test_dia_parquet.config | 4 +- conf/tests/test_dia_quantums.config | 2 +- conf/tests/test_dia_skip_preanalysis.config | 2 +- conf/tests/test_full_dia.config | 2 +- conf/tests/test_latest_dia.config | 2 +- docs/parameters.md | 79 +++++++++---------- docs/usage.md | 20 ++--- .../diann/assemble_empirical_library/main.nf | 10 +-- .../local/diann/final_quantification/main.nf | 22 +++--- .../local/diann/individual_analysis/main.nf | 10 +-- .../diann/insilico_library_generation/main.nf | 10 +-- .../local/diann/preliminary_analysis/main.nf | 10 +-- nextflow.config | 33 ++++---- nextflow_schema.json | 38 ++++----- .../local/create_input_channel/main.nf | 4 +- workflows/dia.nf | 18 ++--- 23 files changed, 138 insertions(+), 156 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 982fcfe..a051f4e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -114,7 +114,7 @@ The pipeline executes the following steps: | Core workflow, library-free, .quant caching | 1.8.1 | (default) | | QuantUMS quantification | 1.9.2 | `--quantums true` | | Parquet output format | 2.0 | (automatic in 2.0+) | -| Decoy reporting | 2.0 | `--diann_report_decoys true` | +| Decoy reporting | 2.0 | `--report_decoys true` | | Native .raw on Linux | 2.1.0 | (automatic) | --- diff --git a/README.md b/README.md index ee88b0e..2d0d725 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ The pipeline takes [SDRF](https://github.com/bigbio/proteomics-metadata-standard 1. **Input validation** — SDRF parsing and validation via [sdrf-pipelines](https://github.com/bigbio/sdrf-pipelines) 2. **File preparation** — RAW to mzML conversion ([ThermoRawFileParser](https://github.com/compomics/ThermoRawFileParser)), indexing -3. **In-silico spectral library generation** — deep learning-based prediction, or use a user-provided library (`--diann_speclib`) +3. **In-silico spectral library generation** — deep learning-based prediction, or use a user-provided library (`--speclib`) 4. **Preliminary analysis** — per-file calibration and mass accuracy estimation (parallelized) 5. **Empirical library assembly** — consensus library from preliminary results with RT profiling 6. **Individual analysis** — per-file search with the empirical library (parallelized) diff --git a/conf/modules/dia.config b/conf/modules/dia.config index fff6e00..36708bb 100644 --- a/conf/modules/dia.config +++ b/conf/modules/dia.config @@ -2,7 +2,7 @@ ======================================================================================== DIA-NN module options — all DIA pipeline steps ======================================================================================== - Passes params.diann_extra_args to each DIA-NN step via ext.args. + Passes params.extra_args (formerly params.diann_extra_args) to each DIA-NN step via ext.args. Blocked-flag validation is performed in each module's script block, so it applies regardless of whether ext.args is overridden by the user. ---------------------------------------------------------------------------------------- @@ -11,22 +11,22 @@ process { withName: ".*:DIA:INSILICO_LIBRARY_GENERATION" { - ext.args = { params.diann_extra_args ?: '' } + ext.args = { params.extra_args ?: '' } } withName: ".*:DIA:PRELIMINARY_ANALYSIS" { - ext.args = { params.diann_extra_args ?: '' } + ext.args = { params.extra_args ?: '' } } withName: ".*:DIA:ASSEMBLE_EMPIRICAL_LIBRARY" { - ext.args = { params.diann_extra_args ?: '' } + ext.args = { params.extra_args ?: '' } } withName: ".*:DIA:INDIVIDUAL_ANALYSIS" { - ext.args = { params.diann_extra_args ?: '' } + ext.args = { params.extra_args ?: '' } } withName: ".*:DIA:FINAL_QUANTIFICATION" { - ext.args = { params.diann_extra_args ?: '' } + ext.args = { params.extra_args ?: '' } } } diff --git a/conf/tests/test_dda.config b/conf/tests/test_dda.config index f7870c4..6c81d4e 100644 --- a/conf/tests/test_dda.config +++ b/conf/tests/test_dda.config @@ -2,7 +2,7 @@ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Nextflow config file for testing DDA analysis (requires DIA-NN >= 2.3.2) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Tests DDA mode using the PXD022287 HeLa dataset with --diann_dda flag. + Tests DDA mode using the PXD022287 HeLa dataset with --dda flag. Uses ghcr.io/bigbio/diann:2.3.2. Use as follows: @@ -30,7 +30,7 @@ params { database = 'https://raw.githubusercontent.com/bigbio/quantms-test-datasets/quantms/testdata/diann_dda_ci/PXD022287_subset_human.fasta' // DDA mode - diann_dda = true + dda = true diann_version = '2.3.2' // Search parameters matching PXD022287 HeLa dataset @@ -38,7 +38,7 @@ params { max_peptide_length = 30 max_precursor_charge = 3 allowed_missed_cleavages = 1 - diann_normalize = false + normalize = false publish_dir_mode = 'symlink' max_mods = 2 } diff --git a/conf/tests/test_dia.config b/conf/tests/test_dia.config index 2d321bc..85139de 100644 --- a/conf/tests/test_dia.config +++ b/conf/tests/test_dia.config @@ -34,7 +34,7 @@ params { max_peptide_length = 30 max_precursor_charge = 3 allowed_missed_cleavages = 1 - diann_normalize = false + normalize = false publish_dir_mode = 'symlink' max_mods = 2 } diff --git a/conf/tests/test_dia_2_2_0.config b/conf/tests/test_dia_2_2_0.config index 3772ec6..b36403d 100644 --- a/conf/tests/test_dia_2_2_0.config +++ b/conf/tests/test_dia_2_2_0.config @@ -36,7 +36,7 @@ params { max_peptide_length = 30 max_precursor_charge = 3 allowed_missed_cleavages = 1 - diann_normalize = false + normalize = false publish_dir_mode = 'symlink' max_mods = 2 } diff --git a/conf/tests/test_dia_dotd.config b/conf/tests/test_dia_dotd.config index cd151ca..8d1d8fc 100644 --- a/conf/tests/test_dia_dotd.config +++ b/conf/tests/test_dia_dotd.config @@ -37,7 +37,7 @@ params { max_peptide_length = 30 max_precursor_charge = 4 allowed_missed_cleavages = 2 - diann_normalize = false + normalize = false publish_dir_mode = 'symlink' max_mods = 2 mass_acc_automatic = false diff --git a/conf/tests/test_dia_parquet.config b/conf/tests/test_dia_parquet.config index dfea81d..ab80af0 100644 --- a/conf/tests/test_dia_parquet.config +++ b/conf/tests/test_dia_parquet.config @@ -36,12 +36,12 @@ params { max_peptide_length = 30 max_precursor_charge = 3 allowed_missed_cleavages = 1 - diann_normalize = false + normalize = false publish_dir_mode = 'symlink' max_mods = 2 // Parquet-specific: enable decoy reporting (Parquet-only feature in DIA-NN 2.0+) - diann_report_decoys = true + report_decoys = true } process { diff --git a/conf/tests/test_dia_quantums.config b/conf/tests/test_dia_quantums.config index 9f531b6..aa8e260 100644 --- a/conf/tests/test_dia_quantums.config +++ b/conf/tests/test_dia_quantums.config @@ -36,7 +36,7 @@ params { max_peptide_length = 30 max_precursor_charge = 3 allowed_missed_cleavages = 1 - diann_normalize = false + normalize = false publish_dir_mode = 'symlink' max_mods = 2 diff --git a/conf/tests/test_dia_skip_preanalysis.config b/conf/tests/test_dia_skip_preanalysis.config index e1968b5..49e0409 100644 --- a/conf/tests/test_dia_skip_preanalysis.config +++ b/conf/tests/test_dia_skip_preanalysis.config @@ -36,7 +36,7 @@ params { max_peptide_length = 30 max_precursor_charge = 3 allowed_missed_cleavages = 1 - diann_normalize = false + normalize = false publish_dir_mode = 'symlink' max_mods = 2 diff --git a/conf/tests/test_full_dia.config b/conf/tests/test_full_dia.config index 67b05a2..edce596 100644 --- a/conf/tests/test_full_dia.config +++ b/conf/tests/test_full_dia.config @@ -33,6 +33,6 @@ params { max_fr_mz = 1500 max_precursor_charge = 3 allowed_missed_cleavages = 1 - diann_normalize = false + normalize = false max_mods = 1 } diff --git a/conf/tests/test_latest_dia.config b/conf/tests/test_latest_dia.config index 10bd418..bd25abe 100644 --- a/conf/tests/test_latest_dia.config +++ b/conf/tests/test_latest_dia.config @@ -27,7 +27,7 @@ params { max_peptide_length = 30 max_precursor_charge = 3 allowed_missed_cleavages = 1 - diann_normalize = false + normalize = false publish_dir_mode = 'symlink' max_mods = 2 } diff --git a/docs/parameters.md b/docs/parameters.md index 976ca7b..462d593 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -52,16 +52,16 @@ This document lists every pipeline parameter organised by category. Default valu ## 5. DIA-NN General -| Parameter | Type | Default | Description | -| ------------------------ | ------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `--diann_version` | string | `1.8.1` | DIA-NN version used by the workflow. Controls version-dependent flags (e.g. `--monitor-mod` for 1.8.x). See [DIA-NN Version Selection](usage.md#dia-nn-version-selection). | -| `--diann_debug` | integer | `3` | DIA-NN debug/verbosity level (0-4). Higher values produce more verbose logs. | -| `--diann_speclib` | string | `null` | Path to an external spectral library. If provided, the in-silico library generation step is skipped. | -| `--diann_extra_args` | string | `null` | Extra arguments appended to all DIA-NN steps. Flags incompatible with a step are automatically stripped with a warning. See [Passing Extra Arguments to DIA-NN](usage.md#passing-extra-arguments-to-dia-nn). | -| `--diann_dda` | boolean | `false` | Explicitly enable DDA mode. Normally auto-detected from the SDRF `comment[proteomics data acquisition method]` column. Use this flag only when the SDRF lacks the acquisition method. Requires DIA-NN >= 2.3.2. | -| `--diann_light_models` | boolean | `false` | Enable `--light-models` for 10x faster in-silico library generation. Requires DIA-NN >= 2.0. | -| `--diann_export_quant` | boolean | `false` | Enable `--export-quant` for fragment-level parquet data export. Requires DIA-NN >= 2.0. | -| `--diann_site_ms1_quant` | boolean | `false` | Enable `--site-ms1-quant` to use MS1 apex intensities for PTM site quantification. Requires DIA-NN >= 2.0. | +| Parameter | Type | Default | Description | +| ------------------ | ------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `--diann_version` | string | `1.8.1` | DIA-NN version used by the workflow. Controls version-dependent flags (e.g. `--monitor-mod` for 1.8.x). See [DIA-NN Version Selection](usage.md#dia-nn-version-selection). | +| `--debug_level` | integer | `3` | DIA-NN debug/verbosity level (0-4). Higher values produce more verbose logs. | +| `--speclib` | string | `null` | Path to an external spectral library. If provided, the in-silico library generation step is skipped. | +| `--extra_args` | string | `null` | Extra arguments appended to all DIA-NN steps. Flags incompatible with a step are automatically stripped with a warning. See [Passing Extra Arguments to DIA-NN](usage.md#passing-extra-arguments-to-dia-nn). | +| `--dda` | boolean | `false` | Explicitly enable DDA mode. Normally auto-detected from the SDRF `comment[proteomics data acquisition method]` column. Use this flag only when the SDRF lacks the acquisition method. Requires DIA-NN >= 2.3.2. | +| `--light_models` | boolean | `false` | Enable `--light-models` for 10x faster in-silico library generation. Requires DIA-NN >= 2.0. | +| `--export_quant` | boolean | `false` | Enable `--export-quant` for fragment-level parquet data export. Requires DIA-NN >= 2.0. | +| `--site_ms1_quant` | boolean | `false` | Enable `--site-ms1-quant` to use MS1 apex intensities for PTM site quantification. Requires DIA-NN >= 2.0. | ## 6. Mass Accuracy & Calibration @@ -77,10 +77,10 @@ This document lists every pipeline parameter organised by category. Default valu ## 7. Bruker/timsTOF -| Parameter | Type | Default | Description | -| ------------------- | ------- | ------- | ---------------------------------------------------------------------------------------------------- | -| `--diann_tims_sum` | boolean | `false` | Enable `--quant-tims-sum` for slice/scanning timsTOF methods (highly recommended for Synchro-PASEF). | -| `--diann_im_window` | number | `null` | Set `--im-window` to ensure the IM extraction window is not smaller than the specified value. | +| Parameter | Type | Default | Description | +| ------------- | ------- | ------- | ---------------------------------------------------------------------------------------------------- | +| `--tims_sum` | boolean | `false` | Enable `--quant-tims-sum` for slice/scanning timsTOF methods (highly recommended for Synchro-PASEF). | +| `--im_window` | number | `null` | Set `--im-window` to ensure the IM extraction window is not smaller than the specified value. | ## 8. PTM Localization @@ -97,39 +97,38 @@ This document lists every pipeline parameter organised by category. Default valu ## 10. Preliminary Analysis -| Parameter | Type | Default | Description | -| ----------------------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------ | -| `--skip_preliminary_analysis` | boolean | `false` | Skip preliminary analysis. Use the provided spectral library as-is instead of generating a local consensus library. | -| `--empirical_assembly_log` | string | `null` | Path to a pre-existing empirical assembly log file. Only used when `--skip_preliminary_analysis true` and `--diann_speclib` are set. | -| `--random_preanalysis` | boolean | `false` | Enable random selection of spectrum files for empirical library generation. | -| `--random_preanalysis_seed` | integer | `42` | Random seed for file selection when `--random_preanalysis` is enabled. | -| `--empirical_assembly_ms_n` | integer | `200` | Number of randomly selected spectrum files when `--random_preanalysis` is enabled. | +| Parameter | Type | Default | Description | +| ----------------------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------------------------------ | +| `--skip_preliminary_analysis` | boolean | `false` | Skip preliminary analysis. Use the provided spectral library as-is instead of generating a local consensus library. | +| `--empirical_assembly_log` | string | `null` | Path to a pre-existing empirical assembly log file. Only used when `--skip_preliminary_analysis true` and `--speclib` are set. | +| `--random_preanalysis` | boolean | `false` | Enable random selection of spectrum files for empirical library generation. | +| `--random_preanalysis_seed` | integer | `42` | Random seed for file selection when `--random_preanalysis` is enabled. | +| `--empirical_assembly_ms_n` | integer | `200` | Number of randomly selected spectrum files when `--random_preanalysis` is enabled. | ## 11. Quantification & Output -| Parameter | Type | Default | Description | -| ------------------------- | ------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `--pg_level` | integer | `2` | Protein inference mode. `0` = isoforms, `1` = protein names (from FASTA), `2` = genes (default). | -| `--species_genes` | boolean | `false` | Add the organism identifier to gene names in DIA-NN output. | -| `--diann_normalize` | boolean | `true` | Enable cross-run normalisation in DIA-NN. Set to `false` to add `--no-norm`. | -| `--diann_report_decoys` | boolean | `false` | Include decoy PSMs in the main `.parquet` report (DIA-NN 2.0+ only). | -| `--diann_export_xic` | boolean | `false` | Extract MS1/fragment chromatograms for identified precursors (equivalent to the XICs option in the DIA-NN GUI). | -| `--scoring_mode` | string | `generic` | DIA-NN scoring mode: `generic` (default, maximizes IDs), `proteoforms` (recommended for proteogenomics/variant detection, requires >= 2.0), `peptidoforms` (extra q-values for PTM analysis). | -| `--diann_no_peptidoforms` | boolean | `false` | Disable automatic peptidoform scoring when variable modifications are declared (not recommended by DIA-NN). | -| `--diann_use_quant` | boolean | `true` | Reuse existing `.quant` files if available (`--use-quant`). | -| `--quantums` | boolean | `false` | Enable QuantUMS quantification (requires DIA-NN >= 1.9.2). When `false`, the pipeline passes `--direct-quant` to use legacy quantification (only for DIA-NN >= 1.9.2; silently skipped for 1.8.x where direct quant is the only mode). | -| `--quantums_train_runs` | string | `null` | Run index range for QuantUMS training (e.g. `0:5`). Maps to `--quant-train-runs`. Requires DIA-NN >= 1.9.2. | -| `--quantums_sel_runs` | integer | `null` | Number of automatically selected runs for QuantUMS training. Must be >= 6. Maps to `--quant-sel-runs`. Requires DIA-NN >= 1.9.2. | -| `--quantums_params` | string | `null` | Pre-calculated QuantUMS parameters. Maps to `--quant-params`. Requires DIA-NN >= 1.9.2. | +| Parameter | Type | Default | Description | +| ----------------------- | ------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `--pg_level` | integer | `2` | Protein inference mode. `0` = isoforms, `1` = protein names (from FASTA), `2` = genes (default). | +| `--species_genes` | boolean | `false` | Add the organism identifier to gene names in DIA-NN output. | +| `--normalize` | boolean | `true` | Enable cross-run normalisation in DIA-NN. Set to `false` to add `--no-norm`. | +| `--report_decoys` | boolean | `false` | Include decoy PSMs in the main `.parquet` report (DIA-NN 2.0+ only). | +| `--export_xic` | boolean | `false` | Extract MS1/fragment chromatograms for identified precursors (equivalent to the XICs option in the DIA-NN GUI). | +| `--scoring_mode` | string | `generic` | DIA-NN scoring mode: `generic` (default, maximizes IDs), `proteoforms` (recommended for proteogenomics/variant detection, requires >= 2.0), `peptidoforms` (extra q-values for PTM analysis). | +| `--use_quant` | boolean | `true` | Reuse existing `.quant` files if available (`--use-quant`). | +| `--quantums` | boolean | `false` | Enable QuantUMS quantification (requires DIA-NN >= 1.9.2). When `false`, the pipeline passes `--direct-quant` to use legacy quantification (only for DIA-NN >= 1.9.2; silently skipped for 1.8.x where direct quant is the only mode). | +| `--quantums_train_runs` | string | `null` | Run index range for QuantUMS training (e.g. `0:5`). Maps to `--quant-train-runs`. Requires DIA-NN >= 1.9.2. | +| `--quantums_sel_runs` | integer | `null` | Number of automatically selected runs for QuantUMS training. Must be >= 6. Maps to `--quant-sel-runs`. Requires DIA-NN >= 1.9.2. | +| `--quantums_params` | string | `null` | Pre-calculated QuantUMS parameters. Maps to `--quant-params`. Requires DIA-NN >= 1.9.2. | ## 12. DDA Mode -| Parameter | Type | Default | Description | -| ------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `--diann_dda` | boolean | `false` | Explicitly enable DDA mode when SDRF lacks the acquisition method column. Normally DDA is auto-detected from the SDRF `comment[proteomics data acquisition method]`. Requires DIA-NN >= 2.3.2 (use `-profile diann_v2_3_2`). Beta feature. | +| Parameter | Type | Default | Description | +| --------- | ------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `--dda` | boolean | `false` | Explicitly enable DDA mode when SDRF lacks the acquisition method column. Normally DDA is auto-detected from the SDRF `comment[proteomics data acquisition method]`. Requires DIA-NN >= 2.3.2 (use `-profile diann_v2_3_2`). Beta feature. | > **Note:** DDA mode is auto-detected from the SDRF when the `comment[proteomics data acquisition method]` -> column contains `data-dependent acquisition`. The `--diann_dda` flag is only needed as a +> column contains `data-dependent acquisition`. The `--dda` flag is only needed as a > fallback when the SDRF does not include this column. DDA requires DIA-NN >= 2.3.2 > (`-profile diann_v2_3_2`). @@ -138,7 +137,7 @@ This document lists every pipeline parameter organised by category. Default valu | Parameter | Type | Default | Description | | -------------------- | ------- | ------- | -------------------------------------------------------------------------------------- | | `--enable_infin_dia` | boolean | `false` | Enable InfinDIA for ultra-large search spaces. Requires DIA-NN >= 2.3.0. Experimental. | -| `--diann_pre_select` | integer | `null` | Precursor limit (`--pre-select N`) for InfinDIA pre-search. | +| `--pre_select` | integer | `null` | Precursor limit (`--pre-select N`) for InfinDIA pre-search. | > **Note:** InfinDIA requires DIA-NN >= 2.3.0 and is considered experimental. diff --git a/docs/usage.md b/docs/usage.md index 8b696df..510a026 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -59,7 +59,7 @@ nextflow run bigbio/quantmsdiann \ -profile docker ``` -For Synchro-PASEF data, enable `--diann_tims_sum` (which adds `--quant-tims-sum` to DIA-NN). +For Synchro-PASEF data, enable `--tims_sum` (which adds `--quant-tims-sum` to DIA-NN). > [!NOTE] > The pipeline will emit a warning during PRELIMINARY_ANALYSIS if it detects `.d` files with automatic mass accuracy calibration enabled, recommending to set tolerances via SDRF or pipeline parameters. @@ -75,13 +75,13 @@ nextflow run bigbio/quantmsdiann \ -profile diann_v2_3_2,docker ``` -If your SDRF does not include the acquisition method column, you can explicitly enable DDA mode with `--diann_dda true`: +If your SDRF does not include the acquisition method column, you can explicitly enable DDA mode with `--dda true`: ```bash nextflow run bigbio/quantmsdiann \ --input sdrf.tsv \ --database proteins.fasta \ - --diann_dda true \ + --dda true \ -profile diann_v2_3_2,docker ``` @@ -93,7 +93,7 @@ nextflow run bigbio/quantmsdiann \ - No isobaric labeling or reporter-tag quantification - Primary use cases: legacy DDA reanalysis, spectral library creation, immunopeptidomics -The pipeline uses the same workflow for DDA as DIA — the `--dda` flag is passed to all DIA-NN steps automatically when DDA is detected from the SDRF or enabled via `--diann_dda`. +The pipeline uses the same workflow for DDA as DIA — the `--dda` flag is passed to all DIA-NN steps automatically when DDA is detected from the SDRF or enabled via `--dda`. ### Preprocessing Options @@ -104,7 +104,7 @@ The pipeline uses the same workflow for DDA as DIA — the `--dda` flag is passe ### Passing Extra Arguments to DIA-NN -Use `--diann_extra_args` to pass additional flags to all DIA-NN steps. The pipeline validates and strips flags it manages internally to prevent conflicts. +Use `--extra_args` to pass additional flags to all DIA-NN steps. The pipeline validates and strips flags it manages internally to prevent conflicts. Managed flags (stripped with a warning if passed via extra_args): `--lib`, `--f`, `--fasta`, `--threads`, `--verbose`, `--temp`, `--out`, `--matrices`, `--use-quant`, `--gen-spec-lib`, `--mass-acc`, `--mass-acc-ms1`, `--window`, `--var-mod`, `--fixed-mod`, `--monitor-mod`, and others. @@ -242,7 +242,7 @@ These parameters apply globally across all files. They are set in `diann_config. | `--min-corr 2 --corr-diff 1 --time-corr-only` | `--performance_mode` | `true` | PRELIMINARY | High-speed, low-RAM mode | | `--pg-level` | `--pg_level` | `2` | INDIVIDUAL, FINAL | Protein grouping level | | `--species-genes` | `--species_genes` | `false` | FINAL | Use species-specific gene names | -| `--no-norm` | `--diann_normalize` | `true` | FINAL | Disable normalization when `false` | +| `--no-norm` | `--normalize` | `true` | FINAL | Disable normalization when `false` | ### PTM site localization (`--monitor-mod`) @@ -283,11 +283,11 @@ Supported modification name mappings: ## Passing Extra Arguments to DIA-NN -The `--diann_extra_args` parameter appends additional DIA-NN command-line flags to **all** DIA-NN steps (INSILICO_LIBRARY_GENERATION, PRELIMINARY_ANALYSIS, ASSEMBLE_EMPIRICAL_LIBRARY, INDIVIDUAL_ANALYSIS, FINAL_QUANTIFICATION). +The `--extra_args` parameter appends additional DIA-NN command-line flags to **all** DIA-NN steps (INSILICO_LIBRARY_GENERATION, PRELIMINARY_ANALYSIS, ASSEMBLE_EMPIRICAL_LIBRARY, INDIVIDUAL_ANALYSIS, FINAL_QUANTIFICATION). ```bash nextflow run bigbio/quantmsdiann \ - --diann_extra_args '--smart-profiling --peak-center' \ + --extra_args '--smart-profiling --peak-center' \ ... ``` @@ -323,12 +323,12 @@ The pipeline supports multiple DIA-NN versions via built-in Nextflow profiles. E | `diann_v1_8_1` | 1.8.1 | `docker.io/biocontainers/diann:v1.8.1_cv1` | Default. Public BioContainers image. TSV output. | | `diann_v2_1_0` | 2.1.0 | `ghcr.io/bigbio/diann:2.1.0` | Parquet output. Native .raw on Linux. QuantUMS (`--quantums`). | | `diann_v2_2_0` | 2.2.0 | `ghcr.io/bigbio/diann:2.2.0` | Speed optimizations (up to 1.6x on HPC). Parquet output. | -| `diann_v2_3_2` | 2.3.2 | `ghcr.io/bigbio/diann:2.3.2` | DDA support (`--diann_dda`), InfinDIA, up to 9 variable mods. | +| `diann_v2_3_2` | 2.3.2 | `ghcr.io/bigbio/diann:2.3.2` | DDA support (`--dda`), InfinDIA, up to 9 variable mods. | **Version-dependent features:** Some parameters are only available with newer DIA-NN versions. The pipeline handles version compatibility automatically: - **QuantUMS** (`--quantums`): Requires >= 1.9.2. The `--direct-quant` flag is automatically skipped for DIA-NN 1.8.x where direct quantification is the only mode. -- **DDA mode** (`--diann_dda`): Requires >= 2.3.2. The pipeline will error if enabled with an older version. +- **DDA mode** (`--dda`): Requires >= 2.3.2. The pipeline will error if enabled with an older version. - **InfinDIA** (`--enable_infin_dia`): Requires >= 2.3.0. Usage: diff --git a/modules/local/diann/assemble_empirical_library/main.nf b/modules/local/diann/assemble_empirical_library/main.nf index 450cd4a..c99e539 100644 --- a/modules/local/diann/assemble_empirical_library/main.nf +++ b/modules/local/diann/assemble_empirical_library/main.nf @@ -34,7 +34,7 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { '--out-lib', '--use-quant', '--gen-spec-lib', '--rt-profiling', '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', '--channels', '--lib-fixed-mod', '--original-mods', - '--proteoforms', '--peptidoforms'] + '--proteoforms', '--peptidoforms', '--no-peptidoforms'] // Sort by length descending so longer flags (e.g. --mass-acc-ms1) are matched before shorter prefixes (--mass-acc) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -52,11 +52,10 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { mass_acc = '--individual-mass-acc' } scan_window = params.scan_window_automatic ? '--individual-windows' : "--window $params.scan_window" - diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' - diann_tims_sum = params.diann_tims_sum ? "--quant-tims-sum" : "" - diann_im_window = params.diann_im_window ? "--im-window $params.diann_im_window" : "" + diann_tims_sum = params.tims_sum ? "--quant-tims-sum" : "" + diann_im_window = params.im_window ? "--im-window $params.im_window" : "" diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" """ @@ -74,14 +73,13 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { --lib ${lib} \\ --threads ${task.cpus} \\ --out-lib empirical_library \\ - --verbose $params.diann_debug \\ + --verbose $params.debug_level \\ --rt-profiling \\ --temp ./quant/ \\ --use-quant \\ ${mass_acc} \\ ${scan_window} \\ --gen-spec-lib \\ - ${diann_no_peptidoforms} \\ ${scoring_mode} \\ ${diann_tims_sum} \\ ${diann_im_window} \\ diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index 58625a6..c0f95e6 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -51,7 +51,7 @@ process FINAL_QUANTIFICATION { '--species-genes', '--report-decoys', '--xic', '--no-norm', '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', '--export-quant', '--site-ms1-quant', '--channels', '--lib-fixed-mod', '--original-mods', - '--proteoforms', '--peptidoforms'] + '--proteoforms', '--peptidoforms', '--no-peptidoforms'] // Sort by length descending so longer flags (e.g. --individual-windows) are matched before shorter prefixes (--window) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -63,23 +63,22 @@ process FINAL_QUANTIFICATION { scan_window = params.scan_window_automatic ? "--individual-windows" : "--window $params.scan_window" species_genes = params.species_genes ? "--species-genes": "" - no_norm = params.diann_normalize ? "" : "--no-norm" - report_decoys = params.diann_report_decoys ? "--report-decoys": "" - diann_export_xic = params.diann_export_xic ? "--xic": "" + no_norm = params.normalize ? "" : "--no-norm" + report_decoys = params.report_decoys ? "--report-decoys": "" + diann_export_xic = params.export_xic ? "--xic": "" // --direct-quant exists in DIA-NN >= 1.9.2 (QuantUMS counterpart); skip for older versions quantums = params.quantums ? "" : (VersionUtils.versionAtLeast(params.diann_version, '1.9.2') ? "--direct-quant" : "") quantums_train_runs = params.quantums_train_runs ? "--quant-train-runs $params.quantums_train_runs": "" quantums_sel_runs = params.quantums_sel_runs ? "--quant-sel-runs $params.quantums_sel_runs": "" quantums_params = params.quantums_params ? "--quant-params $params.quantums_params": "" - diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' - diann_use_quant = params.diann_use_quant ? "--use-quant" : "" + diann_use_quant = params.use_quant ? "--use-quant" : "" diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" - diann_export_quant = params.diann_export_quant ? "--export-quant" : "" - diann_site_ms1_quant = params.diann_site_ms1_quant ? "--site-ms1-quant" : "" - diann_channel_run_norm = params.diann_channel_run_norm ? "--channel-run-norm" : "" - diann_channel_spec_norm = params.diann_channel_spec_norm ? "--channel-spec-norm" : "" + diann_export_quant = params.export_quant ? "--export-quant" : "" + diann_site_ms1_quant = params.site_ms1_quant ? "--site-ms1-quant" : "" + diann_channel_run_norm = params.channel_run_norm ? "--channel-run-norm" : "" + diann_channel_spec_norm = params.channel_spec_norm ? "--channel-spec-norm" : "" """ # Notes: if .quant files are passed, mzml/.d files are not accessed, so the name needs to be passed but files @@ -92,7 +91,7 @@ process FINAL_QUANTIFICATION { --fasta ${fasta} \\ --f ${(ms_files as List).join(' --f ')} \\ --threads ${task.cpus} \\ - --verbose $params.diann_debug \\ + --verbose $params.debug_level \\ --temp ./quant/ \\ --relaxed-prot-inf \\ --pg-level $params.pg_level \\ @@ -107,7 +106,6 @@ process FINAL_QUANTIFICATION { ${quantums_train_runs} \\ ${quantums_sel_runs} \\ ${quantums_params} \\ - ${diann_no_peptidoforms} \\ ${scoring_mode} \\ ${diann_use_quant} \\ ${diann_dda_flag} \\ diff --git a/modules/local/diann/individual_analysis/main.nf b/modules/local/diann/individual_analysis/main.nf index b6c1cf1..f82ebca 100644 --- a/modules/local/diann/individual_analysis/main.nf +++ b/modules/local/diann/individual_analysis/main.nf @@ -30,7 +30,7 @@ process INDIVIDUAL_ANALYSIS { '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', '--channels', '--lib-fixed-mod', '--original-mods', - '--proteoforms', '--peptidoforms'] + '--proteoforms', '--peptidoforms', '--no-peptidoforms'] // Sort by length descending so longer flags (e.g. --mass-acc-ms1) are matched before shorter prefixes (--mass-acc) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -81,11 +81,10 @@ process INDIVIDUAL_ANALYSIS { } } - diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' - diann_tims_sum = params.diann_tims_sum ? "--quant-tims-sum" : "" - diann_im_window = params.diann_im_window ? "--im-window $params.diann_im_window" : "" + diann_tims_sum = params.tims_sum ? "--quant-tims-sum" : "" + diann_im_window = params.im_window ? "--im-window $params.im_window" : "" diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" // Flags removed in DIA-NN 2.3.x — only pass for older versions @@ -106,7 +105,7 @@ process INDIVIDUAL_ANALYSIS { --f ${ms_file} \\ --fasta ${fasta} \\ --threads ${task.cpus} \\ - --verbose $params.diann_debug \\ + --verbose $params.debug_level \\ --temp ./ \\ --mass-acc ${mass_acc_ms2} \\ --mass-acc-ms1 ${mass_acc_ms1} \\ @@ -119,7 +118,6 @@ process INDIVIDUAL_ANALYSIS { ${max_pr_mz} \\ ${min_fr_mz} \\ ${max_fr_mz} \\ - ${diann_no_peptidoforms} \\ ${scoring_mode} \\ ${diann_tims_sum} \\ ${diann_im_window} \\ diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index 3c64f40..13d2262 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -32,7 +32,7 @@ process INSILICO_LIBRARY_GENERATION { '--min-pr-charge', '--max-pr-charge', '--var-mods', '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', '--met-excision', '--monitor-mod', '--dda', '--light-models', - '--infin-dia', '--pre-select', '--proteoforms', '--peptidoforms'] + '--infin-dia', '--pre-select', '--proteoforms', '--peptidoforms', '--no-peptidoforms'] // Sort by length descending so longer flags (e.g. --fasta-search) are matched before shorter prefixes (--fasta, --f) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -47,13 +47,12 @@ process INSILICO_LIBRARY_GENERATION { min_fr_mz = params.min_fr_mz ? "--min-fr-mz $params.min_fr_mz":"" max_fr_mz = params.max_fr_mz ? "--max-fr-mz $params.max_fr_mz":"" met_excision = params.met_excision ? "--met-excision" : "" - diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' diann_dda_flag = is_dda ? "--dda" : "" - diann_light_models = params.diann_light_models ? "--light-models" : "" + diann_light_models = params.light_models ? "--light-models" : "" infin_dia_flag = params.enable_infin_dia ? "--infin-dia" : "" - pre_select_flag = (params.enable_infin_dia && params.diann_pre_select) ? "--pre-select $params.diann_pre_select" : "" + pre_select_flag = (params.enable_infin_dia && params.pre_select) ? "--pre-select $params.pre_select" : "" """ diann `cat ${diann_config}` \\ @@ -71,9 +70,8 @@ process INSILICO_LIBRARY_GENERATION { --var-mods $params.max_mods \\ --threads ${task.cpus} \\ --predictor \\ - --verbose $params.diann_debug \\ + --verbose $params.debug_level \\ --gen-spec-lib \\ - ${diann_no_peptidoforms} \\ ${scoring_mode} \\ ${diann_light_models} \\ ${infin_dia_flag} \\ diff --git a/modules/local/diann/preliminary_analysis/main.nf b/modules/local/diann/preliminary_analysis/main.nf index 50f1aee..815b4bb 100644 --- a/modules/local/diann/preliminary_analysis/main.nf +++ b/modules/local/diann/preliminary_analysis/main.nf @@ -30,7 +30,7 @@ process PRELIMINARY_ANALYSIS { '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', '--monitor-mod', '--var-mod', '--fixed-mod', '--no-prot-inf', '--dda', '--channels', '--lib-fixed-mod', '--original-mods', - '--proteoforms', '--peptidoforms'] + '--proteoforms', '--peptidoforms', '--no-peptidoforms'] // Sort by length descending so longer flags (e.g. --mass-acc-ms1) are matched before shorter prefixes (--mass-acc) blocked.sort { a -> -a.length() }.each { flag -> def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' @@ -43,7 +43,6 @@ process PRELIMINARY_ANALYSIS { // Performance flags for preliminary analysis calibration step quick_mass_acc = params.quick_mass_acc ? "--quick-mass-acc" : "" performance_flags = params.performance_mode ? "--min-corr 2 --corr-diff 1 --time-corr-only" : "" - diann_no_peptidoforms = params.diann_no_peptidoforms ? "--no-peptidoforms" : "" scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' @@ -69,8 +68,8 @@ process PRELIMINARY_ANALYSIS { // Notes: Use double quotes for params, so that it is escaped in the shell. scan_window = params.scan_window_automatic ? '' : "--window $params.scan_window" - diann_tims_sum = params.diann_tims_sum ? "--quant-tims-sum" : "" - diann_im_window = params.diann_im_window ? "--im-window $params.diann_im_window" : "" + diann_tims_sum = params.tims_sum ? "--quant-tims-sum" : "" + diann_im_window = params.im_window ? "--im-window $params.im_window" : "" diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" // Per-file scan ranges from SDRF (empty = no flag, DIA-NN auto-detects) @@ -93,7 +92,7 @@ process PRELIMINARY_ANALYSIS { diann --lib ${predict_library} \\ --f ${ms_file} \\ --threads ${task.cpus} \\ - --verbose $params.diann_debug \\ + --verbose $params.debug_level \\ ${scan_window} \\ --temp ./ \\ ${mass_acc} \\ @@ -103,7 +102,6 @@ process PRELIMINARY_ANALYSIS { ${max_pr_mz} \\ ${min_fr_mz} \\ ${max_fr_mz} \\ - ${diann_no_peptidoforms} \\ ${scoring_mode} \\ ${diann_tims_sum} \\ ${diann_im_window} \\ diff --git a/nextflow.config b/nextflow.config index cb51d8c..22618c4 100644 --- a/nextflow.config +++ b/nextflow.config @@ -49,18 +49,18 @@ params { // DIA-NN: General diann_version = '1.8.1' // Used to control version-dependent flags (e.g. --monitor-mod for 1.8.x) - diann_debug = 3 - diann_speclib = null - diann_extra_args = null + debug_level = 3 + speclib = null + extra_args = null scoring_mode = 'generic' // Scoring mode: 'generic' (default), 'proteoforms' (variant detection, >= 2.0), 'peptidoforms' (PTM analysis) - diann_dda = false // Fallback: explicitly enable DDA when SDRF lacks acquisition method (requires DIA-NN >= 2.3.2) - diann_light_models = false // add '--light-models' for 10x faster library generation (DIA-NN >= 2.0) - diann_export_quant = false // add '--export-quant' for fragment-level parquet export (DIA-NN >= 2.0) - diann_site_ms1_quant = false // add '--site-ms1-quant' for MS1 apex PTM quantification (DIA-NN >= 2.0) + dda = false // Fallback: explicitly enable DDA when SDRF lacks acquisition method (requires DIA-NN >= 2.3.2) + light_models = false // add '--light-models' for 10x faster library generation (DIA-NN >= 2.0) + export_quant = false // add '--export-quant' for fragment-level parquet export (DIA-NN >= 2.0) + site_ms1_quant = false // add '--site-ms1-quant' for MS1 apex PTM quantification (DIA-NN >= 2.0) // DIA-NN: InfinDIA (experimental, v2.3.0+) enable_infin_dia = false // Enable InfinDIA for ultra-large search spaces - diann_pre_select = null // --pre-select N precursor limit for InfinDIA + pre_select = null // --pre-select N precursor limit for InfinDIA // Optional outputs — control which intermediate files are published save_speclib_tsv = false // Save the TSV spectral library from in-silico generation @@ -77,8 +77,8 @@ params { mass_acc_automatic = true performance_mode = true // add '--min-corr 2 --corr-diff 1 --time-corr-only' quick_mass_acc = true - diann_tims_sum = false // add '--quant-tims-sum' - diann_im_window = null // add '--im-window' + tims_sum = false // add '--quant-tims-sum' + im_window = null // add '--im-window' // DIA-NN: ASSEMBLE_EMPIRICAL_LIBRARY — library assembly skip_preliminary_analysis = false @@ -93,17 +93,16 @@ params { // DIA-NN: FINAL_QUANTIFICATION — summarization & output pg_level = 2 species_genes = false - diann_normalize = true - diann_report_decoys = false - diann_export_xic = false + normalize = true + report_decoys = false + export_xic = false quantums = false quantums_train_runs = null quantums_sel_runs = null quantums_params = null - diann_no_peptidoforms = false // add '--no-peptidoforms' - diann_use_quant = true // add '--use-quant' to FINAL_QUANTIFICATION - diann_channel_run_norm = false // add '--channel-run-norm' to FINAL_QUANTIFICATION - diann_channel_spec_norm = false // add '--channel-spec-norm' to FINAL_QUANTIFICATION + use_quant = true // add '--use-quant' to FINAL_QUANTIFICATION + channel_run_norm = false // add '--channel-run-norm' to FINAL_QUANTIFICATION + channel_spec_norm = false // add '--channel-spec-norm' to FINAL_QUANTIFICATION // pmultiqc options enable_pmultiqc = true diff --git a/nextflow_schema.json b/nextflow_schema.json index 6a358dc..abd548d 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -342,20 +342,20 @@ "description": "Instructs DIA-NN to add the organism identifier to the gene names", "fa_icon": "far fa-check-square" }, - "diann_speclib": { + "speclib": { "type": "string", "description": "The spectral library to use for DIA-NN", "fa_icon": "fas fa-file", "help_text": "If passed, will use that spectral library to carry out the DIA-NN search, instead of predicting one from the fasta file.", "hidden": false }, - "diann_report_decoys": { + "report_decoys": { "type": "boolean", "description": "Save decoy PSMs to the main .parquet report for DIA-NN 2.0.*", "fa_icon": "far fa-check-square", "hidden": false }, - "diann_export_xic": { + "export_xic": { "type": "boolean", "description": "instructs DIA-NN to extract MS1/fragment chromatograms for identified precursors within X seconds from the elution apex, with X set to 10s if not provided;equivalent to the 'XICs' option in the GUI", "fa_icon": "far fa-check-square", @@ -396,37 +396,31 @@ "fa_icon": "fas fa-bullseye", "help_text": "Controls DIA-NN's decoy generation strategy. In 'generic' mode, decoys are fully shuffled sequences. In 'proteoforms' mode, decoys differ by a single residue — directly modelling the confusion between canonical and variant peptides (e.g., COSMIC missense mutations). In 'peptidoforms' mode, generic q-values are calculated plus extra peptidoform-level q-values for PTM localization." }, - "diann_no_peptidoforms": { - "type": "boolean", - "description": "Set no-peptidoforms mode: disables automatic peptidoform scoring when variable modifications are declared (DIA-NN: not recommended).", - "fa_icon": "far fa-check-square", - "default": false - }, - "diann_tims_sum": { + "tims_sum": { "type": "boolean", "description": "Enable '--quant-tims-sum' for slice/scanning timsTOF methods (highly recommended for Synchro-PASEF).", "fa_icon": "far fa-check-square", "default": false }, - "diann_im_window": { + "im_window": { "type": "number", "description": "Set '--im-window' to ensure the IM extraction window is not smaller than the specified value.", "fa_icon": "fas fa-filter" }, - "diann_use_quant": { + "use_quant": { "type": "boolean", "description": "Set '--use-quant' to reuse existing .quant files if available.", "fa_icon": "far fa-check-square", "default": true }, - "diann_channel_run_norm": { + "channel_run_norm": { "type": "boolean", "description": "Set '--channel-run-norm'. Run-specific normalisation for multiplexing (e.g., Protein Turnover SILAC)", "fa_icon": "far fa-check-square", "default": false }, - "diann_channel_spec_norm": { + "channel_spec_norm": { "type": "boolean", "description": "Set '--channel-spec-norm'. Channel-specific normalisation for multiplexing (Independent Samples)", "fa_icon": "far fa-check-square", @@ -438,7 +432,7 @@ "fa_icon": "fas fa-forward", "hidden": false }, - "diann_debug": { + "debug_level": { "type": "integer", "description": "Debug level", "default": 3, @@ -446,7 +440,7 @@ "enum": [0, 1, 2, 3, 4], "hidden": true }, - "diann_normalize": { + "normalize": { "type": "boolean", "description": "Enable cross-run normalization between runs by diann.", "default": true, @@ -470,32 +464,32 @@ "fa_icon": "fas fa-filter", "hidden": true }, - "diann_extra_args": { + "extra_args": { "type": "string", "description": "Extra arguments appended to all DIA-NN steps. Flags incompatible with specific steps are automatically stripped with a warning.", "fa_icon": "fas fa-terminal", "hidden": false, "help_text": "Pass additional DIA-NN command-line arguments that will be appended to all DIA-NN steps (INSILICO_LIBRARY_GENERATION, PRELIMINARY_ANALYSIS, ASSEMBLE_EMPIRICAL_LIBRARY, INDIVIDUAL_ANALYSIS, FINAL_QUANTIFICATION). Flags that conflict with a specific step are automatically stripped with a warning. For step-specific overrides, use custom Nextflow config files with ext.args." }, - "diann_dda": { + "dda": { "type": "boolean", "description": "Explicitly enable DDA mode. Normally auto-detected from the SDRF acquisition method column. Use only when SDRF lacks this column. Requires DIA-NN >= 2.3.2.", "fa_icon": "fas fa-flask", "default": false }, - "diann_light_models": { + "light_models": { "type": "boolean", "description": "Enable --light-models for 10x faster in-silico library generation (DIA-NN >= 2.0).", "fa_icon": "fas fa-bolt", "default": false }, - "diann_export_quant": { + "export_quant": { "type": "boolean", "description": "Enable --export-quant for fragment-level parquet data export (DIA-NN >= 2.0).", "fa_icon": "fas fa-file-export", "default": false }, - "diann_site_ms1_quant": { + "site_ms1_quant": { "type": "boolean", "description": "Enable --site-ms1-quant to use MS1 apex intensities for PTM site quantification (DIA-NN >= 2.0).", "fa_icon": "fas fa-crosshairs", @@ -507,7 +501,7 @@ "fa_icon": "fas fa-infinity", "default": false }, - "diann_pre_select": { + "pre_select": { "type": "integer", "description": "Set --pre-select N precursor limit for InfinDIA pre-search.", "fa_icon": "fas fa-filter" diff --git a/subworkflows/local/create_input_channel/main.nf b/subworkflows/local/create_input_channel/main.nf index 2208eca..7b01106 100644 --- a/subworkflows/local/create_input_channel/main.nf +++ b/subworkflows/local/create_input_channel/main.nf @@ -72,14 +72,14 @@ def create_meta_channel_grouped(String filestr, List rows, Map wrapper) { exit(1, "ERROR: Please check input file -> File Uri does not exist!\n${filestr}") } - // Detect acquisition method from SDRF or fallback to --diann_dda param + // Detect acquisition method from SDRF or fallback to --dda param def acqMethod = base_row.AcquisitionMethod?.toString()?.trim() ?: "" if (acqMethod.toLowerCase().contains("data-independent acquisition") || acqMethod.toLowerCase().contains("dia")) { meta.acquisition_method = "dia" } else if (acqMethod.toLowerCase().contains("data-dependent acquisition") || acqMethod.toLowerCase().contains("dda")) { meta.acquisition_method = "dda" } else if (acqMethod.isEmpty()) { - meta.acquisition_method = params.diann_dda ? "dda" : "dia" + meta.acquisition_method = params.dda ? "dda" : "dia" } else { log.error("Unsupported acquisition method: '${acqMethod}'. This pipeline supports DIA and DDA. Found in file: ${filestr}") exit(1) diff --git a/workflows/dia.nf b/workflows/dia.nf index b4781ec..291c782 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -36,8 +36,8 @@ workflow DIA { ch_software_versions = channel.empty() // Version guard for DDA mode (when explicitly set via param) - if (params.diann_dda && VersionUtils.versionLessThan(params.diann_version, '2.3.2')) { - error("DDA mode (--diann_dda) requires DIA-NN >= 2.3.2. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") + if (params.dda && VersionUtils.versionLessThan(params.diann_version, '2.3.2')) { + error("DDA mode (--dda) requires DIA-NN >= 2.3.2. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") } // Version guard for InfinDIA @@ -51,11 +51,11 @@ workflow DIA { } // Version guard for DIA-NN 2.0+ features - if ((params.diann_light_models || params.diann_export_quant || params.diann_site_ms1_quant) && VersionUtils.versionLessThan(params.diann_version, '2.0')) { + if ((params.light_models || params.export_quant || params.site_ms1_quant) && VersionUtils.versionLessThan(params.diann_version, '2.0')) { def enabled = [] - if (params.diann_light_models) enabled << '--light-models' - if (params.diann_export_quant) enabled << '--export-quant' - if (params.diann_site_ms1_quant) enabled << '--site-ms1-quant' + if (params.light_models) enabled << '--light-models' + if (params.export_quant) enabled << '--export-quant' + if (params.site_ms1_quant) enabled << '--site-ms1-quant' error("${enabled.join(', ')} require DIA-NN >= 2.0. Current version: ${params.diann_version}. Use -profile diann_v2_1_0 or later") } @@ -75,7 +75,7 @@ workflow DIA { // Determine DDA mode: true if explicitly set via param OR auto-detected from SDRF ch_is_dda = ch_experiment_meta.map { meta -> - def dda = params.diann_dda || meta.acquisition_method == 'dda' + def dda = params.dda || meta.acquisition_method == 'dda' if (dda && VersionUtils.versionLessThan(params.diann_version, '2.3.2')) { error("DDA mode (detected from SDRF) requires DIA-NN >= 2.3.2. Current version: ${params.diann_version}. Use -profile diann_v2_3_2") } @@ -89,8 +89,8 @@ workflow DIA { // // MODULE: SILICOLIBRARYGENERATION // - if (params.diann_speclib != null && params.diann_speclib.toString() != "") { - speclib = channel.from(file(params.diann_speclib, checkIfExists: true)) + if (params.speclib != null && params.speclib.toString() != "") { + speclib = channel.from(file(params.speclib, checkIfExists: true)) } else { INSILICO_LIBRARY_GENERATION(ch_searchdb, ch_diann_cfg_val, ch_is_dda) speclib = INSILICO_LIBRARY_GENERATION.out.predict_speclib From 2ef313811c3d130c3e6816368918b53c2dd06054 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Fri, 10 Apr 2026 13:35:03 +0100 Subject: [PATCH 46/69] style: format AGENTS.md with prettier Co-Authored-By: Claude Opus 4.6 (1M context) --- AGENTS.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a051f4e..15e7c11 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -109,13 +109,13 @@ The pipeline executes the following steps: ### DIA-NN Version-Specific Features -| Feature | Min Version | Parameter | -| ------------------------------------------- | ----------- | ---------------------------- | -| Core workflow, library-free, .quant caching | 1.8.1 | (default) | -| QuantUMS quantification | 1.9.2 | `--quantums true` | -| Parquet output format | 2.0 | (automatic in 2.0+) | +| Feature | Min Version | Parameter | +| ------------------------------------------- | ----------- | ---------------------- | +| Core workflow, library-free, .quant caching | 1.8.1 | (default) | +| QuantUMS quantification | 1.9.2 | `--quantums true` | +| Parquet output format | 2.0 | (automatic in 2.0+) | | Decoy reporting | 2.0 | `--report_decoys true` | -| Native .raw on Linux | 2.1.0 | (automatic) | +| Native .raw on Linux | 2.1.0 | (automatic) | --- From 00fc343fe5629d5473966fb6a28ae2ff0aa8af66 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Sat, 11 Apr 2026 06:36:42 +0100 Subject: [PATCH 47/69] refactor: centralize blocked flags in lib/BlockedFlags.groovy Replace duplicated blocked-flags logic (10+ lines x 5 modules) with a single centralized registry in lib/BlockedFlags.groovy. Each module now uses one line: `args = BlockedFlags.strip('MODULE_NAME', args, log)` Fixes from PR #36 review: - Add --no-prot-inf to ASSEMBLE, INDIVIDUAL, FINAL blocked lists - Add --channel-run-norm, --channel-spec-norm to FINAL blocked list - Add --var-mod, --fixed-mod, --channels to INSILICO (via COMMON) - Add --relaxed-prot-inf, --pg-level to ASSEMBLE blocked list - Add version guard for channel normalization flags (require >= 2.0) - Add warning when --normalize false conflicts with channel norm flags Co-Authored-By: Claude Opus 4.6 (1M context) --- lib/BlockedFlags.groovy | 93 +++++++++++++++++++ .../diann/assemble_empirical_library/main.nf | 18 +--- .../local/diann/final_quantification/main.nf | 18 +--- .../local/diann/individual_analysis/main.nf | 18 +--- .../diann/insilico_library_generation/main.nf | 18 +--- .../local/diann/preliminary_analysis/main.nf | 18 +--- workflows/dia.nf | 10 +- 7 files changed, 107 insertions(+), 86 deletions(-) create mode 100644 lib/BlockedFlags.groovy diff --git a/lib/BlockedFlags.groovy b/lib/BlockedFlags.groovy new file mode 100644 index 0000000..522f849 --- /dev/null +++ b/lib/BlockedFlags.groovy @@ -0,0 +1,93 @@ +/** + * Centralised registry of DIA-NN flags managed by the pipeline. + * + * Each DIA-NN module has a set of flags that the pipeline controls directly. + * If a user passes any of these via --extra_args, they are stripped with a warning + * to prevent silent conflicts. + * + * Nextflow auto-loads all classes in lib/, so this is available in all modules. + */ +class BlockedFlags { + + // Flags common to ALL DIA-NN steps + private static final List COMMON = [ + '--temp', '--threads', '--verbose', '--lib', '--f', '--fasta', + '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', + '--channels', '--lib-fixed-mod', '--original-mods', + '--proteoforms', '--peptidoforms', '--no-peptidoforms', + ] + + // Per-module additional blocked flags (on top of COMMON) + private static final Map> MODULE_FLAGS = [ + INSILICO_LIBRARY_GENERATION: [ + '--use-quant', '--no-main-report', '--matrices', '--out', + '--fasta-search', '--predictor', '--gen-spec-lib', + '--missed-cleavages', '--min-pep-len', '--max-pep-len', + '--min-pr-charge', '--max-pr-charge', '--var-mods', + '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', + '--met-excision', '--light-models', + '--infin-dia', '--pre-select', + ], + PRELIMINARY_ANALYSIS: [ + '--use-quant', '--gen-spec-lib', '--out-lib', '--matrices', '--out', + '--mass-acc', '--mass-acc-ms1', '--window', + '--quick-mass-acc', '--min-corr', '--corr-diff', '--time-corr-only', + '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', + '--no-prot-inf', + ], + ASSEMBLE_EMPIRICAL_LIBRARY: [ + '--no-main-report', '--no-ifs-removal', '--matrices', '--out', + '--mass-acc', '--mass-acc-ms1', '--window', + '--individual-mass-acc', '--individual-windows', + '--out-lib', '--use-quant', '--gen-spec-lib', '--rt-profiling', + '--no-prot-inf', '--relaxed-prot-inf', '--pg-level', + ], + INDIVIDUAL_ANALYSIS: [ + '--use-quant', '--gen-spec-lib', '--out-lib', '--matrices', '--out', '--rt-profiling', + '--mass-acc', '--mass-acc-ms1', '--window', + '--no-ifs-removal', '--no-main-report', '--relaxed-prot-inf', '--pg-level', + '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', + '--no-prot-inf', + ], + FINAL_QUANTIFICATION: [ + '--no-main-report', '--gen-spec-lib', '--out-lib', '--no-ifs-removal', + '--use-quant', '--matrices', '--out', '--relaxed-prot-inf', '--pg-level', + '--qvalue', '--matrix-qvalue', '--matrix-spec-q', '--window', '--individual-windows', + '--species-genes', '--report-decoys', '--xic', '--no-norm', + '--export-quant', '--site-ms1-quant', + '--channel-run-norm', '--channel-spec-norm', + '--no-prot-inf', + ], + ] + + /** + * Get the full blocked flags list for a module (COMMON + module-specific). + */ + static List forModule(String moduleName) { + def moduleFlags = MODULE_FLAGS[moduleName] ?: [] + return (COMMON + moduleFlags).unique() + } + + /** + * Strip blocked flags from an args string, logging a warning for each. + * Returns the cleaned args string. + * + * @param moduleName e.g. 'FINAL_QUANTIFICATION' + * @param args the raw args string from task.ext.args + * @param log the Nextflow log object + * @return cleaned args string + */ + static String strip(String moduleName, String args, def log) { + if (!args) return '' + def blocked = forModule(moduleName) + // Sort by length descending so longer flags (e.g. --mass-acc-ms1) match before shorter prefixes (--mass-acc) + blocked.sort { a -> -a.length() }.each { flag -> + def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' + if (args =~ flagPattern) { + log.warn "DIA-NN: '${flag}' is managed by the pipeline for ${moduleName} and will be stripped." + args = args.replaceAll(flagPattern, '').trim() + } + } + return args + } +} diff --git a/modules/local/diann/assemble_empirical_library/main.nf b/modules/local/diann/assemble_empirical_library/main.nf index 2323cb2..0a01608 100644 --- a/modules/local/diann/assemble_empirical_library/main.nf +++ b/modules/local/diann/assemble_empirical_library/main.nf @@ -26,23 +26,7 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { script: def args = task.ext.args ?: '' - // Strip flags that are managed by the pipeline to prevent silent conflicts - def blocked = ['--no-main-report', '--no-ifs-removal', '--matrices', '--out', - '--temp', '--threads', '--verbose', '--lib', '--f', '--fasta', - '--mass-acc', '--mass-acc-ms1', '--window', - '--individual-mass-acc', '--individual-windows', - '--out-lib', '--use-quant', '--gen-spec-lib', '--rt-profiling', - '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', - '--channels', '--lib-fixed-mod', '--original-mods', - '--proteoforms', '--peptidoforms', '--no-peptidoforms'] - // Sort by length descending so longer flags (e.g. --mass-acc-ms1) are matched before shorter prefixes (--mass-acc) - blocked.sort { a -> -a.length() }.each { flag -> - def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' - if (args =~ flagPattern) { - log.warn "DIA-NN: '${flag}' is managed by the pipeline for ASSEMBLE_EMPIRICAL_LIBRARY and will be stripped." - args = args.replaceAll(flagPattern, '').trim() - } - } + args = BlockedFlags.strip('ASSEMBLE_EMPIRICAL_LIBRARY', args, log) if (params.mass_acc_automatic) { mass_acc = '--individual-mass-acc' diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index 8f517c1..da135e3 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -45,23 +45,7 @@ process FINAL_QUANTIFICATION { script: def args = task.ext.args ?: '' - // Strip flags that are managed by the pipeline to prevent silent conflicts - def blocked = ['--no-main-report', '--gen-spec-lib', '--out-lib', '--no-ifs-removal', - '--temp', '--threads', '--verbose', '--lib', '--f', '--fasta', - '--use-quant', '--matrices', '--out', '--relaxed-prot-inf', '--pg-level', - '--qvalue', '--matrix-qvalue', '--matrix-spec-q', '--window', '--individual-windows', - '--species-genes', '--report-decoys', '--xic', '--no-norm', - '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', '--export-quant', '--site-ms1-quant', - '--channels', '--lib-fixed-mod', '--original-mods', - '--proteoforms', '--peptidoforms', '--no-peptidoforms'] - // Sort by length descending so longer flags (e.g. --individual-windows) are matched before shorter prefixes (--window) - blocked.sort { a -> -a.length() }.each { flag -> - def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' - if (args =~ flagPattern) { - log.warn "DIA-NN: '${flag}' is managed by the pipeline for FINAL_QUANTIFICATION and will be stripped." - args = args.replaceAll(flagPattern, '').trim() - } - } + args = BlockedFlags.strip('FINAL_QUANTIFICATION', args, log) scan_window = params.scan_window_automatic ? "--individual-windows" : "--window $params.scan_window" species_genes = params.species_genes ? "--species-genes": "" diff --git a/modules/local/diann/individual_analysis/main.nf b/modules/local/diann/individual_analysis/main.nf index e53676c..e263b16 100644 --- a/modules/local/diann/individual_analysis/main.nf +++ b/modules/local/diann/individual_analysis/main.nf @@ -22,23 +22,7 @@ process INDIVIDUAL_ANALYSIS { script: def args = task.ext.args ?: '' - // Strip flags that are managed by the pipeline to prevent silent conflicts - def blocked = ['--use-quant', '--gen-spec-lib', '--out-lib', '--matrices', '--out', '--rt-profiling', - '--temp', '--threads', '--verbose', '--lib', '--f', '--fasta', - '--mass-acc', '--mass-acc-ms1', '--window', - '--no-ifs-removal', '--no-main-report', '--relaxed-prot-inf', '--pg-level', - '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', - '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', - '--channels', '--lib-fixed-mod', '--original-mods', - '--proteoforms', '--peptidoforms', '--no-peptidoforms'] - // Sort by length descending so longer flags (e.g. --mass-acc-ms1) are matched before shorter prefixes (--mass-acc) - blocked.sort { a -> -a.length() }.each { flag -> - def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' - if (args =~ flagPattern) { - log.warn "DIA-NN: '${flag}' is managed by the pipeline for INDIVIDUAL_ANALYSIS and will be stripped." - args = args.replaceAll(flagPattern, '').trim() - } - } + args = BlockedFlags.strip('INDIVIDUAL_ANALYSIS', args, log) // Warn about flags that override pipeline-computed calibration values (not blocked, but may change behaviour) ['--individual-windows', '--individual-mass-acc'].each { flag -> diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index 7ca1723..7005c2a 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -24,23 +24,7 @@ process INSILICO_LIBRARY_GENERATION { script: def args = task.ext.args ?: '' - // Strip flags that are managed by the pipeline to prevent silent conflicts - def blocked = ['--use-quant', '--no-main-report', '--matrices', '--out', - '--temp', '--threads', '--verbose', '--lib', '--f', '--fasta', - '--fasta-search', '--predictor', '--gen-spec-lib', - '--missed-cleavages', '--min-pep-len', '--max-pep-len', - '--min-pr-charge', '--max-pr-charge', '--var-mods', - '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', - '--met-excision', '--monitor-mod', '--dda', '--light-models', - '--infin-dia', '--pre-select', '--proteoforms', '--peptidoforms', '--no-peptidoforms'] - // Sort by length descending so longer flags (e.g. --fasta-search) are matched before shorter prefixes (--fasta, --f) - blocked.sort { a -> -a.length() }.each { flag -> - def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' - if (args =~ flagPattern) { - log.warn "DIA-NN: '${flag}' is managed by the pipeline for INSILICO_LIBRARY_GENERATION and will be stripped." - args = args.replaceAll(flagPattern, '').trim() - } - } + args = BlockedFlags.strip('INSILICO_LIBRARY_GENERATION', args, log) min_pr_mz = params.min_pr_mz ? "--min-pr-mz $params.min_pr_mz":"" max_pr_mz = params.max_pr_mz ? "--max-pr-mz $params.max_pr_mz":"" diff --git a/modules/local/diann/preliminary_analysis/main.nf b/modules/local/diann/preliminary_analysis/main.nf index ad44527..41373e6 100644 --- a/modules/local/diann/preliminary_analysis/main.nf +++ b/modules/local/diann/preliminary_analysis/main.nf @@ -22,23 +22,7 @@ process PRELIMINARY_ANALYSIS { script: def args = task.ext.args ?: '' - // Strip flags that are managed by the pipeline to prevent silent conflicts - def blocked = ['--use-quant', '--gen-spec-lib', '--out-lib', '--matrices', '--out', - '--temp', '--threads', '--verbose', '--lib', '--f', '--fasta', - '--mass-acc', '--mass-acc-ms1', '--window', - '--quick-mass-acc', '--min-corr', '--corr-diff', '--time-corr-only', - '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', - '--monitor-mod', '--var-mod', '--fixed-mod', '--no-prot-inf', '--dda', - '--channels', '--lib-fixed-mod', '--original-mods', - '--proteoforms', '--peptidoforms', '--no-peptidoforms'] - // Sort by length descending so longer flags (e.g. --mass-acc-ms1) are matched before shorter prefixes (--mass-acc) - blocked.sort { a -> -a.length() }.each { flag -> - def flagPattern = '(?<=^|\\s)' + java.util.regex.Pattern.quote(flag) + '(?=\\s|\$)(\\s+(?!-{1,2}[a-zA-Z])\\S+)*' - if (args =~ flagPattern) { - log.warn "DIA-NN: '${flag}' is managed by the pipeline for PRELIMINARY_ANALYSIS and will be stripped." - args = args.replaceAll(flagPattern, '').trim() - } - } + args = BlockedFlags.strip('PRELIMINARY_ANALYSIS', args, log) // Performance flags for preliminary analysis calibration step quick_mass_acc = params.quick_mass_acc ? "--quick-mass-acc" : "" diff --git a/workflows/dia.nf b/workflows/dia.nf index 028906e..6c53625 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -51,14 +51,22 @@ workflow DIA { } // Version guard for DIA-NN 2.0+ features - if ((params.light_models || params.export_quant || params.site_ms1_quant) && VersionUtils.versionLessThan(params.diann_version, '2.0')) { + if ((params.light_models || params.export_quant || params.site_ms1_quant || params.channel_run_norm || params.channel_spec_norm) && VersionUtils.versionLessThan(params.diann_version, '2.0')) { def enabled = [] if (params.light_models) enabled << '--light-models' if (params.export_quant) enabled << '--export-quant' if (params.site_ms1_quant) enabled << '--site-ms1-quant' + if (params.channel_run_norm) enabled << '--channel-run-norm' + if (params.channel_spec_norm) enabled << '--channel-spec-norm' error("${enabled.join(', ')} require DIA-NN >= 2.0. Current version: ${params.diann_version}. Use -profile diann_v2_1_0 or later") } + // Warn about contradictory normalization flags + if (!params.normalize && (params.channel_run_norm || params.channel_spec_norm)) { + log.warn "Both --normalize false (adds --no-norm) and channel normalization flags are set. " + + "These may conflict — --no-norm disables cross-run normalization while channel normalization requires it." + } + ch_searchdb = channel.fromPath(params.database, checkIfExists: true) .ifEmpty { error("No protein database found at '${params.database}'. Provide --database ") } .first() From 22ace63ff8a7e9d966432eb74dafa3d570313523 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Sat, 11 Apr 2026 06:47:00 +0100 Subject: [PATCH 48/69] docs: document BlockedFlags design rationale and usage MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add comments explaining: - Why blocked flags exist (prevent silent DIA-NN flag conflicts) - Why they live in a Groovy class, not config files (safety — can't be overridden by user configs) - How to add new blocked flags (edit one file, no module changes) - In each module: point developers to lib/BlockedFlags.groovy Co-Authored-By: Claude Opus 4.6 (1M context) --- lib/BlockedFlags.groovy | 22 ++++++++++++++++--- .../diann/assemble_empirical_library/main.nf | 2 ++ .../local/diann/final_quantification/main.nf | 2 ++ .../local/diann/individual_analysis/main.nf | 2 ++ .../diann/insilico_library_generation/main.nf | 2 ++ .../local/diann/preliminary_analysis/main.nf | 2 ++ 6 files changed, 29 insertions(+), 3 deletions(-) diff --git a/lib/BlockedFlags.groovy b/lib/BlockedFlags.groovy index 522f849..daedbbd 100644 --- a/lib/BlockedFlags.groovy +++ b/lib/BlockedFlags.groovy @@ -1,9 +1,25 @@ /** * Centralised registry of DIA-NN flags managed by the pipeline. * - * Each DIA-NN module has a set of flags that the pipeline controls directly. - * If a user passes any of these via --extra_args, they are stripped with a warning - * to prevent silent conflicts. + * WHY THIS EXISTS: + * The pipeline controls certain DIA-NN flags directly (e.g. --threads, --out, --qvalue) + * based on pipeline parameters, SDRF metadata, or step-specific logic. If a user also + * passes these via --extra_args, the flag would appear twice in the DIA-NN command — + * causing silent conflicts or undefined behaviour. This registry strips managed flags + * from extra_args with a warning, so the pipeline's values always take precedence. + * + * WHY A GROOVY CLASS AND NOT CONFIG FILES: + * Blocked flags are a safety mechanism. Defining them in Nextflow config (e.g. via + * ext.blocked_flags) would allow users to accidentally override or disable them in + * custom configs. A compiled Groovy class in lib/ cannot be overridden by user configs, + * ensuring the safety net is always active. It also provides a single file to audit + * and edit when adding new managed flags. + * + * HOW TO ADD A NEW BLOCKED FLAG: + * - If the flag applies to ALL DIA-NN steps, add it to the COMMON list below. + * - If it applies to specific steps only, add it to the relevant MODULE_FLAGS entry. + * - No changes needed in the module .nf files — they call BlockedFlags.strip() which + * reads from this registry automatically. * * Nextflow auto-loads all classes in lib/, so this is available in all modules. */ diff --git a/modules/local/diann/assemble_empirical_library/main.nf b/modules/local/diann/assemble_empirical_library/main.nf index 0a01608..70ef8aa 100644 --- a/modules/local/diann/assemble_empirical_library/main.nf +++ b/modules/local/diann/assemble_empirical_library/main.nf @@ -26,6 +26,8 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { script: def args = task.ext.args ?: '' + // Strip flags managed by the pipeline from extra_args to prevent silent conflicts. + // Blocked flags are defined centrally in lib/BlockedFlags.groovy — edit there, not here. args = BlockedFlags.strip('ASSEMBLE_EMPIRICAL_LIBRARY', args, log) if (params.mass_acc_automatic) { diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index da135e3..48404b2 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -45,6 +45,8 @@ process FINAL_QUANTIFICATION { script: def args = task.ext.args ?: '' + // Strip flags managed by the pipeline from extra_args to prevent silent conflicts. + // Blocked flags are defined centrally in lib/BlockedFlags.groovy — edit there, not here. args = BlockedFlags.strip('FINAL_QUANTIFICATION', args, log) scan_window = params.scan_window_automatic ? "--individual-windows" : "--window $params.scan_window" diff --git a/modules/local/diann/individual_analysis/main.nf b/modules/local/diann/individual_analysis/main.nf index e263b16..6f2027d 100644 --- a/modules/local/diann/individual_analysis/main.nf +++ b/modules/local/diann/individual_analysis/main.nf @@ -22,6 +22,8 @@ process INDIVIDUAL_ANALYSIS { script: def args = task.ext.args ?: '' + // Strip flags managed by the pipeline from extra_args to prevent silent conflicts. + // Blocked flags are defined centrally in lib/BlockedFlags.groovy — edit there, not here. args = BlockedFlags.strip('INDIVIDUAL_ANALYSIS', args, log) // Warn about flags that override pipeline-computed calibration values (not blocked, but may change behaviour) diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index 7005c2a..d7e695e 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -24,6 +24,8 @@ process INSILICO_LIBRARY_GENERATION { script: def args = task.ext.args ?: '' + // Strip flags managed by the pipeline from extra_args to prevent silent conflicts. + // Blocked flags are defined centrally in lib/BlockedFlags.groovy — edit there, not here. args = BlockedFlags.strip('INSILICO_LIBRARY_GENERATION', args, log) min_pr_mz = params.min_pr_mz ? "--min-pr-mz $params.min_pr_mz":"" diff --git a/modules/local/diann/preliminary_analysis/main.nf b/modules/local/diann/preliminary_analysis/main.nf index 41373e6..a4616ea 100644 --- a/modules/local/diann/preliminary_analysis/main.nf +++ b/modules/local/diann/preliminary_analysis/main.nf @@ -22,6 +22,8 @@ process PRELIMINARY_ANALYSIS { script: def args = task.ext.args ?: '' + // Strip flags managed by the pipeline from extra_args to prevent silent conflicts. + // Blocked flags are defined centrally in lib/BlockedFlags.groovy — edit there, not here. args = BlockedFlags.strip('PRELIMINARY_ANALYSIS', args, log) // Performance flags for preliminary analysis calibration step From 39531925af32b3a7270dfc1f601fd3b9e25265a4 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Sat, 11 Apr 2026 06:57:44 +0100 Subject: [PATCH 49/69] fix: remove protein inference flags from ASSEMBLE blocked list ASSEMBLE_EMPIRICAL_LIBRARY doesn't set --no-prot-inf, --relaxed-prot-inf, or --pg-level in its command, so blocking them would strip user values without providing a replacement. Users should be able to pass these to the assembly step if needed. Co-Authored-By: Claude Opus 4.6 (1M context) --- lib/BlockedFlags.groovy | 1 - 1 file changed, 1 deletion(-) diff --git a/lib/BlockedFlags.groovy b/lib/BlockedFlags.groovy index daedbbd..9614240 100644 --- a/lib/BlockedFlags.groovy +++ b/lib/BlockedFlags.groovy @@ -56,7 +56,6 @@ class BlockedFlags { '--mass-acc', '--mass-acc-ms1', '--window', '--individual-mass-acc', '--individual-windows', '--out-lib', '--use-quant', '--gen-spec-lib', '--rt-profiling', - '--no-prot-inf', '--relaxed-prot-inf', '--pg-level', ], INDIVIDUAL_ANALYSIS: [ '--use-quant', '--gen-spec-lib', '--out-lib', '--matrices', '--out', '--rt-profiling', From fa687108a5f5d43455b32e1efbef3497bb5cad6f Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Sat, 11 Apr 2026 07:02:03 +0100 Subject: [PATCH 50/69] docs: add rationale comments for all blocked flags MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Each blocked flag now has a comment explaining WHY it's blocked: - "Pipeline-managed": the pipeline sets it from params/SDRF/metadata - "No-effect guard": the flag has no effect in this step but is blocked to prevent users from wrongly believing it does This prevents future contributors (human or AI) from removing flags without understanding the intent behind the block. Reverts the accidental removal of protein inference flags from ASSEMBLE_EMPIRICAL_LIBRARY — they are intentionally blocked as a no-effect guard since --gen-spec-lib produces a spectral library, not a quantified report. Co-Authored-By: Claude Opus 4.6 (1M context) --- lib/BlockedFlags.groovy | 41 ++++++++++++++++++++++++++++++++++++----- 1 file changed, 36 insertions(+), 5 deletions(-) diff --git a/lib/BlockedFlags.groovy b/lib/BlockedFlags.groovy index 9614240..c11d297 100644 --- a/lib/BlockedFlags.groovy +++ b/lib/BlockedFlags.groovy @@ -25,7 +25,13 @@ */ class BlockedFlags { - // Flags common to ALL DIA-NN steps + // Flags common to ALL DIA-NN steps. + // These are always set by the pipeline from config/SDRF, so user overrides would conflict. + // --temp/--threads/--verbose: controlled by Nextflow task resources and debug_level param + // --lib/--f/--fasta: set from workflow inputs (speclib, ms_files, database) + // --var-mod/--fixed-mod/--monitor-mod/--channels/--lib-fixed-mod/--original-mods: injected from diann_config.cfg + // --dda: auto-detected from SDRF or set via --dda param + // --proteoforms/--peptidoforms/--no-peptidoforms: controlled by scoring_mode param private static final List COMMON = [ '--temp', '--threads', '--verbose', '--lib', '--f', '--fasta', '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', @@ -33,9 +39,19 @@ class BlockedFlags { '--proteoforms', '--peptidoforms', '--no-peptidoforms', ] - // Per-module additional blocked flags (on top of COMMON) + // Per-module additional blocked flags (on top of COMMON). + // + // Flags are blocked for two reasons: + // 1. "Pipeline-managed": the pipeline sets them explicitly from params/SDRF/metadata. + // Allowing user overrides would create duplicate/conflicting flags. + // 2. "No-effect guard": the flag has no effect in this step (e.g., protein inference + // during library assembly). Blocking prevents users from passing them and wrongly + // believing they had an effect. + // + // When reason is non-obvious, a comment explains why the flag is blocked. private static final Map> MODULE_FLAGS = [ INSILICO_LIBRARY_GENERATION: [ + // Pipeline-managed: set from params (library generation settings) '--use-quant', '--no-main-report', '--matrices', '--out', '--fasta-search', '--predictor', '--gen-spec-lib', '--missed-cleavages', '--min-pep-len', '--max-pep-len', @@ -45,29 +61,44 @@ class BlockedFlags { '--infin-dia', '--pre-select', ], PRELIMINARY_ANALYSIS: [ + // Pipeline-managed: set from params and SDRF calibration metadata '--use-quant', '--gen-spec-lib', '--out-lib', '--matrices', '--out', '--mass-acc', '--mass-acc-ms1', '--window', '--quick-mass-acc', '--min-corr', '--corr-diff', '--time-corr-only', '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', + // Pipeline-managed: preliminary step disables protein inference (--no-prot-inf) '--no-prot-inf', ], ASSEMBLE_EMPIRICAL_LIBRARY: [ + // Pipeline-managed: set from params and calibration results '--no-main-report', '--no-ifs-removal', '--matrices', '--out', '--mass-acc', '--mass-acc-ms1', '--window', '--individual-mass-acc', '--individual-windows', '--out-lib', '--use-quant', '--gen-spec-lib', '--rt-profiling', + // No-effect guard: protein inference flags have no effect during library assembly + // (--gen-spec-lib produces a spectral library, not a quantified report). + // Blocked to prevent users from thinking they affect this step. + '--no-prot-inf', '--relaxed-prot-inf', '--pg-level', ], INDIVIDUAL_ANALYSIS: [ + // Pipeline-managed: set from params and calibrated values from assembly step '--use-quant', '--gen-spec-lib', '--out-lib', '--matrices', '--out', '--rt-profiling', '--mass-acc', '--mass-acc-ms1', '--window', - '--no-ifs-removal', '--no-main-report', '--relaxed-prot-inf', '--pg-level', + '--no-ifs-removal', '--no-main-report', + // Pipeline-managed: protein inference set by pipeline (--relaxed-prot-inf --pg-level) + '--relaxed-prot-inf', '--pg-level', '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', '--no-prot-inf', ], FINAL_QUANTIFICATION: [ + // Pipeline-managed: set from params for final report generation '--no-main-report', '--gen-spec-lib', '--out-lib', '--no-ifs-removal', - '--use-quant', '--matrices', '--out', '--relaxed-prot-inf', '--pg-level', - '--qvalue', '--matrix-qvalue', '--matrix-spec-q', '--window', '--individual-windows', + '--use-quant', '--matrices', '--out', + // Pipeline-managed: protein inference (--relaxed-prot-inf --pg-level) + '--relaxed-prot-inf', '--pg-level', + // Pipeline-managed: FDR controls (precursor_qvalue, matrix_qvalue, matrix_spec_q) + '--qvalue', '--matrix-qvalue', '--matrix-spec-q', + '--window', '--individual-windows', '--species-genes', '--report-decoys', '--xic', '--no-norm', '--export-quant', '--site-ms1-quant', '--channel-run-norm', '--channel-spec-norm', From 85b83dbd56b68e7c001a94fb304ddaf42043cb4a Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Sun, 12 Apr 2026 06:49:04 +0100 Subject: [PATCH 51/69] chore: bump version to 2.0.0 (Rome) and update CHANGELOG Co-Authored-By: Claude Opus 4.6 (1M context) --- CHANGELOG.md | 56 +++++++++++++++++++++++++++++++++++++++++++++++++ nextflow.config | 2 +- 2 files changed, 57 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fff751a..9c61306 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,62 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [2.0.0] bigbio/quantmsdiann — Rome - 2026-04-12 + +### `Added` + +- DDA analysis mode support (requires DIA-NN >= 2.3.2), auto-detected from SDRF or set via `--dda` +- DIA-NN 2.3.2 version profile (`-profile diann_v2_3_2`) with InfinDIA and DDA support +- Scoring mode parameter (`--scoring_mode`): `generic` (default), `proteoforms` (proteogenomics/variant detection, >= 2.0), `peptidoforms` (PTM analysis) +- FDR controls: `--precursor_qvalue`, `--matrix_qvalue`, `--matrix_spec_q` replacing the misleadingly named `protein_level_fdr_cutoff` +- Channel normalization flags: `--channel_run_norm`, `--channel_spec_norm` for multiplexing workflows (plexDIA/SILAC) +- InfinDIA support: `--enable_infin_dia`, `--pre_select` for ultra-large search spaces (DIA-NN >= 2.3.0) +- Fragment-level export: `--export_quant` for parquet fragment data (DIA-NN >= 2.0) +- MS1 PTM quantification: `--site_ms1_quant` (DIA-NN >= 2.0) +- Skip preliminary analysis: `--skip_preliminary_analysis` to use a provided spectral library directly +- Centralized blocked-flags registry (`lib/BlockedFlags.groovy`) replacing duplicated per-module logic +- Version guards for all DIA-NN version-dependent features with clear error messages +- `VersionUtils.groovy` for semantic version comparison (replaces fragile string comparisons) +- Log capture via `tee` fallback in all DIA-NN modules for robust log handling across versions +- CI: `test_dda` and `test_dia_skip_preanalysis` test profiles + +### `Changed` + +- Removed `diann_` prefix from all user-facing parameters (e.g., `--diann_report_decoys` → `--report_decoys`, `--diann_normalize` → `--normalize`) +- Removed `diann_no_peptidoforms` parameter (superseded by `--scoring_mode`) +- Separated FINAL_QUANTIFICATION report output into parquet and TSV channels (DIA-NN >= 1.9 produces parquet) +- Matrix/stats outputs in FINAL_QUANTIFICATION now `optional: true` for DIA-NN 2.x compatibility +- Decoupled container engine from DIA-NN version configs (engine selected via `-profile`, not version config) +- Removed hardcoded DIA-NN container from `pride_codon_slurm.config` +- Updated Zenodo DOI to 10.5281/zenodo.19437128 + +### `Removed` + +- `tdf2mzml` module and all references (Bruker .d files handled natively by DIA-NN >= 2.0) +- `protein_level_fdr_cutoff` parameter (replaced by `precursor_qvalue`) +- `diann_no_peptidoforms` parameter (replaced by `scoring_mode`) + +### `Fixed` + +- DIA-NN log file handling for versions >= 2.x that don't produce `*.log.txt` files +- `DIANN_MSSTATS` receiving both parquet and TSV when both exist (now only one format reaches downstream) +- Missing blocked flags: `--no-prot-inf` in ASSEMBLE/INDIVIDUAL/FINAL, `--channel-run-norm`/`--channel-spec-norm` in FINAL, `--var-mod`/`--fixed-mod`/`--channels` in INSILICO +- Unnecessary `.first()` warning on `ch_sdrf` value channel +- `--out` added to in-silico library generation to produce a log file with DIA-NN 2.3.2 + +### `Dependencies` + +| Dependency | Version | +| --------------------- | --------- | +| `nextflow` | >=25.04.0 | +| `dia-nn` | 1.8.1+ | +| `thermorawfileparser` | 2.0.0.dev | +| `sdrf-pipelines` | 0.1.2 | +| `pmultiqc` | 0.0.43 | +| `quantms-utils` | 0.0.28 | + +--- + ## [1.0.0] bigbio/quantmsdiann - 2026-04-03 Initial release of the standalone DIA-NN quantitative proteomics pipeline, refactored from [bigbio/quantms](https://github.com/bigbio/quantms). diff --git a/nextflow.config b/nextflow.config index c658e34..180ba8f 100644 --- a/nextflow.config +++ b/nextflow.config @@ -377,7 +377,7 @@ manifest { mainScript = 'main.nf' defaultBranch = 'main' nextflowVersion = '!>=25.04.0' - version = '2.0.0dev' + version = '2.0.0' doi = '10.5281/zenodo.15573386' } From 89b5a9c2c9ebcf9ccd79ddbe0c8943e2bff22038 Mon Sep 17 00:00:00 2001 From: yueqixuan Date: Mon, 13 Apr 2026 15:48:48 +0800 Subject: [PATCH 52/69] update --- CHANGELOG.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9c61306..db463a8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,7 +3,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [2.0.0] bigbio/quantmsdiann — Rome - 2026-04-12 +## [2.0.0] bigbio/quantmsdiann — Rome - 2026-04-13 ### `Added` @@ -54,8 +54,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 | `dia-nn` | 1.8.1+ | | `thermorawfileparser` | 2.0.0.dev | | `sdrf-pipelines` | 0.1.2 | -| `pmultiqc` | 0.0.43 | -| `quantms-utils` | 0.0.28 | +| `pmultiqc` | 0.0.44 | +| `quantms-utils` | 0.0.29 | --- From c9581149b3fa53c8eb1215e5cef767e0c7ea5b0a Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Mon, 13 Apr 2026 15:28:49 +0100 Subject: [PATCH 53/69] feat: add DIA-NN 2.5.0 support with model fine-tuning docs - New v2_5_0.config version profile (ghcr.io/bigbio/diann:2.5.0) - Add diann_v2_5_0 profile to nextflow.config - Block --parent flag in BlockedFlags.groovy COMMON list (container-managed, overriding breaks model discovery) - Document fine-tuning workflow in docs/usage.md: - How to generate tuning libraries - How to fine-tune RT/IM/fragment models - How to use fine-tuned models via --extra_args - Update version tables in docs/usage.md and AGENTS.md New DIA-NN 2.5.0 CLI flags (passable via --extra_args): --tokens, --rt-model, --fr-model, --im-model (model selection) --aa-eq (amino acid equivalence for reannotation) --tune-lib, --tune-rt, --tune-im, --tune-fr (fine-tuning) Co-Authored-By: Claude Opus 4.6 (1M context) --- AGENTS.md | 9 +++-- conf/diann_versions/v2_5_0.config | 14 ++++++++ docs/usage.md | 60 +++++++++++++++++++++++++++---- lib/BlockedFlags.groovy | 3 +- nextflow.config | 1 + 5 files changed, 78 insertions(+), 9 deletions(-) create mode 100644 conf/diann_versions/v2_5_0.config diff --git a/AGENTS.md b/AGENTS.md index 15e7c11..67852f5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -24,7 +24,7 @@ This is **non-negotiable**. All code must pass formatting and style checks befor - Built with Nextflow DSL2 - DIA-NN for peptide/protein identification and quantification -- Supports DIA-NN v1.8.1, v2.1.0, and v2.2.0 (latest) +- Supports DIA-NN v1.8.1, v2.1.0, v2.2.0, v2.3.2, and v2.5.0 (latest) - QuantUMS quantification method (DIA-NN >= 1.9.2) - Parquet-native output with decoy reporting (DIA-NN >= 2.0) - MSstats-compatible output generation (via quantms-utils conversion, no MSstats analysis) @@ -44,7 +44,7 @@ This is **non-negotiable**. All code must pass formatting and style checks befor - **nf-test**: Testing framework (config: `nf-test.config`) - **nf-core tools**: Pipeline standards and linting - **Containers**: Docker/Singularity/Apptainer/Podman (Conda deprecated) -- **DIA-NN**: Primary search engine (versions 1.8.1 through 2.2.0) +- **DIA-NN**: Primary search engine (versions 1.8.1 through 2.5.0) ### Key Configuration Files @@ -116,6 +116,9 @@ The pipeline executes the following steps: | Parquet output format | 2.0 | (automatic in 2.0+) | | Decoy reporting | 2.0 | `--report_decoys true` | | Native .raw on Linux | 2.1.0 | (automatic) | +| DDA support | 2.3.2 | `--dda true` | +| InfinDIA | 2.3.0 | `--enable_infin_dia` | +| DL model fine-tuning & selection | 2.5.0 | `--extra_args` | --- @@ -194,6 +197,8 @@ These apply on top of test profiles to override the DIA-NN container version: | `diann_v1_8_1` | `biocontainers/diann:v1.8.1_cv1` | none | | `diann_v2_1_0` | `ghcr.io/bigbio/diann:2.1.0` | GHCR | | `diann_v2_2_0` | `ghcr.io/bigbio/diann:2.2.0` | GHCR | +| `diann_v2_3_2` | `ghcr.io/bigbio/diann:2.3.2` | GHCR | +| `diann_v2_5_0` | `ghcr.io/bigbio/diann:2.5.0` | GHCR | ### CI Workflows diff --git a/conf/diann_versions/v2_5_0.config b/conf/diann_versions/v2_5_0.config new file mode 100644 index 0000000..1e584d4 --- /dev/null +++ b/conf/diann_versions/v2_5_0.config @@ -0,0 +1,14 @@ +/* + * DIA-NN 2.5.0 container override (private ghcr.io) + * Major protein ID improvements (+70%), DL model selection flags, + * new --aa-eq flag for amino acid equivalence. + */ +params.diann_version = '2.5.0' + +process { + withLabel: diann { + container = 'ghcr.io/bigbio/diann:2.5.0' + } +} + +// Container engine is selected via -profile (docker/singularity), not here diff --git a/docs/usage.md b/docs/usage.md index 510a026..e01fdcc 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -120,6 +120,7 @@ The default DIA-NN version is 1.8.1. To use a different version: | 2.1.0 | `-profile diann_v2_1_0` | Native .raw support, reduced memory | | 2.2.0 | `-profile diann_v2_2_0` | Speed optimizations | | 2.3.2 | `-profile diann_v2_3_2` | DDA support, InfinDIA | +| 2.5.0 | `-profile diann_v2_5_0` | +70% protein IDs, model fine-tuning | Example: `nextflow run bigbio/quantmsdiann -profile test_dia,docker,diann_v2_2_0` @@ -318,12 +319,13 @@ process { The pipeline supports multiple DIA-NN versions via built-in Nextflow profiles. Each profile sets `params.diann_version` and overrides the container image for all `diann`-labelled processes. -| Profile | DIA-NN Version | Container | Key features | -| -------------- | -------------- | ------------------------------------------ | -------------------------------------------------------------- | -| `diann_v1_8_1` | 1.8.1 | `docker.io/biocontainers/diann:v1.8.1_cv1` | Default. Public BioContainers image. TSV output. | -| `diann_v2_1_0` | 2.1.0 | `ghcr.io/bigbio/diann:2.1.0` | Parquet output. Native .raw on Linux. QuantUMS (`--quantums`). | -| `diann_v2_2_0` | 2.2.0 | `ghcr.io/bigbio/diann:2.2.0` | Speed optimizations (up to 1.6x on HPC). Parquet output. | -| `diann_v2_3_2` | 2.3.2 | `ghcr.io/bigbio/diann:2.3.2` | DDA support (`--dda`), InfinDIA, up to 9 variable mods. | +| Profile | DIA-NN Version | Container | Key features | +| -------------- | -------------- | ------------------------------------------ | --------------------------------------------------------------- | +| `diann_v1_8_1` | 1.8.1 | `docker.io/biocontainers/diann:v1.8.1_cv1` | Default. Public BioContainers image. TSV output. | +| `diann_v2_1_0` | 2.1.0 | `ghcr.io/bigbio/diann:2.1.0` | Parquet output. Native .raw on Linux. QuantUMS (`--quantums`). | +| `diann_v2_2_0` | 2.2.0 | `ghcr.io/bigbio/diann:2.2.0` | Speed optimizations (up to 1.6x on HPC). Parquet output. | +| `diann_v2_3_2` | 2.3.2 | `ghcr.io/bigbio/diann:2.3.2` | DDA support (`--dda`), InfinDIA, up to 9 variable mods. | +| `diann_v2_5_0` | 2.5.0 | `ghcr.io/bigbio/diann:2.5.0` | Up to 70% more protein IDs. DL model fine-tuning and selection. | **Version-dependent features:** Some parameters are only available with newer DIA-NN versions. The pipeline handles version compatibility automatically: @@ -348,6 +350,52 @@ nextflow run bigbio/quantmsdiann \ > [!NOTE] > DIA-NN 2.x images are hosted on `ghcr.io/bigbio` and may require authentication for private registries. The `diann_v2_1_0` and `diann_v2_2_0` profiles force Docker mode by default; for Singularity, override with your own config. +## Fine-Tuning Deep Learning Models (DIA-NN 2.0+) + +DIA-NN uses deep learning models to predict retention time (RT), ion mobility (IM), and fragment ion intensities. For non-standard modifications (beyond Oxidation, Phospho, Carbamidomethyl, etc.), fine-tuning these models can substantially improve detection. + +**When to fine-tune:** Fine-tuning is beneficial for custom chemical labels (e.g., mTRAQ, dimethyl), exotic PTMs, or unmodified cysteines. Standard modifications (Phospho, Oxidation, Acetylation, Deamidation, diGlycine) do not require fine-tuning. + +### Step 1: Generate a tuning library + +Run the pipeline normally on raw files containing the modifications of interest. The empirical library produced by the ASSEMBLE_EMPIRICAL_LIBRARY step can serve as the tuning library. + +### Step 2: Fine-tune models (outside the pipeline) + +Fine-tuning is currently a separate step run outside quantmsdiann: + +```bash +# Fine-tune RT and IM models using the empirical library +diann --tune-lib empirical_library.parquet --tune-rt --tune-im + +# Optionally also fine-tune the fragmentation model (quality-sensitive) +diann --tune-lib empirical_library.parquet --tune-rt --tune-im --tune-fr +``` + +DIA-NN will produce these output files (named after the input library): + +- `empirical_library.dict.txt` — tokenizer dictionary mapping modifications to neural network tokens +- `empirical_library.rt.d0.pt` (+ `.d1.pt`, `.d2.pt`) — fine-tuned RT models +- `empirical_library.im.d0.pt` (+ `.d1.pt`, `.d2.pt`) — fine-tuned IM models +- `empirical_library.fr.d0.pt` (+ `.d1.pt`, `.d2.pt`) — fine-tuned fragment models (if `--tune-fr` used) + +Additional tuning parameters: `--tune-lr` (learning rate, default 0.0005), `--tune-restrict-layers` (fix RNN weights), `--tune-level` (limit to specific distillation level 0/1/2). + +### Step 3: Use fine-tuned models in the pipeline + +Pass the fine-tuned model files to quantmsdiann via `--extra_args`: + +```bash +nextflow run bigbio/quantmsdiann \ + -profile diann_v2_5_0,docker \ + --input sdrf.tsv --database db.fasta \ + --extra_args "--tokens /path/to/empirical_library.dict.txt --rt-model /path/to/empirical_library.rt.d0.pt --im-model /path/to/empirical_library.im.d0.pt" \ + --outdir results +``` + +> [!IMPORTANT] +> Use **absolute paths** for model files. The `--tokens`, `--rt-model`, `--fr-model`, and `--im-model` flags are passed through to all DIA-NN steps via `--extra_args`. The `--parent` flag is blocked by the pipeline (it controls the container's DIA-NN installation path). + ## Verbose Module Output By default, only final result files are published. For debugging or detailed inspection, the `verbose_modules` profile publishes all intermediate files from every DIA-NN step: diff --git a/lib/BlockedFlags.groovy b/lib/BlockedFlags.groovy index c11d297..2fcadcd 100644 --- a/lib/BlockedFlags.groovy +++ b/lib/BlockedFlags.groovy @@ -32,9 +32,10 @@ class BlockedFlags { // --var-mod/--fixed-mod/--monitor-mod/--channels/--lib-fixed-mod/--original-mods: injected from diann_config.cfg // --dda: auto-detected from SDRF or set via --dda param // --proteoforms/--peptidoforms/--no-peptidoforms: controlled by scoring_mode param + // --parent: container-managed (DIA-NN model path), overriding breaks model discovery private static final List COMMON = [ '--temp', '--threads', '--verbose', '--lib', '--f', '--fasta', - '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', + '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', '--parent', '--channels', '--lib-fixed-mod', '--original-mods', '--proteoforms', '--peptidoforms', '--no-peptidoforms', ] diff --git a/nextflow.config b/nextflow.config index c658e34..35996d3 100644 --- a/nextflow.config +++ b/nextflow.config @@ -256,6 +256,7 @@ profiles { diann_v2_1_0 { includeConfig 'conf/diann_versions/v2_1_0.config' } diann_v2_2_0 { includeConfig 'conf/diann_versions/v2_2_0.config' } diann_v2_3_2 { includeConfig 'conf/diann_versions/v2_3_2.config' } + diann_v2_5_0 { includeConfig 'conf/diann_versions/v2_5_0.config' } dev { includeConfig 'conf/dev.config' } pride_slurm { includeConfig 'conf/pride_codon_slurm.config' } manual_wave { includeConfig 'conf/wave.config' } From a8bbaed9b8768ca8d5e1d7a73a09b60456b6376c Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Mon, 13 Apr 2026 15:30:50 +0100 Subject: [PATCH 54/69] docs: add DIA-NN 2.5.0 fine-tuning flags to parameters.md Document all new DIA-NN 2.5.0 flags available via --extra_args: model selection (--tokens, --rt-model, --fr-model, --im-model), fine-tuning (--tune-lib, --tune-rt, --tune-im, --tune-fr, etc.), and --aa-eq for amino acid equivalence. Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/parameters.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/docs/parameters.md b/docs/parameters.md index 8623fe2..7541dba 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -63,6 +63,27 @@ This document lists every pipeline parameter organised by category. Default valu | `--export_quant` | boolean | `false` | Enable `--export-quant` for fragment-level parquet data export. Requires DIA-NN >= 2.0. | | `--site_ms1_quant` | boolean | `false` | Enable `--site-ms1-quant` to use MS1 apex intensities for PTM site quantification. Requires DIA-NN >= 2.0. | +### DIA-NN 2.5.0 flags (via `--extra_args`) + +The following DIA-NN 2.5.0 flags are not exposed as pipeline parameters but can be passed via `--extra_args`. See [Fine-Tuning Deep Learning Models](usage.md#fine-tuning-deep-learning-models-dia-nn-20) for the complete workflow. + +| DIA-NN flag | Description | +| ------------------------ | ------------------------------------------------------------------------------------------------ | +| `--tokens ` | Tokenizer dictionary mapping modified residues to neural network token IDs (0-255). | +| `--rt-model ` | Fine-tuned retention time prediction model (`.pt` PyTorch file). | +| `--fr-model ` | Fine-tuned fragment ion prediction model (`.pt` file). Quality-sensitive — verify vs base model. | +| `--im-model ` | Fine-tuned ion mobility prediction model (`.pt` file). | +| `--tune-lib ` | Spectral library for fine-tuning (`.parquet`). Requires `--tune-rt` and/or `--tune-im`. | +| `--tune-rt` | Fine-tune the RT deep learning predictor. Requires `--tune-lib`. | +| `--tune-im` | Fine-tune the IM deep learning predictor. Requires `--tune-lib`. | +| `--tune-fr` | Fine-tune the fragmentation predictor. Requires `--tune-lib`. Use with caution. | +| `--tune-lr ` | Fine-tuning learning rate (default: 0.0005). | +| `--tune-restrict-layers` | Keep RNN layer weights fixed during fine-tuning (except cysteine embeddings). | +| `--tune-level ` | Limit fine-tuning to a specific model distillation level (0, 1, or 2). | +| `--aa-eq` | Treat I&L, Q&E, N&D as equivalent during reannotation. Essential for entrapment FDR benchmarks. | + +> **Note:** `--parent` is blocked by the pipeline — it controls the container's DIA-NN installation path and overriding it would break model discovery. + ## 6. Mass Accuracy & Calibration | Parameter | Type | Default | Description | From 82651863f4c14e487c53be9b4a1b41c44543ddd4 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Mon, 13 Apr 2026 16:31:00 +0100 Subject: [PATCH 55/69] feat: expose --aa-eq as pipeline parameter (aa_eq) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add aa_eq param (default: false) that maps to DIA-NN's --aa-eq flag. When enabled, I&L, Q&E, N&D are treated as equivalent amino acids during reannotation — essential for entrapment FDR benchmarks. - Added to nextflow.config, nextflow_schema.json, docs/parameters.md - Passed to all 5 DIA-NN modules - Added to BlockedFlags.groovy COMMON list (pipeline-managed) - Moved from "via extra_args" to proper pipeline parameter in docs Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/parameters.md | 4 ++-- lib/BlockedFlags.groovy | 3 ++- modules/local/diann/assemble_empirical_library/main.nf | 2 ++ modules/local/diann/final_quantification/main.nf | 2 ++ modules/local/diann/individual_analysis/main.nf | 2 ++ modules/local/diann/insilico_library_generation/main.nf | 2 ++ modules/local/diann/preliminary_analysis/main.nf | 2 ++ nextflow.config | 1 + nextflow_schema.json | 7 +++++++ 9 files changed, 22 insertions(+), 3 deletions(-) diff --git a/docs/parameters.md b/docs/parameters.md index 7541dba..2ed864c 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -62,6 +62,7 @@ This document lists every pipeline parameter organised by category. Default valu | `--light_models` | boolean | `false` | Enable `--light-models` for 10x faster in-silico library generation. Requires DIA-NN >= 2.0. | | `--export_quant` | boolean | `false` | Enable `--export-quant` for fragment-level parquet data export. Requires DIA-NN >= 2.0. | | `--site_ms1_quant` | boolean | `false` | Enable `--site-ms1-quant` to use MS1 apex intensities for PTM site quantification. Requires DIA-NN >= 2.0. | +| `--aa_eq` | boolean | `false` | Treat I&L, Q&E, N&D as equivalent amino acids during reannotation. Essential for entrapment FDR benchmarks. Maps to `--aa-eq`. | ### DIA-NN 2.5.0 flags (via `--extra_args`) @@ -80,9 +81,8 @@ The following DIA-NN 2.5.0 flags are not exposed as pipeline parameters but can | `--tune-lr ` | Fine-tuning learning rate (default: 0.0005). | | `--tune-restrict-layers` | Keep RNN layer weights fixed during fine-tuning (except cysteine embeddings). | | `--tune-level ` | Limit fine-tuning to a specific model distillation level (0, 1, or 2). | -| `--aa-eq` | Treat I&L, Q&E, N&D as equivalent during reannotation. Essential for entrapment FDR benchmarks. | -> **Note:** `--parent` is blocked by the pipeline — it controls the container's DIA-NN installation path and overriding it would break model discovery. +> **Note:** `--parent` and `--aa-eq` are blocked from `--extra_args` — they are managed as pipeline parameters (`--aa_eq`). `--parent` is container-managed and overriding it would break model discovery. ## 6. Mass Accuracy & Calibration diff --git a/lib/BlockedFlags.groovy b/lib/BlockedFlags.groovy index 2fcadcd..b9c8518 100644 --- a/lib/BlockedFlags.groovy +++ b/lib/BlockedFlags.groovy @@ -33,9 +33,10 @@ class BlockedFlags { // --dda: auto-detected from SDRF or set via --dda param // --proteoforms/--peptidoforms/--no-peptidoforms: controlled by scoring_mode param // --parent: container-managed (DIA-NN model path), overriding breaks model discovery + // --aa-eq: controlled by aa_eq param private static final List COMMON = [ '--temp', '--threads', '--verbose', '--lib', '--f', '--fasta', - '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', '--parent', + '--monitor-mod', '--var-mod', '--fixed-mod', '--dda', '--parent', '--aa-eq', '--channels', '--lib-fixed-mod', '--original-mods', '--proteoforms', '--peptidoforms', '--no-peptidoforms', ] diff --git a/modules/local/diann/assemble_empirical_library/main.nf b/modules/local/diann/assemble_empirical_library/main.nf index 70ef8aa..a1cb2ca 100644 --- a/modules/local/diann/assemble_empirical_library/main.nf +++ b/modules/local/diann/assemble_empirical_library/main.nf @@ -40,6 +40,7 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { scan_window = params.scan_window_automatic ? '--individual-windows' : "--window $params.scan_window" scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' + aa_eq = params.aa_eq ? '--aa-eq' : '' diann_tims_sum = params.tims_sum ? "--quant-tims-sum" : "" diann_im_window = params.im_window ? "--im-window $params.im_window" : "" diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" @@ -67,6 +68,7 @@ process ASSEMBLE_EMPIRICAL_LIBRARY { ${scan_window} \\ --gen-spec-lib \\ ${scoring_mode} \\ + ${aa_eq} \\ ${diann_tims_sum} \\ ${diann_im_window} \\ ${diann_dda_flag} \\ diff --git a/modules/local/diann/final_quantification/main.nf b/modules/local/diann/final_quantification/main.nf index 48404b2..fcbd4ca 100644 --- a/modules/local/diann/final_quantification/main.nf +++ b/modules/local/diann/final_quantification/main.nf @@ -61,6 +61,7 @@ process FINAL_QUANTIFICATION { quantums_params = params.quantums_params ? "--quant-params $params.quantums_params": "" scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' + aa_eq = params.aa_eq ? '--aa-eq' : '' diann_use_quant = params.use_quant ? "--use-quant" : "" diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" diann_export_quant = params.export_quant ? "--export-quant" : "" @@ -97,6 +98,7 @@ process FINAL_QUANTIFICATION { ${quantums_sel_runs} \\ ${quantums_params} \\ ${scoring_mode} \\ + ${aa_eq} \\ ${diann_use_quant} \\ ${diann_dda_flag} \\ ${diann_export_quant} \\ diff --git a/modules/local/diann/individual_analysis/main.nf b/modules/local/diann/individual_analysis/main.nf index 6f2027d..814895f 100644 --- a/modules/local/diann/individual_analysis/main.nf +++ b/modules/local/diann/individual_analysis/main.nf @@ -69,6 +69,7 @@ process INDIVIDUAL_ANALYSIS { scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' + aa_eq = params.aa_eq ? '--aa-eq' : '' diann_tims_sum = params.tims_sum ? "--quant-tims-sum" : "" diann_im_window = params.im_window ? "--im-window $params.im_window" : "" diann_dda_flag = meta.acquisition_method == 'dda' ? "--dda" : "" @@ -105,6 +106,7 @@ process INDIVIDUAL_ANALYSIS { ${min_fr_mz} \\ ${max_fr_mz} \\ ${scoring_mode} \\ + ${aa_eq} \\ ${diann_tims_sum} \\ ${diann_im_window} \\ ${diann_dda_flag} \\ diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index d7e695e..97aef4f 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -35,6 +35,7 @@ process INSILICO_LIBRARY_GENERATION { met_excision = params.met_excision ? "--met-excision" : "" scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' + aa_eq = params.aa_eq ? '--aa-eq' : '' diann_dda_flag = is_dda ? "--dda" : "" diann_light_models = params.light_models ? "--light-models" : "" infin_dia_flag = params.enable_infin_dia ? "--infin-dia" : "" @@ -60,6 +61,7 @@ process INSILICO_LIBRARY_GENERATION { --verbose $params.debug_level \\ --gen-spec-lib \\ ${scoring_mode} \\ + ${aa_eq} \\ ${diann_light_models} \\ ${infin_dia_flag} \\ ${pre_select_flag} \\ diff --git a/modules/local/diann/preliminary_analysis/main.nf b/modules/local/diann/preliminary_analysis/main.nf index a4616ea..c09d41a 100644 --- a/modules/local/diann/preliminary_analysis/main.nf +++ b/modules/local/diann/preliminary_analysis/main.nf @@ -31,6 +31,7 @@ process PRELIMINARY_ANALYSIS { performance_flags = params.performance_mode ? "--min-corr 2 --corr-diff 1 --time-corr-only" : "" scoring_mode = params.scoring_mode == 'proteoforms' ? '--proteoforms' : params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' + aa_eq = params.aa_eq ? '--aa-eq' : '' // I am using here the ["key"] syntax, since the preprocessed meta makes // was evaluating to null when using the dot notation. @@ -89,6 +90,7 @@ process PRELIMINARY_ANALYSIS { ${min_fr_mz} \\ ${max_fr_mz} \\ ${scoring_mode} \\ + ${aa_eq} \\ ${diann_tims_sum} \\ ${diann_im_window} \\ --no-prot-inf \\ diff --git a/nextflow.config b/nextflow.config index 35996d3..9c52ae6 100644 --- a/nextflow.config +++ b/nextflow.config @@ -55,6 +55,7 @@ params { speclib = null extra_args = null scoring_mode = 'generic' // Scoring mode: 'generic' (default), 'proteoforms' (variant detection, >= 2.0), 'peptidoforms' (PTM analysis) + aa_eq = false // add '--aa-eq': treat I&L, Q&E, N&D as equivalent during reannotation (essential for entrapment FDR benchmarks) dda = false // Fallback: explicitly enable DDA when SDRF lacks acquisition method (requires DIA-NN >= 2.3.2) light_models = false // add '--light-models' for 10x faster library generation (DIA-NN >= 2.0) export_quant = false // add '--export-quant' for fragment-level parquet export (DIA-NN >= 2.0) diff --git a/nextflow_schema.json b/nextflow_schema.json index 8314f39..8f26fcb 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -401,6 +401,13 @@ "fa_icon": "fas fa-bullseye", "help_text": "Controls DIA-NN's decoy generation strategy. In 'generic' mode, decoys are fully shuffled sequences. In 'proteoforms' mode, decoys differ by a single residue — directly modelling the confusion between canonical and variant peptides (e.g., COSMIC missense mutations). In 'peptidoforms' mode, generic q-values are calculated plus extra peptidoform-level q-values for PTM localization." }, + "aa_eq": { + "type": "boolean", + "default": false, + "description": "Treat I&L, Q&E, N&D as equivalent amino acids during reannotation. Essential for entrapment FDR benchmarks.", + "fa_icon": "fas fa-equals", + "help_text": "Maps to DIA-NN's --aa-eq flag. When enabled, peptides are matched to proteins considering isoleucine/leucine, glutamine/glutamic acid, and asparagine/aspartic acid as equivalent. This is essential when benchmarking FDR using entrapment databases, as without it the assessment measures the ability to distinguish deamidated peptides rather than true FDR." + }, "tims_sum": { "type": "boolean", "description": "Enable '--quant-tims-sum' for slice/scanning timsTOF methods (highly recommended for Synchro-PASEF).", From eb784e1e77f72fd59ab88b7115aceb550ab9e0c5 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Mon, 13 Apr 2026 20:37:08 +0100 Subject: [PATCH 56/69] docs: improve fine-tuning section with complete workflow and question for Vadim Rewrite the fine-tuning documentation to: - Explain what tokens/dict.txt are (neural network encoding of modifications) - Clarify that --tune-lib cannot be combined with --gen-spec-lib (separate DIA-NN invocations, confirmed in DIA-NN #1499) - Document the full two-run workflow with concrete commands - Propose the future integrated FINE_TUNE_MODELS step with a question for @vdemichev about the right integration point Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/usage.md | 74 +++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 54 insertions(+), 20 deletions(-) diff --git a/docs/usage.md b/docs/usage.md index e01fdcc..b85b323 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -352,49 +352,83 @@ nextflow run bigbio/quantmsdiann \ ## Fine-Tuning Deep Learning Models (DIA-NN 2.0+) -DIA-NN uses deep learning models to predict retention time (RT), ion mobility (IM), and fragment ion intensities. For non-standard modifications (beyond Oxidation, Phospho, Carbamidomethyl, etc.), fine-tuning these models can substantially improve detection. +DIA-NN uses deep learning models to predict retention time (RT), ion mobility (IM), and fragment ion intensities. For non-standard modifications, fine-tuning these models on real data can substantially improve detection. -**When to fine-tune:** Fine-tuning is beneficial for custom chemical labels (e.g., mTRAQ, dimethyl), exotic PTMs, or unmodified cysteines. Standard modifications (Phospho, Oxidation, Acetylation, Deamidation, diGlycine) do not require fine-tuning. +**When to fine-tune:** Fine-tuning is beneficial for custom chemical labels (e.g., mTRAQ, dimethyl), exotic PTMs, or unmodified cysteines. Standard modifications (Phospho, Oxidation, Acetylation, Deamidation, diGlycine) do not require fine-tuning — DIA-NN's built-in models already handle them well. -### Step 1: Generate a tuning library +### How fine-tuning works -Run the pipeline normally on raw files containing the modifications of interest. The empirical library produced by the ASSEMBLE_EMPIRICAL_LIBRARY step can serve as the tuning library. +DIA-NN's neural networks encode each amino acid and modification as a "token" — an integer ID (0-255) mapped in a dictionary file (`dict.txt`). The default dictionary ships with DIA-NN and covers common modifications. When you fine-tune, DIA-NN: -### Step 2: Fine-tune models (outside the pipeline) +1. Reads a spectral library containing empirically observed peptides with the modifications of interest +2. Learns how those modifications affect RT, IM, and fragmentation patterns +3. Outputs new model files (`.pt` PyTorch format) and an expanded dictionary (`dict.txt`) that includes tokens for the new modifications -Fine-tuning is currently a separate step run outside quantmsdiann: +The fine-tuned models are then used in place of the defaults when generating predicted spectral libraries. + +> [!NOTE] +> **`--tune-lib` cannot be combined with `--gen-spec-lib` in a single DIA-NN invocation** ([confirmed in DIA-NN #1499](https://github.com/vdemichev/DiaNN/issues/1499)). Fine-tuning and library generation are separate DIA-NN commands. This means the workflow currently requires two pipeline runs. + +### Current workflow (manual fine-tuning) + +**Run 1 — Generate the tuning library:** + +Run quantmsdiann normally. The empirical library produced by the ASSEMBLE_EMPIRICAL_LIBRARY step (after preliminary analysis) serves as the tuning library. This library contains empirically observed RT, IM, and fragment intensities for peptides bearing the modifications of interest. + +```bash +# First run: standard pipeline to produce empirical library +nextflow run bigbio/quantmsdiann \ + -profile diann_v2_5_0,docker \ + --input sdrf.tsv --database db.fasta --outdir results_run1 +# Output: results_run1/library_generation/assemble_empirical_library/empirical_library.parquet +``` + +**Fine-tune models (outside the pipeline):** ```bash # Fine-tune RT and IM models using the empirical library -diann --tune-lib empirical_library.parquet --tune-rt --tune-im +diann --tune-lib /abs/path/to/empirical_library.parquet --tune-rt --tune-im -# Optionally also fine-tune the fragmentation model (quality-sensitive) -diann --tune-lib empirical_library.parquet --tune-rt --tune-im --tune-fr +# Optionally also fine-tune the fragmentation model (quality-sensitive — verify vs base model) +diann --tune-lib /abs/path/to/empirical_library.parquet --tune-rt --tune-im --tune-fr ``` -DIA-NN will produce these output files (named after the input library): +DIA-NN will output (named after the input library): -- `empirical_library.dict.txt` — tokenizer dictionary mapping modifications to neural network tokens -- `empirical_library.rt.d0.pt` (+ `.d1.pt`, `.d2.pt`) — fine-tuned RT models +- `empirical_library.dict.txt` — expanded tokenizer dictionary with new modification tokens +- `empirical_library.rt.d0.pt` (+ `.d1.pt`, `.d2.pt`) — fine-tuned RT models (3 distillation levels) - `empirical_library.im.d0.pt` (+ `.d1.pt`, `.d2.pt`) — fine-tuned IM models -- `empirical_library.fr.d0.pt` (+ `.d1.pt`, `.d2.pt`) — fine-tuned fragment models (if `--tune-fr` used) - -Additional tuning parameters: `--tune-lr` (learning rate, default 0.0005), `--tune-restrict-layers` (fix RNN weights), `--tune-level` (limit to specific distillation level 0/1/2). +- `empirical_library.fr.d0.pt` (+ `.d1.pt`, `.d2.pt`) — fine-tuned fragment models (if `--tune-fr`) -### Step 3: Use fine-tuned models in the pipeline +Additional tuning parameters: `--tune-lr` (learning rate, default 0.0005), `--tune-restrict-layers` (fix RNN weights), `--tune-level` (limit to a specific distillation level 0/1/2). -Pass the fine-tuned model files to quantmsdiann via `--extra_args`: +**Run 2 — Re-run the pipeline with fine-tuned models:** ```bash +# Second run: use tuned models for in-silico library generation and all downstream steps nextflow run bigbio/quantmsdiann \ -profile diann_v2_5_0,docker \ --input sdrf.tsv --database db.fasta \ - --extra_args "--tokens /path/to/empirical_library.dict.txt --rt-model /path/to/empirical_library.rt.d0.pt --im-model /path/to/empirical_library.im.d0.pt" \ - --outdir results + --extra_args "--tokens /abs/path/to/empirical_library.dict.txt --rt-model /abs/path/to/empirical_library.rt.d0.pt --im-model /abs/path/to/empirical_library.im.d0.pt" \ + --outdir results_run2 ``` +The `--tokens`, `--rt-model`, and `--im-model` flags are passed to all DIA-NN steps via `--extra_args`, so the in-silico library generation uses the fine-tuned models to produce better-predicted spectra for the non-standard modifications. + > [!IMPORTANT] -> Use **absolute paths** for model files. The `--tokens`, `--rt-model`, `--fr-model`, and `--im-model` flags are passed through to all DIA-NN steps via `--extra_args`. The `--parent` flag is blocked by the pipeline (it controls the container's DIA-NN installation path). +> Use **absolute paths** for model files. The `--parent` flag is blocked by the pipeline (it controls the container's DIA-NN installation path). + +### Future: integrated fine-tuning step + +We are exploring adding an optional `FINE_TUNE_MODELS` step directly in the pipeline, which would eliminate the need for two separate runs. The integrated workflow would be: + +``` +INSILICO_LIBRARY → PRELIMINARY_ANALYSIS → ASSEMBLE_EMPIRICAL_LIBRARY + → [FINE_TUNE_MODELS] → INSILICO_LIBRARY (with tuned models) + → INDIVIDUAL_ANALYSIS → FINAL_QUANTIFICATION +``` + +This would be gated by a `--enable_fine_tuning` parameter. [@vdemichev](https://github.com/vdemichev): would this approach work correctly — using the empirical library from assembly as `--tune-lib`, then regenerating the in-silico library with the tuned models before proceeding to individual analysis? Or would you recommend a different integration point? ## Verbose Module Output From 51a30002205aad4a4c8909805ea02ff55797225a Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Mon, 13 Apr 2026 20:38:49 +0100 Subject: [PATCH 57/69] feat: add CITATION.cff for GitHub cite button --- CITATION.cff | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100644 CITATION.cff diff --git a/CITATION.cff b/CITATION.cff new file mode 100644 index 0000000..07bc295 --- /dev/null +++ b/CITATION.cff @@ -0,0 +1,44 @@ +cff-version: 1.2.0 +title: "quantmsdiann" +message: "If you use quantmsdiann, please cite the quantms paper." +type: software +license: MIT +repository-code: https://github.com/bigbio/quantmsdiann +url: https://quantmsdiann.quantms.org +keywords: + - proteomics + - mass-spectrometry + - dia + - data-independent-acquisition + - dia-nn + - nextflow +preferred-citation: + type: article + title: "quantms: a cloud-based pipeline for quantitative proteomics enables the reanalysis of public proteomics data" + journal: "Nature Methods" + year: 2024 + volume: "21" + start: 1603 + end: 1607 + doi: "10.1038/s41592-024-02343-1" + authors: + - family-names: Dai + given-names: Chengxin + - family-names: Pfeuffer + given-names: Julianus + - family-names: Wang + given-names: Hong + - family-names: Zheng + given-names: Ping + - family-names: Kall + given-names: Lukas + - family-names: Sachsenberg + given-names: Timo + - family-names: Demichev + given-names: Vadim + - family-names: Bai + given-names: Mingze + - family-names: Kohlbacher + given-names: Oliver + - family-names: Perez-Riverol + given-names: Yasset From 1118b990f78dd464a4229eede5a4a79122b48c4e Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Mon, 13 Apr 2026 20:45:57 +0100 Subject: [PATCH 58/69] feat: add optional FINE_TUNE_MODELS step for DL model fine-tuning MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When --enable_fine_tuning is enabled, the pipeline: 1. Runs the standard flow through ASSEMBLE_EMPIRICAL_LIBRARY 2. Fine-tunes RT/IM (optionally fragment) models on the empirical library 3. Re-generates the in-silico library using tuned models (--tokens, --rt-model, --im-model) 4. Uses the tuned library for INDIVIDUAL_ANALYSIS and FINAL_QUANTIFICATION New parameters: --enable_fine_tuning (default: false) — enable the fine-tuning step --tune_fr (default: false) — also fine-tune the fragmentation model --tune_lr (default: DIA-NN's 0.0005) — fine-tuning learning rate New module: modules/local/diann/fine_tune_models/ - Takes empirical library + FASTA + diann_config - Runs diann --tune-lib --tune-rt --tune-im - Outputs: dict.txt, rt.d0.pt, im.d0.pt (optionally fr.d0.pt) INSILICO_LIBRARY_GENERATION updated with optional tuned model inputs (pass [] when not fine-tuning, actual files when fine-tuning). Version guard: requires DIA-NN >= 2.0. Cannot be combined with --skip_preliminary_analysis. Co-Authored-By: Claude Opus 4.6 (1M context) --- lib/BlockedFlags.groovy | 7 +++ modules/local/diann/fine_tune_models/main.nf | 56 +++++++++++++++++ modules/local/diann/fine_tune_models/meta.yml | 53 ++++++++++++++++ .../diann/insilico_library_generation/main.nf | 10 +++ nextflow.config | 5 ++ nextflow_schema.json | 18 ++++++ workflows/dia.nf | 63 ++++++++++++++++++- 7 files changed, 210 insertions(+), 2 deletions(-) create mode 100644 modules/local/diann/fine_tune_models/main.nf create mode 100644 modules/local/diann/fine_tune_models/meta.yml diff --git a/lib/BlockedFlags.groovy b/lib/BlockedFlags.groovy index c11d297..6950af4 100644 --- a/lib/BlockedFlags.groovy +++ b/lib/BlockedFlags.groovy @@ -59,6 +59,8 @@ class BlockedFlags { '--min-pr-mz', '--max-pr-mz', '--min-fr-mz', '--max-fr-mz', '--met-excision', '--light-models', '--infin-dia', '--pre-select', + // Pipeline-managed when fine-tuning: tuned model files passed as process inputs + '--tokens', '--rt-model', '--im-model', '--fr-model', ], PRELIMINARY_ANALYSIS: [ // Pipeline-managed: set from params and SDRF calibration metadata @@ -104,6 +106,11 @@ class BlockedFlags { '--channel-run-norm', '--channel-spec-norm', '--no-prot-inf', ], + FINE_TUNE_MODELS: [ + // Pipeline-managed: tuning flags set from enable_fine_tuning, tune_fr, tune_lr params + '--tune-lib', '--tune-rt', '--tune-im', '--tune-fr', '--tune-lr', + '--tune-restrict-layers', '--tune-level', + ], ] /** diff --git a/modules/local/diann/fine_tune_models/main.nf b/modules/local/diann/fine_tune_models/main.nf new file mode 100644 index 0000000..c74c808 --- /dev/null +++ b/modules/local/diann/fine_tune_models/main.nf @@ -0,0 +1,56 @@ +process FINE_TUNE_MODELS { + tag "fine_tune" + label 'process_medium' + label 'diann' + + container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? + 'https://containers.biocontainers.pro/s3/SingImgsRepo/diann/v1.8.1_cv1/diann_v1.8.1_cv1.img' : + 'docker.io/biocontainers/diann:v1.8.1_cv1' }" + + input: + path(tune_lib) + path(fasta) + path(diann_config) + + output: + path "*.dict.txt", emit: tokens + path "*.rt.d0.pt", emit: rt_model + path "*.im.d0.pt", emit: im_model, optional: true + path "*.fr.d0.pt", emit: fr_model, optional: true + path "fine_tune.log", emit: log + path "versions.yml", emit: versions + + when: + task.ext.when == null || task.ext.when + + script: + def args = task.ext.args ?: '' + // Strip flags managed by the pipeline from extra_args to prevent silent conflicts. + // Blocked flags are defined centrally in lib/BlockedFlags.groovy — edit there, not here. + args = BlockedFlags.strip('FINE_TUNE_MODELS', args, log) + + tune_fr = params.tune_fr ? '--tune-fr' : '' + tune_lr = params.tune_lr ? "--tune-lr ${params.tune_lr}" : '' + + // Extract mod flags from diann_config.cfg so DIA-NN recognises modifications in the library + """ + mod_flags=\$(grep -oP '(--var-mod\\s+\\S+|--fixed-mod\\s+\\S+|--monitor-mod\\s+\\S+|--lib-fixed-mod\\s+\\S+|--original-mods|--channels\\s+.+)' ${diann_config} | tr '\\n' ' ') + + diann --tune-lib ${tune_lib} \\ + --tune-rt \\ + --tune-im \\ + ${tune_fr} \\ + ${tune_lr} \\ + --fasta ${fasta} \\ + --threads ${task.cpus} \\ + --verbose $params.debug_level \\ + \${mod_flags} \\ + $args \\ + 2>&1 | tee fine_tune.log + + cat <<-END_VERSIONS > versions.yml + "${task.process}": + DIA-NN: \$(diann 2>&1 | grep "DIA-NN" | grep -oP "\\d+\\.\\d+(\\.\\w+)*(\\.[\\d]+)?") + END_VERSIONS + """ +} diff --git a/modules/local/diann/fine_tune_models/meta.yml b/modules/local/diann/fine_tune_models/meta.yml new file mode 100644 index 0000000..b097909 --- /dev/null +++ b/modules/local/diann/fine_tune_models/meta.yml @@ -0,0 +1,53 @@ +name: fine_tune_models +description: Fine-tune DIA-NN deep learning prediction models (RT, IM, fragmentation) using an empirical spectral library. +keywords: + - DIA-NN + - fine-tuning + - deep learning + - prediction models +tools: + - DIA-NN: + description: | + DIA-NN - a universal software for data-independent acquisition (DIA) proteomics data processing by Demichev. + homepage: https://github.com/vdemichev/DiaNN + documentation: https://github.com/vdemichev/DiaNN +input: + - tune_lib: + type: file + description: Empirical spectral library to use for fine-tuning (typically from ASSEMBLE_EMPIRICAL_LIBRARY) + pattern: "*.{parquet,tsv}" + - fasta: + type: file + description: Protein sequence database (needed to resolve modification declarations) + pattern: "*.{fasta,fa}" + - diann_config: + type: file + description: DIA-NN config file with modification declarations + pattern: "*.cfg" +output: + - tokens: + type: file + description: Expanded tokenizer dictionary mapping modifications to neural network token IDs + pattern: "*.dict.txt" + - rt_model: + type: file + description: Fine-tuned retention time prediction model (distillation level 0) + pattern: "*.rt.d0.pt" + - im_model: + type: file + description: Fine-tuned ion mobility prediction model (optional, distillation level 0) + pattern: "*.im.d0.pt" + - fr_model: + type: file + description: Fine-tuned fragment ion prediction model (optional, distillation level 0) + pattern: "*.fr.d0.pt" + - log: + type: file + description: DIA-NN fine-tuning log + pattern: "fine_tune.log" + - versions: + type: file + description: File containing software version + pattern: "versions.yml" +authors: + - "@ypriverol" diff --git a/modules/local/diann/insilico_library_generation/main.nf b/modules/local/diann/insilico_library_generation/main.nf index d7e695e..1b4b579 100644 --- a/modules/local/diann/insilico_library_generation/main.nf +++ b/modules/local/diann/insilico_library_generation/main.nf @@ -12,6 +12,9 @@ process INSILICO_LIBRARY_GENERATION { path(fasta) path(diann_config) val(is_dda) + path(tuned_tokens) // optional: fine-tuned tokenizer dict (pass [] when not used) + path(tuned_rt_model) // optional: fine-tuned RT model (pass [] when not used) + path(tuned_im_model) // optional: fine-tuned IM model (pass [] when not used) output: path "versions.yml", emit: versions @@ -37,6 +40,10 @@ process INSILICO_LIBRARY_GENERATION { params.scoring_mode == 'peptidoforms' ? '--peptidoforms' : '' diann_dda_flag = is_dda ? "--dda" : "" diann_light_models = params.light_models ? "--light-models" : "" + // Fine-tuned model flags — only set when tuned model files are provided + tuned_tokens_flag = tuned_tokens ? "--tokens ${tuned_tokens}" : '' + tuned_rt_flag = tuned_rt_model ? "--rt-model ${tuned_rt_model}" : '' + tuned_im_flag = tuned_im_model ? "--im-model ${tuned_im_model}" : '' infin_dia_flag = params.enable_infin_dia ? "--infin-dia" : "" pre_select_flag = (params.enable_infin_dia && params.pre_select) ? "--pre-select $params.pre_select" : "" @@ -61,6 +68,9 @@ process INSILICO_LIBRARY_GENERATION { --gen-spec-lib \\ ${scoring_mode} \\ ${diann_light_models} \\ + ${tuned_tokens_flag} \\ + ${tuned_rt_flag} \\ + ${tuned_im_flag} \\ ${infin_dia_flag} \\ ${pre_select_flag} \\ ${met_excision} \\ diff --git a/nextflow.config b/nextflow.config index c658e34..3aa55b4 100644 --- a/nextflow.config +++ b/nextflow.config @@ -60,6 +60,11 @@ params { export_quant = false // add '--export-quant' for fragment-level parquet export (DIA-NN >= 2.0) site_ms1_quant = false // add '--site-ms1-quant' for MS1 apex PTM quantification (DIA-NN >= 2.0) + // DIA-NN: Model fine-tuning (v2.0+) + enable_fine_tuning = false // Enable model fine-tuning step after empirical library assembly + tune_fr = false // Also fine-tune the fragmentation model (quality-sensitive) + tune_lr = null // Fine-tuning learning rate (default: 0.0005) + // DIA-NN: InfinDIA (experimental, v2.3.0+) enable_infin_dia = false // Enable InfinDIA for ultra-large search spaces pre_select = null // --pre-select N precursor limit for InfinDIA diff --git a/nextflow_schema.json b/nextflow_schema.json index 8314f39..02dd45b 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -500,6 +500,24 @@ "fa_icon": "fas fa-crosshairs", "default": false }, + "enable_fine_tuning": { + "type": "boolean", + "description": "Enable DL model fine-tuning after empirical library assembly. Trains RT/IM models on the empirical library, then regenerates the in-silico library with tuned models. Requires DIA-NN >= 2.0.", + "fa_icon": "fas fa-brain", + "default": false, + "help_text": "When enabled, the pipeline adds a FINE_TUNE_MODELS step after ASSEMBLE_EMPIRICAL_LIBRARY. The empirical library is used to fine-tune the deep learning prediction models for non-standard modifications. A new in-silico library is then generated with the tuned models, and the rest of the pipeline uses this improved library. Cannot be used with --skip_preliminary_analysis." + }, + "tune_fr": { + "type": "boolean", + "description": "Also fine-tune the fragmentation model (quality-sensitive — verify results vs base model).", + "fa_icon": "fas fa-flask", + "default": false + }, + "tune_lr": { + "type": "number", + "description": "Fine-tuning learning rate (default in DIA-NN: 0.0005). Maps to --tune-lr.", + "fa_icon": "fas fa-sliders-h" + }, "enable_infin_dia": { "type": "boolean", "description": "Enable InfinDIA for ultra-large search spaces (DIA-NN >= 2.3.0). Experimental.", diff --git a/workflows/dia.nf b/workflows/dia.nf index 6c53625..d44af67 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -10,7 +10,9 @@ include { DIANN_MSSTATS } from '../modules/local/diann/diann_msstats/main' include { PRELIMINARY_ANALYSIS } from '../modules/local/diann/preliminary_analysis/main' include { ASSEMBLE_EMPIRICAL_LIBRARY } from '../modules/local/diann/assemble_empirical_library/main' -include { INSILICO_LIBRARY_GENERATION } from '../modules/local/diann/insilico_library_generation/main' +include { INSILICO_LIBRARY_GENERATION } from '../modules/local/diann/insilico_library_generation/main' +include { INSILICO_LIBRARY_GENERATION as TUNED_LIBRARY_GENERATION } from '../modules/local/diann/insilico_library_generation/main' +include { FINE_TUNE_MODELS } from '../modules/local/diann/fine_tune_models/main' include { INDIVIDUAL_ANALYSIS } from '../modules/local/diann/individual_analysis/main' include { FINAL_QUANTIFICATION } from '../modules/local/diann/final_quantification/main' @@ -61,6 +63,16 @@ workflow DIA { error("${enabled.join(', ')} require DIA-NN >= 2.0. Current version: ${params.diann_version}. Use -profile diann_v2_1_0 or later") } + // Version guard for model fine-tuning + if (params.enable_fine_tuning && VersionUtils.versionLessThan(params.diann_version, '2.0')) { + error("Model fine-tuning requires DIA-NN >= 2.0. Current version: ${params.diann_version}. Use -profile diann_v2_1_0 or later") + } + + // Fine-tuning requires the standard preliminary + assembly path (not skip_preliminary_analysis) + if (params.enable_fine_tuning && params.skip_preliminary_analysis) { + error("--enable_fine_tuning cannot be used with --skip_preliminary_analysis. Fine-tuning requires an empirical library from the assembly step.") + } + // Warn about contradictory normalization flags if (!params.normalize && (params.channel_run_norm || params.channel_spec_norm)) { log.warn "Both --normalize false (adds --no-norm) and channel normalization flags are set. " + @@ -100,7 +112,7 @@ workflow DIA { if (params.speclib != null && params.speclib.toString() != "") { speclib = channel.from(file(params.speclib, checkIfExists: true)) } else { - INSILICO_LIBRARY_GENERATION(ch_searchdb, ch_diann_cfg_val, ch_is_dda) + INSILICO_LIBRARY_GENERATION(ch_searchdb, ch_diann_cfg_val, ch_is_dda, [], [], []) speclib = INSILICO_LIBRARY_GENERATION.out.predict_speclib } @@ -183,6 +195,53 @@ workflow DIA { return [ new_meta, ms_file, fasta, library ] } empirical_lib = ASSEMBLE_EMPIRICAL_LIBRARY.out.empirical_library + + // + // MODULE: FINE_TUNE_MODELS (optional) — fine-tune DL prediction models on empirical library + // + // When enabled, the empirical library is used to train better RT/IM/fragment models. + // These tuned models are then used to regenerate the in-silico library, which replaces + // the empirical library for individual analysis — producing better-predicted spectra + // for non-standard modifications. + // + if (params.enable_fine_tuning) { + FINE_TUNE_MODELS( + ASSEMBLE_EMPIRICAL_LIBRARY.out.empirical_library, + ch_searchdb, + ch_diann_cfg_val + ) + ch_software_versions = ch_software_versions + .mix(FINE_TUNE_MODELS.out.versions) + + // Re-generate in-silico library using fine-tuned models + TUNED_LIBRARY_GENERATION( + ch_searchdb, + ch_diann_cfg_val, + ch_is_dda, + FINE_TUNE_MODELS.out.tokens, + FINE_TUNE_MODELS.out.rt_model, + FINE_TUNE_MODELS.out.im_model + ) + ch_software_versions = ch_software_versions + .mix(TUNED_LIBRARY_GENERATION.out.versions) + + // Replace the library used for individual analysis with the tuned one + tuned_speclib = TUNED_LIBRARY_GENERATION.out.predict_speclib + indiv_fin_analysis_in = ch_file_preparation_results + .combine(ch_searchdb) + .combine(tuned_speclib) + .combine(ch_parsed_vals) + .map { meta_map, ms_file, fasta, library, param_string -> + def values = param_string.trim().split(',') + def new_meta = meta_map + [ + mass_acc_ms2 : values[0], + mass_acc_ms1 : values[1], + scan_window : values[2] + ] + return [ new_meta, ms_file, fasta, library ] + } + empirical_lib = tuned_speclib + } } // From b80b2bd045a9aff4e4b520a71bf50af76fd37ed3 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Mon, 13 Apr 2026 20:53:59 +0100 Subject: [PATCH 59/69] feat: add social preview image for og:image --- mkdocs/images/social-card.png | Bin 0 -> 66263 bytes 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 mkdocs/images/social-card.png diff --git a/mkdocs/images/social-card.png b/mkdocs/images/social-card.png new file mode 100644 index 0000000000000000000000000000000000000000..7ce482a5d919df363931e77c90fc10e987efafb1 GIT binary patch literal 66263 zcmb4r2RxPS|Nl8SoNU?qMB`L;R>@vvlO)O^D;gxD2*;shE3z}QN+>cjD`aFRn~+V} z+yA-`da7r9fB)CxdG)+{+~>aU>-voM_*}nYQs0XusnMy+E}K%T_paD4UA(rqdVad3 z?9D{mkGy-L1ZPFa7tH!Ux)l#2HJ7GU5D&K*J`ix3Fx(_ib97{2U-Zj4Q+C&TtU zn(&n3ARg?aw0&{0p9xVv%q3IH#-?#)imk&TG532c>yXPL^u{mNGmxUZn<>ToDg!Gm zvny3p=VZcqNaQu#tHZWrtmzt%wVhT4M>Jh$c=nj za$~aWc2_;3XybDGWyO7L-tcN|>b?6KteL+?ao4<6n=vJ8nsgI_uI4Tds8(p{wo)o* zz&|l%VOBaeuL3(hb0iQvfkoqfaVw^g`QJ%Z6B`)e&DNq*#$wN?M`q z)kF=*n~g}vcix$r*=5-C7vEBA@|QV*i=ATj`t_CNWFvWMkV5>H6|U5= zTNcHQzWB9`e^LI0CU?O!M~oF=il~nl<~AmaXNF7QEQqcz@l?PT@9WSRvyqDj;=p5{ zm@X}V<@9#wVqebWaDB;Lh@L4$dkownAUFcRO!9BA(g9!pkgjM(TgOz03uV zSKL>}t1TK^TH=yll)vm;c||Kif-qHg^$rFP;21@d&3#qrj5hX`z<@X@?mMg|cl|3t zy(3tpyrM>5p1@n+*Yk*+*A_;RUCSW7!HZ*r(#>dA5+?6A(@Q!HF>N zAD!$tup+@bD=*`EU7LI|9R+#%48Veo+ z4i+WnzXPg>lqUqiEch4>sH6Y|(1nrBWxvHeaSN@gR!&v5| zL;}i@Xm>xsbuYr*sroH;?zfLx)uS-`K-w8UIty{t&1=EQkU zhPXG}H(nQn#UHD=x07Pj2vZT*0qBhfl$-pq4H^PiUXX#05KJV5R=p(hhuy19KsdMk za|~4_8fP3sJp=JWw1MBd-S^bOAftN^K(Bt0hRqIazdb#Z1TM_&x+B;^M8u5LYivr) z*8W?=R6a6Hejx$7!Gr%lAb(IV*Ntl&oA7J5SCF9023cAP?fN^q;6uw5!L9v2UXn_@IdZm_i&Zt()CJiX49j4Bk z|4I#(u~7p&49t#m{WZ-WAv8gjj4m(uxO4O1Un&IVl&&?MKmEbdN;UZtxZJBsG8U=K zvIk$7?A1d#Su6AZj3ONckkDU`J6K&+3TAjoQZpsY{ZK(5KIW-0Oaiwica&7%=A7oF z6qC?uYYRU<{LSq+4Pm^SU#3ztxA3ua71RWe@e{Zdf~!j9fwu2lwQ+fkNWEfyUzr)Q zP%Anw3mcLneK#kEH3j0)25aX^u6u!k|FbAr5+Y1raUPRDK_Dv8fc!!rG$RF8sXK>= z5b8VS;!J1ox0j$K^uqbR?t;oaJ{Lftv&sMHs!i0|TMf3B&i_1>7nH=U{#agY4e4d^bh!>=nuH=Y_vF2d}(mIfDAef z{i4@WG22RFOq`SpG?U)Le6sU3+bV4^O>ARm_n5n33uA{=^@)|MdNa=qnb`8NX`&8J?) z2|Yg~2o`^evH=+g+J4Q1%X`_^xgPUEnfnt_&Z&NUL9;5C=Ft6HOSMJSn7zIKMLQ-^BcP9=Y;SfSq{b9K$%c*(ik7o@Lf@rM%x zg;_CAL9i+@y<$Je6#NnAkD3=M4C2K?#iR9zyTKa0`C?1Igq9Wn0Pq+1w?Y!{R7bEywl&Zb zwgj=us!D^U##c|7+eh5HL7Dy_NP!q~e(V8S9lg1;5_IVlC`h2fip!CLaO(g$3GiS1b2EB)P-1-Cwd8xj~O06+`h1aR;vBo2OkiM)uPUN#*z zL1vO8Aq-_HW9lXv`Sv!iaiFO$Sd`a039-9n@qa`_6Vw*Lvp38%wagZPrs!X!ESw$@ zfAiAgF_0Ch%y1~Ya=65Y0VDxWy7RPSrMg7P1Zn(U;IiRafcFYOMgu4|uBDWKvYp%q zR%(9(QqP7&UAf{tz7aw|1>~uWYr?O$1DH{L3$*oMnme_mq*Y%f%|oGHO!<-oeB2$S zY-dSoMbt8))*kb)ls%ZMSU#i-LL#6%g5bSK14UN;o$+u#Q+Cp-D}*M+JKeA;MA8#l zs0i?c09t`Xa`Xo93S4kKYC~OFR-53LfYA5Fy_)a{4&3Jf9{Q8UjHxm{Nr(t*rici~ zrtnK`2sRl4r=Sx|A))e@Fz`&L?F7Org%bb>v@h6O5J%eJeewJHPZ1GZ3ig)gMFz+f znTZBr?i#a45cZCu(V>$Oz}AC=`7{IJU_73Mo*48z?i$zY@QZew1g`0=_k zpxO&BN7VHA2S=dI0d~rz$6r3E*DDpN%!f>j8 zi$5v^G4J@11_FWtpImRjEs6oEfZh@U!AAfTM=ig?BQ3MJ)MU&~+ZvY2gGFm;aal}Y z4YUL(YY613q2;DCf3q8re*o#-TEh+1nxF5650J6M3Lh+d9z#FW2@jUdQRF zX25spk8xUBzjw>s5acle=*0|=W8Kx+!!3xW>Y$=;IOg^hpAEZk_U~&7nnr6cLwn&utVKzyRNQLQ6fW(c-}0Terhq5Jt1*}nMTX}LNFg=dbN;co z?1a*BxeB5|P*%KjxnB!&=Mp12eUQWjTwwp4pmK!oKI#I-d(inWmI8ou%dMt`C-ijw z_u&A1wglEx`Wvr*I8beZ{Y*GPc9an;9=ognwjow_IhpGjKLE2wUvpE!@BkKqMyE80 zwW*sFYvBn)t_J{mjiq5zmi^}*U>GiM_)9VAM|8aAdu&^zj%ZQr| zRY6Z5YH2CUD7LGc!xevNrVq*hzCNwa*yoxw9NjEYVxy#)g*d1L-XWP4MDp0 z{;Ztasm{e_5I;repXS!50C*IL8{!a8jvx^FwFN?-_Ijs9^6Z8dcbgmoSjq*!u)a(& z6vc_-jA79LR2u?D!c8VXNu0)SCIWrbZpr_(Jb7ycC((5?eRD!KYXfR=4?F@90KU6O z7fMPrJ)pKbU5(Cc7k}&_qjrD;5Z(L% zr50xjOs?WA^L<>rqktGh?tBxOWfdcpWUq>nYhh}vf8k|+BS!H@Q_B?Cf zJ1b@q5X%sx`deuLME;j*gh7P)$)jt+`FtZ`az~*3N=9RiVUbU@v=08dcRxuWo=a;} zdy!!A6sk3R6^-ojZ4-S0hsgj<6e??baBx=}!nd?4fLM}HNGhyaF+U%^W&tUwB>{;zX-y1j8X;IZ zJc|O`8-bTj0F6Mu{1v6;E=UC8ex>q4qzG9?3YH)owPHDmoB>rZkjkiiAUaYno(jbMPV#m` z|AI%d0y-^TuaDaxH6GC|3qX`vzJn4V9&{BlyZRawqxfAzk&8nb)#rjZ2r%V9WMJeg zKvMVUh;At6(I8CQ-5?z*bFX4KL8{wy{GTt)Y>zpH(}5%j2nAjT(iA^CgtLW44QjEz z)*#`cpoA+(1iR1+seX9v3932dm4H9(k_HH{?lCrRAyBR4WCIDJXjT#zL19qumTm1I z)F!~e6)#c{1{VK>^$=P6opYg+w5aWyjTyA5VW+pKTK#lhb01R*z8eWiB670V;ml5) zK&YA?mJ*c!VvxVUmANB-r$yU&{F`>SK->xeJtr9fo<5We0^LxhvBSixE#Eg7k^)j_ zx9r{%0g(7>`vR9fU$^}Z5fnQ|KTB}Y$=%yYoDl#J6FdT5EC2+wcsvB40Eo`n6t6=! zU2Sm`=r~J4T2%?vn%Yj@Sei8cXWYdvf4dtXHF(4oTLFc=AOf^J(6+fhd2^G1s6$)k z-vNv&I1x##Qv%7n+0?{%-4G~mc)indNBR(Un#op?r zSN|tP(o`+G6tk3ss)@OFl<-)s5}p4CNcW>NU-Sn&1&ASku(Dn3Y?>nhYQ;>cJzLTY z%^|vBRSu9em3Y_=HVVAC9UOX9NNC)RPZ0KiYtK2|+o8>0CQ9vhVg;38{Dj~WUTyfI#_~aAXPq9Kp(m)2IsU4$}PyI1cmzKuyYBI9@xETsA#Z z_u&9|*=|r^L;=iBp9RuAw&CUve~?q`oi}<4|$qP{=0CJ@zrSVo?TaEkFSP zvO%{j#0!4g!0wVP;V+Ar3V1fWH)dY&ekO;gKYetQbHGBkU$z2L40P+a@ z4XNJ$VkhkZcu1Z9Ns_|cc3h(nkUiJsWPKvi3HOuEnX`bFMg)pfF1_&qa=ateMGS!C z4w98r6~Ie5K=Aj%=|PAt{x53wpqE<(U30-f5-cSPWUA!kcmMxVGj0&bER!bEx(}e9 zM%Ll^J0%o{;Hd1KHOsDF)r&u)ii>XeczUce9vVT_PMq`#Y0YC`6O3#A3&sCwIRJn} zLuzM61{MLT)c=#4(e8r1U7zePjC5pPTZ=2j{~{bvM1GX&B>X?<+0S;DoGh(+!NV}; zhv^Y6x;7du?FKp7GxDD5{nIGI+GzS*V_;m!`GQ(n$zXi^R~GS4M(AGzEluQ`f{ZsH zNT4jfMG^-C`*Ei#{X|Rq0oDf+aTPCw(UDZZ2n-Jlk%gcV%Y9taR~Loo+r!y$LdUk< z0wKTzq@o=oRdhF{{x1v-?5q$FHg({jQ264D;@@sx0n`IzB_XPH#*a*(8(-#k8X0-9 zawq@*i*7>ucl_90g=1H4fXrhcR0H>IiW2&anjHfauT!HzjzhRZDDDh|(S5lfof`d+ zT^5ogK^*_dR#eL%?8 zvkeRmKsE<3T!CVu_b-#8lE4pBzY=>nSo z@0|ml2fPj%(iD2I0n>67`is!c$>?Q~BqE1PWgvFF8@K#}yza zd-aE8q8|H6Eki0YBzqjq|Kawf)S?mq7vl9ojdigF)a}e<@-(Ojbh!^71Twq<%KqLC zy!y+9K?2$pbQB*$5PbQAs+9u50}q)1It5ro;AQ1`rhttk`e_|O1Q8?xUiiYSu|whk z1CTe!%;`vRd@Kj)bi);7db{^M8wM3PK7jy;Q#kkta2dYPK-&7Q@PW6$_pCB)YG#ik zauvCtk=1blD2=8!S@$II`sJB zY3?Rx;O*}2<8M|OK()OKALGK@6fc%6gGKwiHJ!vDNw%@qT9`k6x%&XuU66Pl0Py?ZPM~d?`Mzz;LZ@Vk^{w6h}>6>~! z_Af#9JSf!G45V@_VAY06)Lg&Uz!RoV5L93jfi&^Eakjft z;#NLRS_LnZxeA!tigP^PXH$ltxFt1k3&ER0!xrEze)Z2lhvJu$)sW!@IR{7Hh=Z8z z@4#(ux*}?0*9e3xT<;wjXg}$5Rj&A%X(AnU`3il2<6IMTcU-qWq7%|r@oL4#etSB) z9cKebhY)gl{QyTHm_Q8OJHv=df##|tHM-R`;d4Qwx)~@qe{J0FHrj5Vy8+5l?;~2Z zQVT%2COzyNL1;3Wyoav)%L;;|n%}KeYSOvnY2(ZgPc)kG+@bg-7llbW8MFC+WoZK9Jq|0zi;B zQ}|Cl+8+ceoD!-jCiu&fT_ik0&!U<9R@p0sn<3#X*F}>0kS(|kG|U4H;pKV$>u4Hq z!**i^iHKBzRr+U<_~4^$*Neap#Z}>^uoo>jPM(+f zvc&AMCU@#}chTPzhP`n=u!_pEJ9^}8`B^SdrU1OCc;=#zuv7zz^aNS@!!J6tCK_x7Ssx>lXli+ z@@HHRWJ1#)Kh^R*!*-X>;N8=*A)t2;fdCPsL%gYqL7qXUWk6#D`%nj&27I-s4Y5!< zfol14;t)a-4Kq+G;Qn^b(rC2zGO#7&r$~gXTxFGe#ZcU<((eBwjf&@zVmvZ`Ko$z<@i7)w0$3bbTy98AvREO)xnIT|KTX5C(+k{~~g+vUH)4 z(1Yi4@3YAElO9XE$m+bO2J8nQ#hK3k3AtArqYk9Q{^joB;LyMW)Cc}bT=p{He1n3| zzauD<9Hfh(1Beb@`}s77A)(?C7`49{@~<(NHi(B?>Q|u21DR{aGS>hrxq;9J zodarqdo_)g^a|OY`v=xOZM>M>42aEv88Q{(#&cAyL;Isk*a0!zzb3bqi3nbYRPS;N zF<0OO#iMa>`ZpbWv&=_nNRQQ|L6u+?4ywV|cw-)@lz~|F zQM|S?CDQ{Ob%!Yxm>wfY03(}|9_>Kw1;h#XU!sNlzF-3854YYy2C@Ki-Q}rv%sc3H zHkBz8;&Tsb3+NG088($Bhbe_WlxK(tTM2d=xuBolgd6_d$ z;o&AIMcf3>Z*+7I2&Y67fRr{PibWz?fgP5Wkzg#ZMvr%D?1qrU3ugJ02uUnrv#E6) ze#eB4o!$Raps1PEw+He6VA&LaIRsK)F0&rjzdlej2^HZoERzR#Dl{VvK^WC_Ik8K@ z5;mW7Yg7=W%a=yJjMx0Li*v&p{-T!=jP02#0+Tl7#!kU2D!btozEl;DP}ZnjP6yaR zdNuhWbA4;Ufir;vAFq>~Cw=IpibKOwU`vi#%9{=s7n+TUsMRNN)+|KWOO$;26E6D@ zsDsYM12)?AQZxrn`dKVb=0prgWntfkH=0{ztv6kV4oW~1_1C;xK%C1WfyC_6QU2bwG815OZQWh=;lN`hPIm{FBk6IcxS3@*F(M{(FAB zJZTrve?Zt2va@moZ2PcIALpXHi5q4XOXqWHzw6t9XxF|?1+&ATCdmHO7xpZwct1eV z+7HoWi{wX~CAKF(&|eWP78Fkh68|S)GQScKD9+zfm!MIu5xAAx8)@M72zAYv&A zcShy$L{`1Ksn{E5CwQKK1;X+J&u|Rak&uVwvYs_ERtLsJ8)m5Y=69f1;(Y)IwtMl1 z3m%Z+I%iHnD#x1U38?@Trx0z-!QuA>g3~_Nk3*UZG#LP?O^_P}1t#8q%+3H}T3ZH% zWe1x`5-bP!j-8a6V#Ol3(ql{ysglP-bI>S#JV?~YR?YKUAXq> zJgc+(j(1hW3g~7Lu(?~=sp$wrwCLk7e{7 z@J=qe_Y4EVdlo399&JC8N27T`rH7psrq%%iw$sI;_hvp=WU$d%L#qw>?mI(d z&O?xw1vp>P&R$Uz}ZsriBrl0mC-~iyLDQ z&)(!?!ea*jkRFR&BS3ch;ZQx)5hWbIjKdjYwVL*PmK_rUGD!bMw!ws z7*M!Ulu=A)M0QMM{6RuUpEIlGD<4;)1_%a4=bwy+uYiIv0~}?7M&6S2!tGPZryrCw z&+YA&9fC-<%oQQ)3ie7USG;ienM!K%0Wh?O_<%O!TE!FC$ zv&Y&thk`hdZq{v@*5_kf&TqvQpPJk%y!3!-bB+0y&#!fDhm?c-Y;hST(xFM`KNzC8 zHa*83<#0G&Vb@kQhi>&3$_8g?&}iQ#q2=3BLmw__gkrS>&dUJd4M1+@MLdZB`}mxh zk%pk?EO~Tq3K;3(%9q%aCVmc{;_}{M+h_c^mI!M>u;ZeAj{z}0Hpa%V7B;EIV=Gs-{pKXr8p}Gd)V!AFpv<{-zKVq zp3DbMOq~QJOJ4?1dovgUB*g=K)!R)9h)&5@%<#>Y1}zIyuFY!(lx8PYYCgdqEh`&W zEox#}8mIyy#Zh}Xv-I`#L9J_PBNlt3KGBM0+6EQ!GNZAYFnmQntdOye!~ zpB|bK)_y@J@=?03Z)hfw6_3$6i?th|aaxzXaMsqOq$vSbKxN?t?xcqf3g@BAZSVlc zI|dvvq(q_!8p(SrF8aPPmX{kk=L%J-&2NlaVqbkyRT?ERnJ%yPt0Z#)E((BjZo(*$ zFb%??T6f=7AUWJPZTA=$&e}x3oNU~iN8PgyFOvwrbDesK#x=h}MwcA@tK{zRFUa(p zma^kI4VnnL{o&{3`*NRa1Y&&4-^v)KI~&%pP~R81*tAGPo(5*2wE>Uj!?q|bbm0MAO|+oBf(KG!KPiK*Ip3`Md6gEI%PTBfZL zA(&ReZ;b>^bEJVY?FBIQlc{3pwX~h{1Ot2%?aPi-bT{SS(+m=Q5vfcW;WRuJNjw=U zYMglS#&fMn6))ZC&LbTMGNKZQxLZn%cqhmX`7LmcE;C1%LMZPYYBX zarK_dHRuHTc5bID=RMBaLxnuxoH?DpIhxcu;3N$Q-HTqh7B0GsfQz*DubKF$4FZ6Y zHxnP937u@hyXc?lzrr&vPGDSb*$z%z!eoWND9U%W=Q-tg?3AOB^ViXl)co_$E%U5; z`>M__=2y&0v$0EY2<6nXeh=e)&M9qAY}E6i*q;Cv;BKPK!FN*ndr4G>U-Fn`_x%JW z1A*?cLEoq2YH@qB0LFut+>OJ;T*BgPB2pX4l|AK~2efk6+iTxxKGCFT|IGbkby;Ki z^UM17&7nTR!B?~ol!O668e^)YcQ-B++Y_|Gac7BsJ-&%dz%OPgrI#mwTsT zMGxFvFK$#_fj4~cO~~M@j>_s-KI!>;g%++Eyi%j@)*K37@YiKsd@}h|iXIM2M1G11 zaGc6G{lngBGSDtL(S9w<@M-tkqh)q?Z&sY{R~26dzmPYJw&~XHT)MKlcw(Pba3{&y zWHM9gn`xRsUlH;zt{#7MGI#8PTfDY-1^akyp(i`Tft9(8AUD)`PY zCuQlqpYirq?T;am=F?Q4GxqTn^XQjJMVxHSl9sJX8mfLb8hc!NQ8#@i;8yvwD;D+N zKWDs@o?f0cHN%t~E|{7#-dwGymr`2l2=s20GTa(JmpoKG%rViW9|T7u@8uwYaiklX zDuF^1&NRIM*DlV$W}e1e)KyJ zTwNKEPfpE?eOLK{uhw-TP_8$^JC?_NX)@y?<-jLQec4FS+Vrqb#h941R8&mJ=IG0d zVe3Od_M%(sE^bcZCv{6sQ;BANDjDzEs1hkWc<`z&Q)MIO`Nr7CmPRLCIjfDggS7*$ z`5o?h{af1OTkgl@Hh9UX+y;v_cx`GLV-B3J&2)X@{?>g1>D0fP!n^8J_EIX{pTnhS zW1Qdr{B~W~i@D_uJJ&Quv4P^ERdFri1t1Us$U%zcV)YHNW&6Tl!X% z*j!JU_3f<(*XEChb=-DHS}HSG9ekGzMzwju;jq5!%`9@y&>fI3TXJrY{*fw0^*N5S$KJGhZJw~L^VJQ*)j?%n!~C!#oFRwjXC^Zz z3nnEO-G`jY3^yh&R3Dx!{81GsR+8S-ef)gGOz)FM*P9Dt%|2X*xAeb7XBB4-NtWrv zZMyXHx-EJ%i9OT?$U;awj$)bFBfGNPj(cs%^uR)7Eanl$G4 z)l1|YaW4P(_4jH28(BIIWmk)>kjSnT(3<*Nw|2WPO1$sptcO{ z?>UEDEF!ZVx881PcXLW`B$CGE8VpEDkbRqe%xzzR{ceCYc4)KbMIG?-T0fS${l1&A ze#w2C(CVtd=M1CST@%c zeF}c{0qd!oCFj4Uj112^x29~Y_k};GKD_wJ$&v3;@El=J?$Pbi<>Th=)zeSOp1*b} zVtB0-x$UxlKit9Q@T2DxX_lnz7ZPF>v$d^&0w~4DadOczPEArO!Pu9k>zmd-Cat^R za2OTP@Vn+Wy0aQx7jCSI-K;ISb#A7kKelW#pwRDvGvbqmJT3V%T>{dtl(6eS;bNAN zJ>h0x8{fD-yUbvbHh5`UZMG!eMCOc^yI&6`pUbIR{dMu(v%SuyZi_Y=M}BZAq>m#*e53jFB++p zzDzeC34=o}7iUXLZpXfFsJcD6QnotC_oR^oX0hZf*6$_WtSeP3^5S;>+d~cE8Yi9W zqREc4a-<3$nz`%RtXNU9?3f;h;w;El-MSVWQuIA>fRMB8(Wd9+SY>pYmZpwc2{`U# zeeAK8fV!X+;nz@7z$2zh&hnJ#+jJ-syrtMVDn&|jfIC{u-UTKJMg}Dpto}1ea|dq$;(Ew>#k3Ecg* zBL{FpM=C*IOsamjC1o@IwP)k58(+dw>`nf37e{M>Q?>`yREqRP}sl-XG@+O7k?CgE@%zdwu3q@YoT`*#x zBlVxV$Jyt0ZIDyyUGaGl^IzB-Ln>8V-Ik4ev6pjpqTUw3=DPA zUgYKRCieR<_Tz|(a-bLf=RvBbcWybjq;#)zeZ|42yPfA6) zhULqKy55bsEdB18&zBZfg+o^ogSsseQ4t;VZLHL7K*;n&U#+5k7A*Eq)c5YFY>Kwl zC7C6YazcIKZbPS7_pQehhVq@Hc73%7@Ux|Vw95hA^-8+ORXj99u59nv>`eyk3t9$` zXE5$o*^bOEv-=gla|VT${?NK>cKz#9+U>I99Q)qnLd;vK#sg>d6yATawlYLq&8r<#3>ir*ug?8)0@ksYnwB9;m5j-qHtkA4B*_# zP5n`_>~8usGR~pWeM|ao;=&X6fBeyr%N^;aX`D)j)sOqS~VM{#{~f4Wfd@hABVhe%G{>0C)aC#%ufO{hEK!f}g>2n&G8O6tkZV zPxaj)9GVU{V&jqI&!25qsJhb{z#hZ*^?30yD!1F$M`u)eQxdhlXLiMGCwG(fMii^P z5%vn_Yj|1~=z4qjMV6k_X5{lDjs&ps@3z9H+1O9Al`iP4_HVmxY|MKW_qBd}UrE7= zivnfkt=sCzoKxw!p3;(~vl=w_K6););)-WyRGdl26d4ivC^O6Lps#-2lX2W5A3A5p zqbfe~ZQWw`2dAWRO*A!kMyB<^ph8ihPTSk}!eSI9ioq9TzScz+D%u)9aOzdszR=Io z<8V%ET~YScsrkMK$y?JMBxR0QmiY@}Bs>E3-gRLY@|LKz`r*oIPG&Q?R1T&Bwmp}E*y3Ian zsg@EnkoTmp3F7*)rZdtgCE{oip?#5$IiJ|Z5P;-5;} z_<`|{5&y8Y!QZLB&3e;WlXH+$NStnS#5F0d#C>DRR(E-+xp|C#+r<8O$jNICV`o-= zl(DQzW0)nnzbCmL(Ri@E@!=Io=|{KF=C1RFQ5vS_;q%|t$n^VP-JAN@0`QfQvZsR$ zCx_3ByYU95tn$pnUEA9_`6qFuMzdjFHeq_lzVl<))^p0(+{`aXQV^7{+*wG9quc&& zYKd{axHSHzD3j3WHvM|z-7)VELEGcD>=bg-xwMMet}d%t(t>tNJnK{X9NS+WRGM5& zFPGkI9gEvuJp&prkw4d$(l;B@+rv{IX{Tt16aAjd^t{v^$cba#^pj?_dR8I6p7(M@ zdX@8ICx5eZbWtCxf^79YKk|rg#LX6ZX@eZvuJkPwiAN{TvQi(o&#J&GQpgh-fEy4c z#Z9D&CDj#Qf3v+MhTy!XYTNLjM{VY^yqx;%65r*J;>mzj7n_fmVnMMNOWi`B6H%sK z8cb)xXwebWgEz#6UGz0)nK%NZwt|Kvmc8rqH7%G=v2F&fSLKoMJ4AcqUPVX?-5NNv za^HrbF2tQSAIn|+PIHSa`s~`*K6VZS-1sfiX1k>lr+Z30zp*WPEtNmugp=p|=*n&4 z6b^$hql|pA?J!WSr8ib;$oTahzh+eZ?&HqJ&OUNJ@0!hssk`XgYgDtJie|U63x?>c z21nPktr4sz!Ch$-zh_Zp-!p69ZVK|t$_`vrB-58{M8XEx^XVg&qx6p1_FF1(xIK#~ zNDhK=-Pv3mqm5(hTuQ|5Th!R9=ZqA;lRcNrAM}Kk)P!Tda?pdgHQ{y9u0A8oNH!a1 zkksnOM@Ql^tO+h`CWpkm8?P9fkK;6T%bWkU8dH4gL+7@|@Y!a~t1%q@QrjmTmkQ@M zI#)0SCg|x7L)?85nBejekMNXEeB7t6q`S z%kcKcN)=(;0c2hJF{-t=jq9tr7hp`Q@93lZ@?%TD7U89^zoZR0Cb= zQ{785>ngYU2eqm;uk4!vXE(`zw7I@Yd7aic(1W~9!sPNg%iZaU6W{2TYVmQ!&vDYX zA260lez4khisN)2Ojd|`{(0%bG5$layW476%xLt+?FY(@3YSIRrj@FHg-Dmd+ z-~TB#06^y?^X5iN1)6)y-cz17Rl-Yg(gJ=z7}-v;1O~JResf#Fm^vIDtzWv9GT_td zz9u|7r`pM$!0%>0)j3Klkd=qMvTtkaT+)&Kr!*?_Q=8ZrUTt*ePw>lpr}DXAbd>R; zdPn4mCwlKR-JR{b^SY{xI;s+lYK(STY;$({`vCASt_E(&D=qBm^&t4Ya&-nPkuPP%(gCIDm|dQ zFfKV4^7S+0dtvd5jq{C%%``XBm%+&i8q!4H!S`*)eYj45Z-3hF_KA3&brC!<)BG{h{kccGoP@jbQkN>8%~@yR zPZ>}r#RpDZqHEK`s%WlMHB7mTmacGqEQ^kbpOrK!U8GnwN_R;(G;vLcw2CESM9k>< zN!x=wuV-N+R$dNCMLAkVjMB^(FdZ`6lE4OL&C)z~Z z@3j4K?J8?ReT=2T24$}p&HI1otbSl0d1hI1_4KW^b!N-z1ohabc(C|PT(ZIaVcI1d zWgD3o7QWk)6lv%iE(52MDZc2?(WUn)Yq%;)c{}BGxmg0*j0W~4Fv&C1Ux>6P>1os=BhDIW~8KKUDdk zXjA+7IPnjMxwiP(YS16n%)w%&6k-=6Wvc=D^{Xi)14n(^w!myB+m>4~+%8MLA#78#&y|DV6?26-Z&lA?v&*mvBWGsSF0oIrmK-Lgf60+l|+{ z8J(Ez`+|KAFvr=K0w;3> zC6B6kgQ!)Z{mq@jF6KMVmL2@f6{OBfpq z-=OdJ8n7X8Fz;}i$$dIRHqJ-STY4wm=&qA6N`L$Mg^78tN3!@QbDp_em`x>dg`mH`Yw>b6q9P=rO&DysN zz|Zv4%06Zc6uGdeLzke@iyI-P!Jg;IHYrp~M&_CFzgWpu$dRZ`E}W8xUsJJ`!uFL% zH&YB9Vq*4Fsutl~+C1@=mvJPsOu%VHp}@hWz!ak&>iUonrVmZkbUgs)=IZksZxm$0 zVI+Ym+nGmUZ~4}?p75e0=1rRKZNEiqO}V)*@z}&SMoZSsPA*1>bz0XOWws?Pt7mY07`XF-#{jffhgm{93IqIw=^9%92`+bB%l5OHO&$Gez%@HnhS!bG6ki#%c+^X-vy&2(-w`C0 zr$}`_>Ek5azCGNUOkcObns0xmG%LTO_})3&)f5=aNP?!cAZc#?#>g~(7^_>{Q*F-q zPco5NNA1G4PFS{S9yGJYmP&1RU7Me5q+;Ozlw8ocKh^66X|m4kZQIhXQ*#|6P0+-d zoUF**B0`*;j9f)m#Eom@8ov4CO+mm&^TIjM;NO}|`aa2J#2DY5r>~1?VAvR{)*@qr z@C!WeYV?JLi1)4YWj^lTKm3-Stc0Fy^M_9Er13(_i@_OriUuyZ3blCf-h5TT&0e$i zAI*azWX9f1EL^y|J~zK%2!!f8u9-fUDSg&dX{Tk$vIq%a{;%B*HmWU`t(31!S|;W9 z>Xf^^yRn3I-+Z&R*zrwSDKSp+f*YV;pDxp1cTLLZKN=a@v?=Jr$*p$eoA)bws~KLK zqnGZ;HsLr>u-P1*R_bJ(S9hn`cWnF@4$$$Ybrr7=I5=}d;l+>$X7mh=v8y#& z_6(VGa1ilD60#$zOYCrawRvY3;|!#wa?-7j>!*{Yx8?c_%j&_8v;<4Z#>9KivpF-R z_1&$|*CB;1-amCw^QhU)v5{U*tCjSd<4B=F+7bBh$^NvjrIMrXZj?3M>Q-M}0^iVJ z_dySw!adqSF7ox1amxYbiu}^cJtf6z>_=&Fck$B1#+wVH80X}T0kLYQY1y8t zlTo%QXNsi5>=TbZToqiC+W5-PaPzKe!Uy!&p|*3GsBPPqapwxTa_G3g+$J;z0S07X z3C1rcJhDZ)Wx+Vv;?kXHz17K2^Tqp%9FHlV{p3BPF*d$H;2Lz_uiO7JB1`Aw@)Yy5 z*hTfpxlfphb&eQjO7Hz{g?=G2-L4`Hos?H{RNXLVUSahrVe`-5=ia;y7vX0-r7wGa zi^yu(MZZjx&tGc{E4|X5NuSYg8rlX-Ddkwc{yPp^rw{8I=Vy(@z7@E#DV=Kp7bQR( z;(EJcHxjfx`aQ)D$J^nUVHl$7lzwx1Vqn=_Y^8Yaqfltr=?kLbaQV=nOq?N^D$ABWmCZ45b5`+1Tj9|yGCjM zY+W({eLSJK|DsH@@gc8sp3N0WgQcENHc8%p+aslseF#<@nHbUJ=Wk!aMgJdNUjY>5_w~OtE-4}1A}dG;(%mRZD=3Xh z3rP3UNQ;!xh)A;_CAo_zUD6#AOD|o^?!J%T@9+2j&%875%reXn_j&HUCqC!g&pB7> z)Yq-MMdsKKCA-9u@z5O8c1WiU<=Y`cINuG7_i1mTRS&l&E!K&Qri_HOznFo;zd@nv zCtIW-3D=F_RXs|gn~q(#=0Sh!JuypOR-MN?el9%#ST-GRtKIZ2Zcq^9CwNEDlC#sTke?D z@;5p=2)&QOG%mfEZm#q^cz+RkzxFCn&*Z8oL~rn2Ic&(%kC$q#@AcmheY-r(*pecN zokBO8l!QceKI!(S*swv_uF)8POE^}-LqbZ_1n~F-M81fH^yUUJLX54Z3hbu}9@%NW zo=|~e44_+bZ&QS^TpbSP7PYy$5H70xVp@NhHiLDuPQ8O3JrGFBpMSnxa=u2GnVmLR zlN-J7QTy>P@!%nJqxtp1f4KntJ@yaUR(8IPHMe<^D6y7kzKj-;CNqp_jK5=_KOBG( z=9QBGFOU2_h)&Aj2>2>zfJ2j{dvSQ8e-A0bx8_b_Jm3aVY{JIgt!8*3iIlGE?$Cuy ztIjK{L+(5{`I`HIcB2<|a4Ct@a4>VdJR2L-Y6%}-Hh{u$`?569^woovgO%twU4`c# zu9~!x&1Oc5Km09z`S(ESFKL}#^}Q%NQ5$JKRyJZ!UHL8EPzS=~CXmi=+@~n^32dwEwJ~9rfYBKS5dp0`AG6X4_e2V_+r@TKuW@9k_tn7Jjo>HKI17kbg(W>)h-%mQjB?W!{iU*ZX$coq7|GpC^- zm(QhysxZwYd42!gn9+G@YyfK4tTywlfEG1kR=f$Bo7bG}<`3D;dkXcX16%ksN{W6s zDou;KwW9N&;@dG9RU~L+q9BD> zo+^yhbE80v>N4AGPI>?MTz>PO(ol3K-)9a*Q@=H2n#Ti3aDdo?F1ck^B+h%iGbt-F z3zU+Rqkg`{1KsCP82bKc`q-+oY?~m{oMQ|_QGTe)^9r*U8Pk>1iuSmYclP*jSQ;OFJdV$j<*419heFkVYkW;h!X!=aPKulBL1O{Jt*nIMTZ ziC%AS&bCyBAiGh!eb?Et1RB-opA2_yXIKMqPebnoo-z(>9uoVvwQoIo_3@YsIW_+x zrC(WyM4xnBWF;WP9U9EovD5iGrojH-FP{ zQ4VLTERwB3|KbijJ1&G?!iy!_r%eOSj{r(Vzwm3hoOKFQ8TJ5H+^%rQlMKQS!%Ytt z=05#CC}#v!yIK-?728+%c?1no8IQqz#@R)ls1>#gX_*7Pe*$EgQzi0$nJaI7{1|e2 zGoe{C6}t9?=*Pxiyd`O!-va7Kq5;o~WdCeZ&lC#+=yU}uJ?Ht2-8-mL+b8F1@Q0gz z?~l0-Xv+@+d(iKb@@Z6fBRr_2QZJ5qLd0lUr+ZS`skArzv^Ul@x~x#iZphHh$AhbQ zJwftBAA$Uo_x?+sDWY=YabMq;dfkAISofwkA^b++d%zyoQ0ESB+mPDcptSq7Hyz&aqc(DEI62B6XCap zKA9)UvR!&`JiW$w?*Tm{GVb*XTISA$UrM7*jGc8Zj-1j)cSF^nSBq^u20WmU9Ti}dV_G2i_> zX_lvG_hqZso2Mm*8Jl>^fWVyXGNmN|H|QW?WRq^FK^J21SIX&$b#pKAb4^k)9<)9+ z(53-)Y56ZcV-5${ofQD{^btb?aG}Z!uf7{_{QE1=bzKEM(h-`tUZsSYG466FRm z_XjigpOpLGf>-nQ7zcG$mvxK1h}UUj&tZ-2B2?t+i6^T?x6TADeig^I^Q`1{t{irh zY^#Ipue__bmgwkT0j(*9F{*@$K?UvvWVSBwrqvjvz{}N}JxxI7+o%7Q3kT=Q-a{qd zOQL$EkJ$_7D?@X)Ur@V4K3+UUMBoQ#OkxdS zomoKHey0+LFp0HxopFM)<@8gy$yYQLh1&O?b-8AnHQ&Y6lXR1iyoR<4BRkpP*%waK zBWq`g%Z)J;UDqu)Y56Yofw_` z!Mg3__N~xbH{rA4(H&8*r6xV6a+luhQRSktChXSL1W=u+Pjs|}wtD!}K9Xw38tC!H z@}yJPY5(ah_1FX72TT^DHc5XrGOI9~QtSrQqoQLZ3Bg*abM z1gKIW%^5AyKXQa2MgGp@lKK0X5;erF0lq&Uw_=k?OCa)NR)wR)kFNt+<=mNv6ehwo zeIhL)3YDn|th4l}%k--gSnz`&J)wBR;g_|oJHkC}PTFq2E`ygmH3*sR*AhzikXZ1Z zB<93Y=Dmv=QX#_R+>H7`Jrs{fHvFBOqF+OJDd$NjW8C&>to4bLJKQq}`&B!f_jpu+ z9zjU=Q%>bs5eR5a`b4ecV!g9lV0h1pBT>?sQsF?y8c7CbJ7__nQZZ*%m2x2fYoUWn zBSspKO1lD|%k`C>6uRncb5vprUf|QEoyHB-vnK=Tq1Xq=MZbVqP>SlYJnLSH}M~(`>@VPpWG;3cvjkx6H8wsLNZ2)2f zh`^V!mhxBxuh2O`YjcQVxUngxHIlZI80(se6V)ET!ni%bgcEXCaexy1;EL zH_6usq+Ao>f$fseTRTm{jv~}+fP2~5I+22NOUY0ct|lXueek||1<2=c`ieLZ7OG2$ z;dd75e>AgDIIz`;UL0XzO48`cH|YvWWj$|^X|2}=KG!KQ%>c^i0IJB062M8nsu~0F~%4@b8&zEbq6yku+@&*Y7fL57(mFv2arFM+#r`(a&ON z*_PKz-6jXF;62C7`?4~w@JhEMBcprj0Mqa=FTEW@(1iY-l7z5bXo}V&1W{p1e2){@ z#=cHutoj`!&VeOewl02m{$dkOJKuvDEe)=qMSK#W6ZDtmux%>^E@A*teMUj^Fn4pL zX)(u0P3UUj!hkVy=oGhn?r$Nl=XbSrR3*PYrK`zPsIQwO3C)FGu3yFp4QMYs>$~Z0 z;D3EcLraRc+$V6%V96SYL;uR~PEBIHp{ZH;JP2D63|&zH+{5*^^>23FNAyiQjh?%U zPq%g>AYltFp7YWKpuxG3VuSxS%m9kufcp33Vjf#TYM?K4CRnTQkMe_nuZ=c9%tdvh z>@}?ahx3gidUX*w;?9@JC{^8J$u^c|=Jqa8iwLnoH}8nUv70fGmm_2Ar=H5Cgq<%% zWQJb$c(_U%znAjGk;9-G`yVEn!n-@=qdQe$PcaIiGy8G0Qc+KjS2F?^8PO}S&0%Wq^ms_X6ti#|EBMVcCgmKcn6sT=Lziy>qJADE?8 z$0R}PHHTw7(?7K>&bbPA4sgSZS(?3X z&Fyj}xp~S(%fU(oA^2bcWg6g?D^4x9Vu*e!Z!-52$3yP95BFjk)?EAo=jH8eFIv)pc1`f|G>LT9?@D+ zQ686829$bNLOhy{-83@xda<%qQVb=GihFmQxT*w&&%hzq!758Yj(pd7dB<(aX+2L~ z+Q;cYqp#k{-DzqKbUkEy;<$wEBkss}^+mEMoJ2qvZr0p|I*S46K^T1&CW5BSSKi`> ztRl|#ippUw4Snm&w0RX@tsMIcj!DP z4({$uQ98<5LN=%T8y-c=%bkg7u!bzx{_<~LIr8e#bZu~f3;FaqDl(@&Z*!pU4~uN; z8VQlEMo<@bXv|1Fek|QREEOG{lk6d49M(h=pB>)WIrMvMXy)M!W@jS>*AropDB=OE zLZLu@VPZNa`vwE~r$(1rg)ShyDGg(W&vbKLlHTXC?h z;FfPGqW7LfE&i0CD{?6ydi~f9CmjKE?)jk zx8>=Zp7xeLTHSK<=toM;xmeVzFr7Pe0PEolI9;X>&YsO0Rm2_dK8GI%yguD?9Sl5X5%afXs}yAp@W+3^ z7%(XLs15z$L2CMxUS~?7xQc?Y$Rj!BT>o{S=KRcpjlTPJ%$cA)2a=#vB06$QPJVJm zV!D=uqPTc~;TFc-ploEe4vyEA($tX#KC85eC_%i}uRrRz-XvyV>Ay-?#2;TBPr-7k_$=)P+lod7_l>4i0r#UhfT2b~$+_x3Hbo)PU5Q z`)C->-_b>=`FE4YkTF~obIgadwasReHM%dQ6(9DFk?5_1>n7f4NEkI0*fE&-p{4Yl zuYLDZvdQf)DC)x&ijTy`hHdk7c=sMNY#fik5sb_tx28y< z^z*J`DWaqI^4bjAQFiNl0Rptat2t+{IOz2CzJiR{g}(dG=~bVuTb3f)ma;G{=#8&m zi*RKGn!C_*hB1<)zQ=g2<*-UAs&1Rv|dG|2Uih16`AReEXJ0ytEaLrv| zY|`BsFLU`*m8~S)&bY9*k@SHkRll4UUavzZiJQ|I{Iu=Uis1*^?CPRO3oJhvdQ9ba zu^WH9g#B}UzMpm2#uf3s@Abu*;vleO4jrx!SV^#0j$$};X&_sUyX{cpQW0C+Td1j@ zI5G8=RmSMDRxyj2DMoZ5;Ox_A-Zi?{*rF7hOtq*eS=2z5|MdRneA-_(%e$nQ{X?hn zEmWmn13il+!5ZQ}Z$Fcc4^xgk)sDE#2^@tjYH2b~3)d4-V;eU?`kwT8M64l(t3#!S zef?;wy?nplk6!*#^8J#))3N-LLT5xS_X%CR0n=C8?9%6MuN3yn8PcSg z1NE7F(%$`!DLr!onkW%1Cx!7F+zveeh1GJL9dzdjw0*C=uA$mk@_DJihcHGK=K$vp zxQiO;81q5Aa+JK@<*H&9_Cgrx+B~~`^SkXp94sAoSrvw7xVLzR$a4vBZMt z;5o~?z8rTA`i|~$x|gg76wvOq^aJ%Tv*?CmVMX_O>>o=VoGY&RyRW{<0VN1gv9$Z^ zoFj-s=m_!kcqZ%{G;kW51eH16TH0(sd=aXhG0v_ayTi~`F`or(!cPO&Sya(V(+-~x zDnb|oy}CoAIdbMO{+B+Are?J-uI@dbi!$~}We>F9C)Kg9LSkqih<*P--l3&np?GPDcv>?dj?N-?jx;Hc|#Y=ba z6`8&LhVRE2VOLz3IEjYxjAef?Sm=e7nTOwgrr_k)IQPr1+LqF8E~7I6)knJm>p8B^ zm?b4}zma675)zV=x&Wc3>)a_PRSEQrvcY-4yk+7=)4W+7!lVu%LJlIiGbG!sJIkjt z$EPEMrZl2N&y%a(Mfay?qTtfdnXYV9&i=5jw5IO#aGP21)`Y0&ja^H7=eB`n`^qNw zo;r6c#;!{T4I`g7_7CCWW ze4SO+5q_bvwVt7kO;I8Nc-`DD`=2>2G|%^?HDwch#X}ZntC9$y&z>#%n%nD@S0)_B zh{V2xfVYjn%>Ki={@%I{IeVMpgyO0i9mJhLJ2J1dq`>>j+I%(LWb6gVG)@uwXMW$8 zw`Z*pB3AZ@>FQat+JS>?n<*(8x@z4Qj~1#?h!OAQwUS!gyP&nyH_<^ih9D3Y&=Wf6 z;O}@r@?C$3G%R8FnvAt>*Mff8rBVzL<6*zbA3)g!xz4 zeeu2cMA^cZi6gr@cR&V^5=JF|{pzREw3Ai6f~K~&$TVt5BV9W2K4!Xq{hGbyVE5{$ zw0qrU*GC#KmYkVUgya@W97ERnhnE}ihxY@V=cNNCtCn3`4-2hQuCC%k@oHhikXs3* z4tCDt>?|HCe9hZ7bU%3)lp5RIjmYVLLEgR7r#24I8%dk4v=vJXqn@RP0OhENn48F- zT15g>JLy?0Qy=?pNyZZr?WUv?-|921E*UnNHf3YXW%5~6u;q}l)g9)yv%D8F5-kSq zV=q{*M@@^|-+D}F`BP=nRX?U9)Zp%kk+r;bi}vGepF2a6a*xlm>|(!hY`;Hovb4fm z0nI;kW}#%OVwEvl6)|_keorDO~S<}};q|kx! zYve!2hkaigq8~3O|I8Udco!6L;|cN4XS$MLiBf~mnj6TGYI$Rc^mEk$?csXW)zl$H z4v;g+P>zpI!e5BQW0%up&^M>7p{#z_*0Os$=C$Rh^4dzEcgXJcp0YA7ayBeX-$G4p z?l-p|b9~!e@`G1&YCj?G;`FJLZ-y7Y;{r#NYA8R!t0N%_Agiue0{r5FDV@$BqJws5 zWu*_H^77K=5gMUVPjj*wQFdkBla(OnH?Yh5PQ(QsvO+8BOBuMV+jc$hqq$)X6& zPc~a1^;YL`Qa5J&0C*2#G(~d%ST~P6w0#xP^W4t&CzGp;DJV*+G}P0uk+%jon zporejRv%UE^79>jAn;$Rk7?;mTf5KBK*y!s@Y97Jqzj&j#pwHA74$ktiQDEed-Xk; zBBJhTUzZd~eIc(0JkX!8GsGFR_Kkxbw72U7=B%7yXyA?vQ*us%0FMA3TJzvUU;g1! zN)hnq&*5QJ_iDPuz-7cEh9R#pU9pqN`O}>&01vs0* zGDTfh4sA^av9%n&zY!+?H`v+EnA5yXd;08lkqUD>!@ zNvnlzd_3_dAwdFLhh#ccLeo8e_T0W;|(Mggn?#lK+ z5+shrh9!J+7peQn_<}}41gLWLB#fDA{|Yi7Cny~X{z!1T+fRsqm>AnK`yW9d%MZas zq0{X8z&_LT+R%-2S3V#WRO*2+%7?I{+6dTEM|9vd1 zxI5KoU8xMP@#<|XBXmj?A|f&?YmXV3Lx?K*P`m&iJX5gax8cxlLyE@xCuUm@`tmq@ zd6cb5jNj&46@L9{{c2u8HgtruFh5D=m5`AsZ>7lcxmzU%qHgJ0yv^^cIQoae7iRxA zu3MR5w*j4bovG>ZUoOC|r!ugg^CN|i0OJ+V$M}Jr{6iTl^}Y^9SCr>Bg*`Z*!lU33 zkxrrvpJq<`=lc(=?wP+bLJ8hHriGNJ&oO=;)luy@0>9<9pBP zE3_s|BaG83=TLkt%w{j4ApNH$=92_DKwF6VcoIVD2bj?PsII zY;u9`T2Cppyo@&`?$b#s@_A*XW4m z>FuY|2*^AB35ZFM78@xO(Q9L>pyn)+C~ikGutcATl|HP-CZ^_RTY|MVXgYzgb1#dY zmg4UeJ#-SKR%t1l!F%2!|M-Oyj~=}Gnrgm-SLU8o5sOJM<*MVH=#XEJWrH@e8i9^vn1N z?mrmw?vy9bpkfTiXy`atX|WK$8keJq_);sp#d+iggbNe0H;4!rV!!(@-@M&^;|~h& zSD=>o$?W2&IMF|*Y%2^{yzd2?X*P6pXQ}sZ%QFJPNR4c#x8@v>0zZ;@C)etb*6Uz* z^wGnfk(hz0zmS~Vp)({tQ!^|-J1BH%Y9bVfY4Yhn!;!Cyr(OWFpT+M18l3ew34Sj4 z%t&;T*AfVLe>if0WBo+I+}yz6=Xc3JD}CJV1^LlA8?Ev_j3kL-3ky@=QPvx3N1%58 zH0ecv7>QpQ4+A~m{^MWkHB*6|F66ZNyA($2ph8Z5z*rm>=9i0b0?VKL!_3}&_LJN; ztBgJiY16A2{0nHgPwh3I8!v9hH~=L+iYsVPq@*U_*#cEcs9@eMJNBKe#jgI z)2z6IL^vOdgZl`T{DjuTLD)D z&eHFcE{~0KqZ4>Lp!d;@laqff8X)|lHV2P8tVE_qfCoNp-LqPTQGi<58^X(fNBzewa{g}xU?K{NG=E*!Ij6i0 zVf0noK`&o3MPWnQQ5bf;^KuDofkS|8! zC;ocWbws_zWPimGK_Sntp$y~O>fkKx!xM#U_HmGw0zJ|Rg3I1fk-t>!I7sGnJL6&R z@F*a~^F&||Hl7;f1d#?^jt#X}>mE_aziXZ3X66Shk^a$N{8RB4Z5Ho8Q!|JU69s@F z$f^#XtI+D5)@Q4f>QHNE2%`cG;rcGwb-Pe5nL25RwUW0Npr`-11rCD%wg2yh0Ph3r zH}Hx|G(_glD{)`;6Xmk?_0Hb)!euOxm_QYO*`#!jhX9fgo8?Qnm7r9YB3aiz<@-_z+QM7KgJxVOBPDkiFMf zTGR%)L5k52IzV1@IYXe@phtFLG9g*d#R7tp}} z8iEBKMtf#Pp*_ip5Ha%g|7Z+=l&W%V7?*biV7Y&434R6w4!Ve#h%0m`UZ47AA*T>9 z@hg4mQ4z-R7t$7g4dj)yCp%c8myPk+u!x!S+35i8k;`q;f*C5ZWNW2=T)v&t9JQ!Qw%8N_36g9pNoyl zGrpo5e^UYheo~k}HTJSEEnJ`t5NsNs?l!=%CgtBC*hlDjdkwYk-UK2VY7mMl!z@Q% z@9_zXKeb<_`UAk1bADbEh}PFL`A^uk5Z*CV$;ufO#+iE$R!~HDzfx`}_p zCkhXAG1$^FRduB3>Q%MI%F>WTF$W}#x^<@4V~6<#(m%m53Yn5{g4=Td9)7Ue#%Vulo=hK-TgrAx7&dpd#zrs#-lY!i> zBQSfYp=A<7jgB;dfu)Noz=5?vZ!-7!O3rUJTo9RFLJ=Q)49FauLI$VFmfA<&j!@{lS&2u>{H| z!^46Ul*97kB>ul3C3k?;FBQHC#6|R_eW5za%EleWO!NmfbgA-TNR^xT%T4dsw#Tgz z7|iF$t`EPI*}Gz2Z-^U%06xqc=y5?CR4D(h(9?<#-a%u=<2Wc4&*h~#2IbOa%lJt1JXKB;c=;br(ou`w- z?6AYMS2D!J1CeCq(7pwZE$keuJ}}MDNh1JVR&#phhdl|!d%5!2ali*!chCqGBf{(5 zE|)$zb<8Tm|MD3KCsS|G86DgfXMEi34!EZAhmcMF>rO;>KOWB)t`XI6YTZG3gLtl2V#}j3g0baOtaWxv4M?2R zmDURddu)gNuLOp5wcQ}EQ~IVZj>b2Kfn;|wFzNPhP!XLER=zqNH^O1%TU`FNxT>9F zj`qTxP9M)ovlTxADx9e&A@$h?%KRMs;^cw8_T#< zl6~JBk9=F-fMZn-KWMur=e3nL_*sY=TRfK?Qxjy{5gKTsZ7pNf^iJArCi9r98fvUB zV7L&~agyb_3oD3vG|9YjJ(q8xP9b`586z$1-k~O*YS>?{Tkq?sX@5I86=C(qik3+A z_p3;zzQ?323VAAt@2=1qhh|#tr=A0mguiAo%;ecTxGd!lowru_kec4lD5Ho6>H7zs z?0A?84dN-)fbuZD=aTKRE$44K`WJ@!A7tHd%U?2S?2_~RnRw>VczoLDv!C_mIzy$g zCUC?=`qkoDdtkvFD8$D+W9icWvZp5MjTc)kV8veqON%-x9$m14;`81tK43&mRq-!< zy>xi>Gx+4UG)QXhC|WJGv4G+Ca9(!6b28vZswpzw3Q;{TxNF41MO<76Q&wi-!V^nMvFu z?yk&Gs+F?7RaM{eV{uMXQH4#3=$!~epBYhyOjZ(&$B2tz++VoCg^L6UYtQ>akI`r}6!?tb{Mi_vt<6Mres&wh(gU<-E6M=Th3w zjFZvc*utQ3ed+e7*wHhI=`STwU%rMR43&+d3bTmmxqtF&!hVwTe&D?}CIbO@WY$_q zoh>BHf@Fb3tj8{oo&2KDC(X{1e+j4MbyDNlGvBK0ex5@xWv0Jp=F@61gMCr|8g5gY zM*Ehg4|{#HJ9>|&yX<93fpQ_XKal;%Ps9mT-%m0q&}hOkC8=qZS#8tgcbYcYQZ;PZ z%ryn?v8Uk72oGc*m=#MKX1bKSzzUlCeYf&*=`WfbaUY{D@g0kxSKXK}ej#a|w=`bu zZ)nj9jS~~Ga$bC8?icCRF`K>qI>M%7x>~{K^7Z7OUn_Z{l{3Rg>%HxHpii1>GZ%F0 zj#KXkrIaeU3X7aM|46US(@X71Z~9S6QAOvH~EiRvX!KKF5g!we#rN?A`<3!=iNVV&$k5R54y zhSfJN-xkopeLdZ>YG0Qe~`S{m* z|3(hOV&cL6@JJ}qHQs|AvqtT*`Zh&*= zuAXCRebjo4LEE0SiQ^(*&vVcV!?!@oJ^JWHSKq38Hkod7B+bBny81I647RNUr+Bhn zeJ*lGQ9DPCi?b7kCq^UF-!a2WSwToGqTyAJ>)_#$_cuUPilXQNvE!`tfPe(tagx7z zZ1A6$CcfZFx6nr<*aNBnVv;AIi_wg)I@#4z6}<}{x_F>O)*9TIYcbw3ndN4w<$GFC z@Tw#jq^xzfPx6-OeI8u7WW16LqgcYO_d$6)o-Q|ynCx$71&v%IGaeoM5SY+# z{Uut%Mo}N_+EyXcAmh~~?f-+F$ty9Z1P4+X41mVvG}1@xmJ0fs92p2M5C{P!%w2QWPGl{XXt;m@YbM!+{zER)0Cx zHP4`6{N2sGf4i+qK99c@H)+fK0Q9U<1d2P4m0Vb^z4}44fmASb{-O*r`Hr}_fO)&N zh~fgy>5RokIDvOy&bxRryxqjQCdlK;WWJbyAiIhmUNCIE{Avr_ji4?00)G6-#Z&2ni=C4{07k*HxHil;esREKJa81j~ppqA2Zf zL&KqiYIBV0k%oe+=Y+RHg9ariO45UXpbLd{sYxbYAaR+06?ByHD=>BNcWC5u{tj*7 zXK#IefeTX9wuNzmruWY2J+JD5u0P6)+Hk-I5wQ3Mr@G0c^!_1?V#T8G%PoWB4b$p` z!giE;0;M*wD(fq5BkXnBza$f_=avz2^zn?kMyk?VIY17Flww(+TZ_2RYO_53+!Kfb zWdKC@t=z}zERyvz$w=;QAz7qgz6l7AO40WV*x209WW?!lHkDM>dLFE5k0GeO;kjq` zPTEs}$Drvt1><|nV`G#@Zf~2Q{mV4Rf0z}s+t=(}nI2vd$lNRE0+ibVt7-ST42G4r=T2GDeaRlY^!&UtIpI4sscN z%Mm<5KN+oH@+K~cqtRzyNLNRu{8?s}4XDAq<`OV|f;tgPAC)i+1 zW6pF_t(=^hG?JnazT_!j-CbV9qq$opG#&V9{b)s<0a?|`kBiI5M8(r^*ZEUTiRPk2 zf&cC8MHFT&_H9owRJz^@dSmRjUd9S?p0*c~7Z1$uiOJ%#hMV{*uejs1n>TtIRE;zE3Q^=q;hfbF$QvNkL!4h#HGrz(j50DA_ z`8A9)66jD3Qi%%H(i8#r5sy5a0qT+Sr{nDT6&)d)Wi`c?-=?WZG3@PurO)AqnDQ2< zy%YW?zuK<%e`BzfkM^??P?jYUo_gOm?JRJ zjF}J^6p%3e8_I>F?JQ^XgT@yd`G?zH1!Bb_Ux$Bn_|d(;jXjzwW+Q1p-6KG#Gi(qO zyMwBCPLHwz%2{|?jt)93%SM;u;1KzP6zOwEZrFYpU*28JzAHnzpco*LzBx}xegIDq z%yJb!hg^hbDR?#e`?MUs zMK+up8|USCv|pWWU0I9)Vx)jBpFSZ@D$P^o%w&7_Y5-&0>MeFSwFIHc zt5ZBnb^)0{-&VEEea!I^dT#kL&3v5$Yd?7^?Mdyu3&E8SGZ`$gIT9_ z`G7#m+)v*7m@Chq-we6JLEB@$PIo*&)z+#y%%u9_?P0{}A5&5kvWiUM6U=6|RThSr z0H3{*uG#?O%LXbh0wp1m3`=XwAN5eFaMwGB2dcFh&UO9)31&mz(L4)F7gyI+FFGU- zG}$v3s(ZJ$#NE^1PHDE_s!qG2xPe?8ZD~|{5CpQ(`%e~CDGEpeJwN>;7?fu5uL&p0)br@1ye!-#^euc7sIFLiK@>$0y>~d9 z)6qRF4WQwLs#$J#+4>Dj*QW8SbDWbd1`a)U4Zx0ZoML-%i^l>!UgyGP*tf&&CNg4w z#7ccHCyofHh5`2ycrYn+&NS21VpgmSkdm_D=(BjVL%U;9lc;^IrC_tq!bcxivHNza zIq-Xc#^%#*+4ikxBjK6r8_OM%c{Hy~wUbYiPIwhSdThcDGp1}s{tFRZe9>w!c zIx-+o3yTh4z@OfxE-|Vl{}xf>GL5Eus`7QqvIY2CoM~Ed?pd~0<4ZdSn^8Q_SOnQ* zCP+_F*VXs?=!M+0gd93>YwaO=?O{v%`DkNn|7$6?{8)|C$^v=$E>YCb8X3#Q!RNTL zlXfWl`Uyj8(aCh7hZnPFXK)j7XxyzEZ+7XXW)3eS8P4O{$ELc&NvMpvoG9&vWsax9 zZ!Cy2dpmWM-bM0JJrg+o5m#Ahyp9L$cDkRH=^lKf6y7P(yVrn^QfyVG)4w-&&H1^& zXuaf7g?n8C_lM6x>x_$>%)IPGsy9D9f{-$)E8DAee6&_tUkNKW1WryTipZ3_L%W_b z63u-+pB+H{Y8n&r`Zk6O8fVH2e0`tgZ(X!lTJ=XebJ6OE++5oo;XQ zYj_6deOaS9OKn`bHl8;uy_9SbXT7>d;^Pm$=UeGr{9fG2gFx9BO5fgpUBy<^cxFtY z=vh3>RG@rQLZ;Q+<2z<(-{yUIwYB*}Nq}8K{Q*zV(i$>-maDMU$CVIVuy1(p`-6Q!b+xd&A$70djds{rR@(aG^%w9Pj^j~r z?x)t{rDJDD9hVLDhr8|N4JJv7^y8f9{)Pg2;y!w*pe_ZkuHcut>q-cU7QQ<<9<=q# zc%b}Zdpa;ND7D{hGv`uM94iQT@rAqA zMWU>ZiA~Lj7;E&8$qBrn9FNS8V^ss)4j_j#3zz~bFE%+nY2T7>xXM_xTe4(C5V{Yv0IEMGW-GD0v<2eW$;!zO zyc?O~(*0rXO>f~P1HRQ~-4b15_gCsHXgsQ9Hp8-+FhyJCxr>)^vyUH3j-d|RbZyDy5O_1OPDnFLh+>GXj(Q}z!H~dsO3v_?REyeY-|c#+}sjZ z@YOs|f{LtO)+;R?BdF}49B=JEAUsM=$J8t~v$AgK(r;Z-|2Hdewv#HWRsN}6*k_hQ~)RJRYFFKjVf6HZE2xTs1T zNrONO=23XnuO6OWcp!H-c((6PJ>%GvWP$pfnJpK|Iv2}?gRd(nE@-!hqX>ZNhUp?@ z*6EhUKwNmO&%(@}#YC-utMk15n|b?%U>0DxWPQ?e1KK^S)?-DVFmw3m>w#xW(OZi?1!ZKx=n^eY% zlfsnn7r{u|{qVrEGy)}KWd^+g1vM&cS$}!L#R<7lv~!FH!Gr&D0ZL-kR}E@RfkhRW ztg3>OT;U>Rd^G%?_&Xwx28W0@irQ)29s8mq?3dP`Z7vQusm#*zEmxR04< z6&A{Kr}lFx{sHANA7xMjL*R)OZQ(TRAGogkX&rJS0#t1k`u2Fx9FB`*KbOh-ezu>Z zm{4uc zTNE9p(^U8FPgM*~G?vT+N8U6NOz=9I7xU=kE9+i3{jD;TicQH2@OQcJ4OiV^2|;AM z9hGle+<2T1aO$3ZL8?|i)u$jxNz;q?YvLXEMPbH2(s(i}E}Z59j_+P4|M>GaODx9# zHf)`tl2e&u`23%Q0d0T#7uA#_zwe(Xv)bDm{k>VS?EdSor3FYp0Ko^gd*a&GXrAm5 zV?TFmC#Mig3mGq)YEY~3dM6*|u2a?q%d;RlrC;@#p9s|+&?tSx+rAFB0wZ;s2gHV6*Q{dW)>*&i{0F3}!Nl5lW+9d0cElY)>g+XPEwIU%~p-3oHh?GPk$ri%zJTu(te(Qek?|WbO zkM3eT^E~Hs&inm7=Y5`p;?-eUkG_B8lSHSx`uHCb^8X&(-V~7f2y5rLC;kddd3^s? zZp&12vS(e*>#ji@Zs)s9v0Oc;El*?K-y|w{-(*0e^$tq!zC>s1AcAIywdIMo*0{)L zMY$vTL0e;1k*+#^F6T>h;_>`3d3aY5pxh&gVdoHBp7FKxTQbVwT!C|IK6n5wC+tunvZzj2nlxKC8t*@z2 zSD&pFayaSH-+HS5+wxA7%E`v|;yd?`I<+e~y_xOKo$Bg9qp~c+_Fd*hv5V$5!029A zE!^puxE}hqt?@_Sg_L>SD0bWlmNZwBGs)V9OlWKCBl_i~_%4s{Gj2NRby>N&^SAPb zWf$IyFXU~PWyvkmFK3;gja3|-b!iBa``vmaomA=$veQ%qnG8s^8qte7JP=16$Hq5wr9o(elVI57Dt$r3c#i zFhRAQk<)DQWv-%@uhqnr`L|}1R^|QZQ-!|G`v|35A7qu%cfPSMd=fX#6CHiY`|hQ} z!(pB`Se@La9^QK^IO1R{jAGhwU#!SBhz^y25#A#5{76b}*TB0$8MmSqXQ?~MBE4f% zJfEk{-zFRL&VXFHt9Yo@7=3@fJR&WbyBez}wKw78R}^sC&N_7%Ht(0f!^ z(;>}}!=17ym7}%EwgB4rbbt4|DGH#*!~F9=Bw-(c_8TnwLuc&Uz*JYL7g= zji%;=WNXnBacBgb6+m?HTw`qOV#sU8I(r;gsl!j28s}p#5_tOeaJXJPMwc2Oe5iMU(m)`%GspWRdcnh zQ%EI*h;Hx>;re=7gDzx2PR(Mg<#H=%sUWQ)}qWlu{YTw#`GX|G^nebCEpsW->q)k81|?}ItoveC}+ z@bU!T$&`T1V-J<5YtBX=c=qkPQ)H}$!c^lkqhi)9i^R?QQ$&T8=qht737T&s_{)+dvf&l+ev=lJ%c)ArdnxmUGDSlKU+l&>G@{?bh6 z{q5QGiRo#+?JsSv05Um^sqO!~UUR;s_+-qhzT0z09WX-(6A;7ahSj zKFN#kK5p;-(ew@;>))UJuEqFLjBw6-xn4?E_7nLmi^m(A8j819%==82eEN#|VpT!# z{ho1Wc9PJTd^|L?f#qeN`@LKBlBYj)Zh6+)>Y@2|(DtF&OmnKy4f+Y!s}<$WyK{}N z-QIj?8OnP~ZrWGGz~HLJ>E_|fn8K5DY|~G#XylRf=Nwyl#@_}cO{A8tOj>s+=Z(K- z*k)S~$w8I;*}~^+n~&2~)^0y~CQV`E@WLuL()aoIey8h8PbO@r(D^$5l`V93GXCU< ztaW+YBV38Iy+O>5Lt|$L*ITI?t$-IZg?gp=mDtKAq}%edT#@28+JT-hx}82rpZYH4 z5zOlC=O=5ZTVn&Y_PNnvk!S2N$}UyeGLu<#|{PPIliLn{@x;n*08H&(^0|vxY63FW+TDOpB6QCi@DSrH%=SdIyZ% zxivfSl||k>BRp1PQ_-nAZ7-d6uYB!qH8otMTpAtI5v!>tx#AUg#lCn>ubuNG2=|OP!-;9C_JvF_t_5 ztVr@~UVv)f_Tam3o{nv(DCoE+s_GaUr3O<7gO8upw|DprZt`GNwafHxOZt$HzCFV2 z8O$!6{k_lfCdcef4+DcC`2fF}!k1U9ra@C3+xPNI2Y3?7zZQZAPo~hIU_rZYyUn$k zk4=}I{6LtTAADjK=4&ZhSn+jpOWts@O;@JY^60QFZ414Va^)^L3&RN=d7l;(7Yd*{ z2y3|qOg}Mu+W(Ob&aV1+^UxHy!yk@*6zKOT-&A%oPpg0EHKBZ}`(wMnv*+2d^ShD@ zuH@#Ky-`-%#zJLJl5A~E89qGe-G zM&0uMX8}nm_M0^Xp0fMPSFU4vwClu`lt=i7j3&(jsmUBw7T}3`BVt0Q+btHkePM15 zkHn5oj3(h>maCQ9YV*F^R9W`9Yjm8nkreGnB?@uB$KA=C_voKvZo*bMty_9p)x zu8wkcK3&}TFd;@+0jwo!lFY>`eUTg052dKM#HJi4mLA#t%1L(rp$=ZKz9davFt*9N z<#ndm>qdJUy1!8s&d!Lii-th9!vyc=LC*uPL_~6xaz&t5?XL+U9snob;#2eYH!2=T zQFFLtq!~t6S)cHshop+N-7v7Rai@yC{bWS%WM;Ef=hkREMUPJwyOJX@A#$VQJMRNRb$dS*9J#PAtNJx&4&A>n8xHTCg;o4%3e`WS*&#numy3!^v zyBNPu;|Tv_6%3=u`RJiAIZ`6b{GAWWSU+pW4inhjIw?YcNa2DY0dFW>o{cfB9v4PDhiJ}*?RQi@EbdIo-so6E}oJb+~X z2EE#aI$Mg2R!Xd9G&aDoJr5%0>bSSAzMX&7l#M}PEAH3Jk#rCZzyDo1C;8n@I(ki@ ze}=Q#ZRqhhBaAyT!LU_3ii}K<{dEbb;<>l*q0Dlqj&{CD@T`2o z;hI;jSj4>IuD#EJa$oUmd%3=^!m ztEC0!6@7F!F0$xGb0T8 zCJZ9;nl1s}aI5~oT2YWjipq#gJ;8IVPb2J{9f}vRq6pL@ZUuEdBV`z>JLV0;kBd{N z8(`v(=0l;)jPgu2Fk?z6bTI4><9aZ0xa>ckFDl*EI>}|AE}ax8i98ePIO5ulW+x7` zIGa);v3_*AXg8?F`XY?Z$Sk|}hBc?b4>L3v1ris&>Bsc*T$b@R+H@NT1Y80N-ijDk zypGfWrc}W%B;k20J9S)EPbB%~P?#$92UjIhN1xvfN4aqrkKRO4C*`hFr?HWdd_-nA zo-XFhegOt89Zi}E=3)o=@%v$FZIZdM{16P-`S<8>S#p44y%~48KJSi6bAXGot_$+! z@c$ZL$0aZ&7npQH>H1(?n$dd$kKIJUgTR?i3&Vdg&kU3Mg6oLH+=GGVOz@{BcIi}A zo{`9H#FFGtFj(Kl9YCEF<)s-d;ASAVg?36{cG8wdkBwUoXL&&T@lo+rSs8_^8dO-Z zn8<7inBR6=v1s*yP4e2{kATny?J+q;DvjiB%&Ys?K&e0?0qGIM-0WH#r3e#ngn{b48krm)sT^Wal|plRIPE*_9l06J{0$4E&RMBp zQc95TgkTQ+0M8+8Yc!FhYDk-9wR2OyIAX*K2G6(REH6IqT>B7}PWnzAtqk<%#|Ly` zmV-OM3YBDl`}=t1ur5x)xGN|8I@sp)Fle0mo+zT+Ox z@=T6lsu<@M>vM~Sv5Xxg5>XO@a+@W)3(vA!q5F455eJ{g5b2VoSL6wwg(pFGof?hs z3I6OZ9X#5r&<|JQ8_KUUtTU)0BPoTN;$R;|ZJEhV=q~Z4lcHc?))A5w6k|jRv@d8*QvN^hMEy zJ}+30bHG6GaUtnQ#N9S9F`nqv*MmIeEin77j1vEQL>W!ogzTb_^8Q~<>>A)!gpH6K0r>_{g0$~nO3}|%G%8(nkcVfFJ{V|UUOP8;Sr>5?(jZ>V z?86$Ewx&uo=DUNTN zQgbBFp75T0xYCv<>L3l#j4ojcu3d_`3 z4lY7o9xL4x4IZAcs6&ZK^%37xX!r`u8}I6?*zO>CraApK9mI20IOx|#eFZGf)Vd~smV>vj1Td9l8`utg(3;A{ExkVjC-74SZ9dX z@htbTY3QX^y*gQCC`ipx)!_Yj99|V?f)D0L2loY-2*7o>{RD)99zU@$n?Ak?Gb5kl zC?z)|MQ*#}Xy}hkq%6+T?>CNJ55@-CT)mwsM5F*jB;?37{5^=%X8d8`yUvyAV#twP ztd;-<0WZr!@yQt?OMEXA&>A=bmx(aOL z8iq)QT95L=R!SwpT>ydPZzkF$&q5VBd7tWjD3{9-TV4U4)7jd8B}#V1uvj|ju{4;t zDn5q{y%YwxrWHmF$B1DLC!W@WWQji&%4$I$1KjI49yUOlWDV7o4+8<6thF>2W^{J? zDT;oVPHUN@BL{l=)X{yJ|)$I8|;lV zCsfbWdp&xcLBwLqs!lY)pP1O)ec0?8b$84qC>i;=4N8AAvadKia3_p-j!|$2uJ7A- zAvk-2d|WwCV%>z#F+QN0HLwk-52^02FLxcO)`Q;|O-pUp&=5OPNDfx_)Jc(XQN#-v z9h~I@?ZhKI@y9q%sic#J2H^{>jrVINpd#f1c4lRGn*}CvWQHcgyQL?x1;Dv_790Xm z#9u=!66~Y=oHwZ^CLQ)?45VbgI$D%j)S7VWumql%JmZIW&@^k6{=tK$%}T3=YOK6# zb%HtBLz04mcIn^-SosZS_ZiL|mRS=R)2JCz{@umN=2aNDf&T>j?ZV()ZT8sRmRL?l zM)VxtyJAjhq&>bdg!ofs>|9+>+i)Pu=4Wn9qYO~YAo9`zxSlmPX=;hRFFqHk$d1&r z|0UkSLx{QewZZ3?w&ViU3cH&u+wm4W{n`pY{ERd^H|+v0b|TK1Ke`}|3R0V3^`M|L7M;*L=7AXIto6)UFeMKlmzO~?)nDv-~kMEgxQDiig6Ow6`qbFKTrI;91SPU01X%j^<*d1+Cgw zi8OJqN@(^C4GItun{bM5=xvx4sGbaSS^g%811Ya0(lV;f_2wQ?>73J2L$CcX4GKoa)ZA z>?w{`X(1!}Ijo;s8$|@8K2Q8#1R!YZxmV)Zncvlf&~uk8HpX8(!L#fz4*443u!G9# zK45Z#pE{AZje`*~J*O^?(-`c-?7I5OkJ4cghQ#BZtSF3Z^EPZHcnGsHco|#Y+|>cZ z(7IClrU|Fsem5>QWWS2kbiEx9kHsw5*ML-UQHyqK{zDT^H76`{q3!#i{9ZiPGc_D2 ziZcrk3a^RkM-K|_$REa=t zw6u%7dgBi%F3kcU1}bpgnX>s^!Bg2Le8Hf25=n-K*({m}q=I^0t_MGU*b)zw^Dihz zRm%{f2FC-crU1H6&lQWXKi4gN<&NYuo>Wp63UWIBIu+kL!4nT$1ah#u#bn6p-P&Jj zWD>E;79;(r{|%hu8|-N7hbxggag3^d{8qa_Tz-ArpcHMUZiIPdOBO=oj-I2LWD)p~ zW*o!NlfFg)ifBcug9%1DBN)I>jTBUQH>#<3skB1O z)ReE?{~S4=i#AdrAH#>XKNnUd7D%B}!VjckK!%{hhXqCbY#sF=E<@a?HEqqh@y2>f zm2oM936`+op2CMKK&Zv9f+~Ot5r7X!C+$GNmLLSbT_gV#QR61~ET^aVDY$BUQED21 zF&c=oWM51oCR^~F@mmg~b!`e!c8l^YI7RbIUi>j_(NtUb%>1J=NNp|1O63E|rdl63 zTVwqwJ)|G=YAgxvzWxn0WDf{c+WVc5fiO#^}u>n&-ekcuY#lL5pDCmxxnI_mP zuu4IAn2g4p0f=aZdS<+m@E__0O`tspYS5Z!qIPzw)31SKq246!lkkjD0b`V#tb%C0 z46%<5{zNo0Lp5ny8kK_P*I|X3&6p9sNTZ7BwuuZ*HeheYMCKsCD)N-UriLATdcMNz zc|t%}1E_bTuZJdf`mrg9?GT5q0C0;^HH2Fr-2l){{!^X>i2AOyVKp29wh7M-0g4HbF)?Lq$CATnEJKB@YZaZQcGGqPZ5%_-Vl$?N>oanvY*jio4BG zg3-Rk-S$HXJF7`00T+N*FG4fKYWxGH!96e>uXC|H{Id7f-i2-NN6%mC2fo?Xf?S7DdSbcU7-2bXUu#twk2!f^{A%fO=ZYln@IGIOo zkENuxK@2L|ez70$VX&#LdpJHqYy<02;NSD2y|P&gLM z>&~JUMR>|}1U$nEgH5+Z?}FMKmpirkS7q&Yq!Vsds%_uy_tc*SC5@EEBe1URX9hr9 zx%Hc}2_ir%p5m1+)Uq{PY$x^60@3rne0y;k>ffU@MA0~OJB=RX;;)GIzt#E5jq7T& z7wD?JVe0^i_%FiCVEaXNPN0zBv@gc%(98-Eu6s3VB+o`ppK~3ENF!M^fs97F4d4oq zS;rHBwS#I6HycrKSNM5#5IUd5^%>GZYKgZr2J7qSVVPdoJB0?&O-P$6%dveg9PcIhm|)?Ejh(a+ z&Ry!W2P|l?VJ#xDrG*nqG^ncQ&Jr)fbTLjhWPgIOsS@qb5N&Ti2eImVAhuVEydLUz zJ58->eUp5wO!Q9+N{nPM_1=i!1`I`ywHKLnL0S@h|Gl&HtEPd-;I*LM3NEU*h?MX@ zMwngo!8$$XY0ae-8LJDPngXe*Lmf~D85@tI0W5{dJL$h7wM8`p!34b`tS=na`|Vc5 z-1ONH&xwK6_pmMjX&})N)2cjz<)J3uL zNDnj=wks`eX8I?ti70w{_-19jAZ!~%X&?w1rqO@fdGqUvH`jqqc=f^JRxU(L1UUi~ zRk?hL*3@ThUF<(P+)R~>1tAjAE0#I~Anb-HHPZk*5E0JOaAdI(zrvo;`&zBRE_6^W z4RF(|Rc3F)G|vl2F;bIqqZeqk7!iSBqYdoTfKJ*rIKs)$x!PH^JGz1;pdba z4-a*HP2;`EFFYx-(tlSlmT=l%Op_K{hwG8899B^#cw^Ksj@maA@m?0O>cRi7fq8)1 zN%`PikS=z;5UHCwp6V8_eAM}#ucY;#BIEtwQFDm3NHI&~6^4gKLb$~HLh}!k^`BA( ztx}#7;W>6@pF_NzwiUEcE@Vk+uDl5ycA4LAsG&B$bZ|Xq!w0E$G}PL};#oJ&J(Cbi58mH4jIj+GLJp&Y<|LXV`roAtv#FK5aA@kokrYXVb8vqEOdXJ778J_A zoC-JP-#0kVg}SjTU|zJ2!nrxX4{#3}qY)#Vw8;Er^a5ZUxo6wbw!5I+$j(Uq#h%(m z=+Ll;p7@om2wPA5!dT=xqTGL7E{*VC@XeiCh-a z3XzW81zHBI#J6L?Vhj}5v;i-CXY{+sT2Ys##XmHFv}9Vb!BPW8E0{CWZC}wxAgbNh z!S#*hiAZnpmEg<}vPmjZoCZC9)-a^duxV;Hw>Z4n{jSYXLq;l1k|N*t8K*6)q_)h9 zs(*s^h5@=9F8r+2F`;N4%-xFAM+lBukyaHVy_I!QlMxRapevA^9nk5(HwW+{ot{W) z(>w)se*K}vNo%%E9R$kB7_!V3IbR`o&SQl*gmB(E=#)-ccESHmr|gbA9Z zC2YI^8Ju{Ka$D@>iBtkO2k_cK(rXtP{z%}F+Evhy#I zJ(^b?^@y4XFV78z3b|vFp?p$)98CR>~^5Uosvtb_Ti&?40@xIZKKebgZC9 zE*xuz|1p-{GX-ZZ@_}Eu0jv`f-fHo+DYtJsi`GNkWkm7DCPXt-?Uv)Zv{>=(v>LU=LLDsIcU^F1 zQz|JR*|CAt8>dnYFHz~XQK)BoB-NFVeu@|`~|5FMpi7Sof4CME-5T)Lx`j- zHq#|qK;4h2M-|ARSi(5Yg9Fr5^@AB~Qy$OM!)>GWfICc@6Vy3x`fa%W?QX&mo{(mG z5_J%ft?cgR)jR*WO(6CHDH0M?Z5^Zm&Qh-(JD?sgGPh{*nf^63zBS{UOY5 z&ryeNTG*l0@xOC|bHJ8Bx|5)K0nbGA zpFH7U1a8p^Kf(~;j>71QrN!JmAU7*?aLsLu>rbRzwlE&8OZ$h>Sz4&hn5U7hL+yt% zgh4uMr%ymJeAfvb^!yLazv8ujOFem|)b*1G`#~B*x$Ee2$^eJgrE>YWiI>CfWDIvKHLfL`YKGd+!xR6sKt(sa}G}jRKVpMhJeWY7n(bFwe=&c_A!lC3P%kN%KGm9J0YUQ9}2x z_2x*MfAJ(!Spp=tbFB&Xut&0v+xYpJu9^3(K_Gnad;c7D$~+B8`O{u*&ic6?K?<6V zDJt$9Ddc^}kNdl?Og`n&QO*O4RTiQyRMy#5zQKUV>Z3e zx#i_&6^aa71;M)O!rDC-nca4mp_kQ;ObZp}D3;&s` zYG2iThudwZuLm6Lu(D#&s(4-cy>4F2IC)ZDbCs;@3@(t#>RE^D`26W>^WLVTfh@+1 zPNIh!nofVoFBGsYdp_I3)YZLzuqGpzJL7xo!gJk_j1-m+F+N%g+fMr_FY7tw{VucT zy+D9_qfMF6{sqPR^A01KPQk-W-V4`u1zlQq+*^C1qH+e0W}?p|GPV-XI~5PZpXkkT z;^(BL(p?XDzN%Y(JC01exLb{tfZfVYz-wQ@_-J$X^aUJ}OZn29A}cq${mFc9zmQqW z{?fA3MK$^!E;SYYa>Bkt4N*k}?-)w8#?^g4C46{(n}gJ1z}YjdJO454`rwhyOtfEsX>5@!ry!wHr;f(pYwEJ?j+{E=5jJ}s{{yabK&OL zlk4E-5c&e8r5I!GU>;P7xup^nf_N;7S@Gzc*!}y^I~lckhtiGECI@tF@M!x+EiT!V zFAq5{b${G9ecIzhi{xu_`Jav@_? z=xmd_dt>SGjo%-ZUdxeFpEmdWtYd$whUrUtMp=blM{pQldHYqHFCwQ3xLDr4OlCp( z`WH{lpYA%bMNP5SZ>;laAMb*U&Ef0JcFRz)HmnxAzS)Ho1$0h-HR7nOYnk6{y++dq~pD8wI8>~#e8lRGBij(Tpiv78T)<8rA>q-B85VIyLRjbn#yY`=2PtLlCm%wj`t zYbU;Rv~hCHwD>4=vHOp@&c1k>|MFpu&|9aJV;VgJ-#`1Qtva@))Ht^*reszk;)GDe z+;JyA<-sAnl1LM?cs(3@y}i02o2jSqT7FjyO8r%&=KWS{ ztLU4tYqIP|j=S8wKgP#^3b@}o_ik)IzXa=%VOQ^uQ>#$OQD2wxSg7LA!pDc7FIoC@ zZP+U}^7;&_a;xYut?8pv^9IK}G-kT;I^D7?7KXgwd z)$;jxOwAPY+WAmL*6aeW=M{EEm%EQ>=rkQR_SpO&KF)05WJ-u%#$m3+OK-V@_a4IU z>FfDYmpOCErLlfnr$a%`d^wG7u*~F~SUG~e0mu3O1g{{qTP7G2GXFOws8!`3d zy{mn`y+@K3H%fEvUZl=Rl` z$c-smnUbShzE2yUnB!kfjTgw;gS%&m(@0=w#6$*N<>evwWs~e~?j%u^ z(vc-J&#iuSapOvUe~kSM*rl;}^F1>g1&E@y)1inCV%MZAlk%pk#$V7wS;g8&Sdn z%e&#>@l*q@?tr4P2iu=our#TQs*m}425c-jp?rTve`NCMm&&s%M13d8e4;m6c6}e4 z40d+fa{2P)hnUYc8{6NYB^^(^U}$zpFN^a&WFr=hZ)<+J8-3L2^0Jb8qDw5h?qX_}J)Ex^!nRRtCsAZ#BO~^iw92_{yzyL(O|9`v zV(RgNBJ(Eo_S~DBG+%pv>?}32+tT^0)}Ww#rJTsvyTgu*Ox){kzPZiM|G4Mtv$+6K z&F4*8pK@|SW3RF7x;Ebs9_>RbSa1i!voOb#21+}h zDSLR2UwfE8>-Iw~u9FIopj~?m|adz(A~V z^~Q5sV}w~%)kbPq+?XHX@pIEP_pj8GT?aPGJX&jx+hPvQD&hgs242~Z z(q=4VqK+5=Z+plbTTAjVJoRCdvF7kw$1bZ{ROtMjmkSObe7>*Wddu+fM%1X(fs(ps zT@^FM=}|&6*_9Hk%hXEuHP#zj&60F z3>cVIIpI4Z&b5+nZW_%|Nn)el-90PB`ne*g@!hy?wCMLwtuk*8*Q~x@Az(d!eX3Tz zbF{kK_wuN_d#!2zDjTEZ=(25%)kYof-Bx_+Yb_}9JhX5tIn2!Iw$`gDEyga>+G<-D z4<|e9XMgUb^nm_E>HYDk!2**G4*IJbW)DQWhENVQpWW9<>T=of>9BOCs)1b}k8HA+ zf%B|_w^Z|mYFo;@VZ(8++w@1L*qC(WQce|1C!PqbO5YKJVb==0fjQtsXs!x-b}-Af zVP-G?l?E31O|xf(M1AfZ>1o=Pog0eUb!{@^rUoiG%&ENl>b0>Mm#)Uo3&TRBu>Sqd zjg6;7dinzjCB$_2>9y~CL_5xQtntq0g%TubJQIB4CF=>}#<{1Ql|xsjCit${1l$s6 zI+5GOW`D)+!x>V0$Exwx@y{>PE%QQ3WprBh$4wM^+H25%nebSabCIcXYIOW4Zh=Gn z=DQE}U$FWNVJkTZ_#2o(mw`_dBL6c!1&f-EFQ})8LgFXm(cCh3r04c4=V3lPj`vA1 z-66IUeOM%UOvHUO^*xPIJKLrrvg-tPZ*5^Z{NeQ!`@Hs zzTd`u`$sFr=3ejxjA+~wm6M%$BI(=oaM&JA$SPMh;RK=fGnN@r3rS`-N)ov3P&V>DblTw{$EoYo=XUoM% zPk4$+YrI2by5}$V{pqLSYljQ=s?43M@NVez=UnBPZ?*ZsRQG2+@x5{YJ#yCN`U#Ye zm9kIz`)%dw_DwG<<~pSb0Ix}NXtQ9qsMf&zyS8i5(ax@zSl=7B0{V5E__dYto%(}w z20Gfh1nzsI9T%j+L{<2U0*aVT!i(MvWp_vWa>S~4-1~CSb1+JD*}Zv(Z!d&O14LFA z%d?_h-xrx5ES#J?aqM92>Gv|xcVd-pW|T#re&(jtReY&%Y!HBiynjdeX_-Sxo`>q- zg}YykzD;^;LHRJebn>#m9#wZ|VC%^yiLbt0^*X^GE3liv+ zdfulPi920#E1VZJh{?)nYY8z^9t*dQ^k6?0Z@TX1*siYLuc0F2tcGgDOqwXmgs1?@2!|WdU`=q^F#lfBg@q0^x)2{p)Na@ zx|F1e7cbLJf3jQfSg}E5taDgujswdo`u4m)UVHV6IjQ4EpQWy0@Z%UeJAbQpfBIVu zZ_dNrEKKr8ghSs{e!pMg+3>00R=J>iS8UXN;ff%Oma_{Ywf3=_lTY6|HS73orl|v} zaON@he%rK?Jyk0dKcx~?;hPJSUERa(5WPs02pj=U+a02mncAc!Uvw(wgHgUR9 z|9dJj1BXyIvN;>i>+;60&r02T{!O?$i{_Bsg!?@oOAPDj(zB{z(UO=Wvlii2b&vFS zmtC18XLD)3iWE8cS|2k$8B<;^n42ev9@cD8etSqb?OEl?1qY1ChPm>ivyS(=&Z4|U z8r5H;^8GiPwl$jpMY7Hk+t-9CVGpr}6wiDegK0HIvu&gjrnq#XLRGVUJjryImO)te z!<`J)IhfCDqlz#$MLRY$^Y=?LI|T~|KS7^0%c{rev-#bZI_6(>`g6^PN213)_Ho2E zc-v^aQ8`l2ar)bLuco=n7j^LHxFm`sBbAM*;{9}O9?Z6Tp)bN`Cd_lxRT9=n6)2Bc2GTGK5UYa-L_7k+^ z9`vW?W=VG!d=<{fWk61mki(Tz;$@yiyIS#c39I3U5Ux65h8t^84+_dUm4h=VIognz zW&46ome7pF5v6zY0LN%!^LuxZQXiBRTlEl+9!|wv*5-}hNZ83rp$JRCPVSe@89Kc2 z+oCcCM+jLt!XG(W`i(!DN%A%rkuV?5&_1JvO=H{w>b%5Hbnj#g7bG(hAV=EW6O_&% zbD9Fz+r_KICi`H`rt!E8;xrIyxbz#Cdh-qlqNjLe1-KVvTpu1ig}+WjjgP1*jY{*| zZwN8ReM;xE#KqC!(X)6o+USjR9TrG9Tx%)@bI)VIBrKvz(0o^YlX)iTZfG))B7ZOB z9CI1uOpx;smlT1wh+(CjgeV=fT^aoo39P(1?kJh4+8=i(qU_eT~gFjo>NP z(jIm`9cPM0Q}To8*`fV?6q_lQ24abd7oAO_UYL6fe>abn2mg;D^sH zm%r69? zBWp39l+(;{Nfe=(>YNBX?(7Z-b%G^3p&3OXal)@dxl2f=QUPZwratL`H~aFpsSr3H zQPt_;O!?-xhs%~zY)!~SMZ)Is2rEHC^YNQI#o!y^3m32YM=-tjy+tO*(96-Hi9lfY zigMCsenO~9og!f_i(Hd_6^1-g9!2<1OVQ+KO=w=zvX1(!k%(5LtYbuevSK&R)##cz zRwhU~k&i+`-%zFn6bsX?9eq5$SEYINAS~fqSj5ntEGDQoo9GGSdZaSQ4i;`?GI5i% zvm%OOn-bwN(3WKzvx6Yb?BJ#czr!3yH=l5d4uXPANzK622T=~;3CMFuyYK{k&7yJs z!n5-f+uV?M4E)iUNG7kF#E)B3F(u3e*Rw-K^!-;mUcGug01uA>ZVPD`pRu|p7zzBIY6KIMyZ`6@VyZ6rL;Fm&a*fcJoY*F<( zXd#j7OM`1NIG zUfIia_1hSs9*{7Ypc3=^D2*^OaT1SyiiA|IIo5zjhhkf6Qq6!zv!LO&y=rC&qD2xy zHVKw7u8{VSQ|&~#O|5y-;gl0GshE`#3ds}}D?(poAqXK{4~Co){5c>tpFNqg-p(tF zA>eNghH4(!P8FU^k3(8vBWGbQAM8Mm35|>xP`?RIRhs{(A#)a?Dtj3Pp@jNyW#p25 z@YJd2CLRuTtRYaxrQ>y=Mz%{hopIMMQ~RHy3AL%dPT-{Z#3%mM}vZE!dMDN6gN z6jdD+xA7~75ru>z;NhTGUbA0DMnC}Vwp`{nr)Ar8ld43qPZ4Z$PP~^`)YN9aq(=oW zmf1P!0i~nz=cJw42-wZ$RRQp}}MF@;d2@#vil(MNKL zo|_Dj^Kpp4#Lz{>m-U8@hWNf6j!)q zQe(or3(78ZbMa_>O0F=4M0cEki(9&DfO_i5)F**YCyp`)x1Lw~6ePPU6aWG>vBjGd z8vJ$3{~S?#yvur4b(@Du148{>c1obuPI55Y3zCJ0p6Fr8NHeINBb_`*^F_3Y>Wk?fLa85W`Kdm$oMXvvwOIYc|h->SxHeutvD zb+O)=ZgNwJ0zGOML74@?8(HnJKjf`qwVP_w*G_HiK>b$Or~gu zMK$5iq6l2jX`3m!@G(6gY;y%D#M61@+iW^u$A#FpdCmxaxq$me-DhzAggSOuh()`>G zpo7<2sL-JqzsmlHLIHJn+VVK&3%OSxnH^;HKrW46M{~R|^ zWTe|w9Tm;7j`ukCo=DqZj#rP|$h$)eLul3~t)(DmxR07;JjI#|F%L$?7C}5g*JeAf zQbWOVs5E%Aq!HTwIRe^VZ|C=bpDwT_DKru|8yS(rI^=~^fj^bEBwaSY<#Fy4*$9Az z#2PBC5+u#<#K`3S1dl$NT9LYS`Gg*C!PWq%2S+!m0o>ol%d-ep?Q@CzSyWzy6yE$0 zWj_Z$E;D$E1(Q2R`rs3^TN;vzkngYZV)vY$7c~kli+^=*0Op}%OwEORrm4evw&zb@1=%yBP1gP`dE}y(eseYf7JIQ zfPne0(^OvWOriqQnR9<(tU>5=t&`_N_{vX^eG%J;oO?L73_-aB&KM?mY;YYx6v3Ij z6hxV(9vnm723@?5(^Ulp751N1=nI!v5C8+E3rSf7V}#%Unt@1cNDVCynYz=5kv&@p zHwZwh+y!~n(0fbPh(Wd=z zVIzvt_L)T0`7(~*Mb1BxJsR$C2+jGpkq^+|zz%dvtPbxnF6V7~*fb*VOG1LC28jyi zz+UuQ?{hbXR3Q_<@nI(5krRN2^LggH{D>mYgQ$qABlxOu-A=S%x()i8<7ae9GXHfD zI12O#K*n}BuG8703g{l`1!FP1Arn8s#}v&HS7I=ZDu#ozTPmM<=2L{UclaPRdIYm< zKktase#sr&YxPsLTjJZ)bhZ1zoy*Kz^uF14P+3%MXx6 z7Qypk(uLj+6|_1C=YC|leW*DJQ1cn-0g2`dw*`H8P8t7KP6wA-JKbkp<(*|alR8Wv z{Vb2TYvUAsn=$K)Mf&_tNXfkF!9C~^2DVxQlZ)}pB}%0xlZjzamTYz9P3?IbuDPrr z`yc@MhaBLrba)XibTR*?aT(^pQy1B7XnS_%x7R8EKN4s1qSTnsM>@zG#SmPblYP4X zx*h-}(;GzmT_kx?A&)W_L}FY*9P4$AAntF0D`{yQ_=`*hadCaH9%LE3 zC9hK5B2+j7IXsFchEhm|Yi%U1yMtyX%TMF(|BA&ybQ+u)t8>wDo>rEm;uO#mkXm;W zL8?@N#7JnS&J5RD#ps4tU9V+^+?NQTckHqq&3wY)S2KSSW_BcTtG1uqhDq&{lHQ^!pN%O}_S}|`ougaEM(o|p+B*|28CJg~3<~Ts+QIydAmoOKR>ennnA*s@& z5t13&ehr-ZNs#rRRMRU{NS}aS5#02$a)sL)}Mf+v={lNus zaT`MV=5udgR-Rb`>(nDiD}2A1=ER|KKyGE0L%2dBYNoxp@o1;p| zwhiT+rs&xnJ9P8*qmLe>n6_q_@S#xjvfK1<=1XEoL28ORuimOd`HTs1UpFE*<_dy^ zD`C4tRnX%_q*jldWlZaKbZ@2YfORHu1HIgOM82OonC#=VoX|{9CD4Ie9x>CjAnNU) z>Vim17|^9d?#GS(dh_)#exqPSXmVlAQB7Duu2C6EEWEsao^@i=>e)aracC3oWV=wZ9cfsCWzKrc)Jk5QHNHZ1?DqZJ+I+;wGrp z1_a%_?dT`3j)qG+a}b(=ieCjd$S=ejb$%XOMmJ%ClPdjxQZ;zpHto09uk!*{COkZD zCkL4M|H0Hi6{s~u+sh&q8#K}=8irm9cYi(U#4SIy@AshJ!h|o5VyjTQ8*M)?I;xht UE~)I?918y0roT%sSI3t4e=JU-i2wiq literal 0 HcmV?d00001 From 034587e3f73863ee2ff9ad1e68d2d7d06c45720c Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Tue, 14 Apr 2026 06:55:47 +0100 Subject: [PATCH 60/69] refactor: restructure fine-tuning to run before main pipeline MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per Vadim Demichev's recommendation, fine-tuning should run as a separate phase BEFORE the main analysis, not mid-pipeline: Phase 0 (when --enable_fine_tuning): 1. INSILICO_LIBRARY_GENERATION (default models) 2. TUNE_PRELIMINARY_ANALYSIS (on --tune_n_files subset) 3. TUNE_ASSEMBLE_LIBRARY (empirical library from subset) 4. FINE_TUNE_MODELS (train RT/IM/fragment models) 5. TUNED_LIBRARY_GENERATION (re-generate with tuned models) Phase 1 (always, the standard pipeline): Uses tuned in-silico library (or default if no fine-tuning) PRELIMINARY_ANALYSIS → ASSEMBLE → INDIVIDUAL → FINAL New parameter: --tune_n_files (default: 3) — number of files for the tuning search subset. Uses Nextflow process aliases (TUNE_PRELIMINARY_ANALYSIS, TUNE_ASSEMBLE_LIBRARY, TUNED_LIBRARY_GENERATION) to avoid duplicate process invocation conflicts. Co-Authored-By: Claude Opus 4.6 (1M context) --- nextflow.config | 3 +- nextflow_schema.json | 10 ++- workflows/dia.nf | 143 +++++++++++++++++++++++++------------------ 3 files changed, 94 insertions(+), 62 deletions(-) diff --git a/nextflow.config b/nextflow.config index 3aa55b4..849c81f 100644 --- a/nextflow.config +++ b/nextflow.config @@ -61,7 +61,8 @@ params { site_ms1_quant = false // add '--site-ms1-quant' for MS1 apex PTM quantification (DIA-NN >= 2.0) // DIA-NN: Model fine-tuning (v2.0+) - enable_fine_tuning = false // Enable model fine-tuning step after empirical library assembly + enable_fine_tuning = false // Enable model fine-tuning before the main analysis + tune_n_files = 3 // Number of files to use for the tuning search (largest/best quality recommended) tune_fr = false // Also fine-tune the fragmentation model (quality-sensitive) tune_lr = null // Fine-tuning learning rate (default: 0.0005) diff --git a/nextflow_schema.json b/nextflow_schema.json index 02dd45b..51fc45b 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -502,10 +502,16 @@ }, "enable_fine_tuning": { "type": "boolean", - "description": "Enable DL model fine-tuning after empirical library assembly. Trains RT/IM models on the empirical library, then regenerates the in-silico library with tuned models. Requires DIA-NN >= 2.0.", + "description": "Enable DL model fine-tuning before the main analysis. Runs a tuning search on a file subset, fine-tunes RT/IM models, then runs the full pipeline with tuned models. Requires DIA-NN >= 2.0.", "fa_icon": "fas fa-brain", "default": false, - "help_text": "When enabled, the pipeline adds a FINE_TUNE_MODELS step after ASSEMBLE_EMPIRICAL_LIBRARY. The empirical library is used to fine-tune the deep learning prediction models for non-standard modifications. A new in-silico library is then generated with the tuned models, and the rest of the pipeline uses this improved library. Cannot be used with --skip_preliminary_analysis." + "help_text": "When enabled, Phase 0 runs before the main pipeline: (1) in-silico library generation with default models, (2) preliminary analysis + assembly on a subset of files (controlled by --tune_n_files), (3) fine-tuning RT/IM models on the resulting empirical library, (4) re-generation of the in-silico library with tuned models. The main pipeline then runs from preliminary analysis using the tuned library." + }, + "tune_n_files": { + "type": "integer", + "description": "Number of files to use for the fine-tuning search. Use the largest/best-quality files.", + "fa_icon": "fas fa-layer-group", + "default": 3 }, "tune_fr": { "type": "boolean", diff --git a/workflows/dia.nf b/workflows/dia.nf index d44af67..fc4154a 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -9,9 +9,11 @@ // include { DIANN_MSSTATS } from '../modules/local/diann/diann_msstats/main' include { PRELIMINARY_ANALYSIS } from '../modules/local/diann/preliminary_analysis/main' +include { PRELIMINARY_ANALYSIS as TUNE_PRELIMINARY_ANALYSIS } from '../modules/local/diann/preliminary_analysis/main' include { ASSEMBLE_EMPIRICAL_LIBRARY } from '../modules/local/diann/assemble_empirical_library/main' -include { INSILICO_LIBRARY_GENERATION } from '../modules/local/diann/insilico_library_generation/main' -include { INSILICO_LIBRARY_GENERATION as TUNED_LIBRARY_GENERATION } from '../modules/local/diann/insilico_library_generation/main' +include { ASSEMBLE_EMPIRICAL_LIBRARY as TUNE_ASSEMBLE_LIBRARY } from '../modules/local/diann/assemble_empirical_library/main' +include { INSILICO_LIBRARY_GENERATION } from '../modules/local/diann/insilico_library_generation/main' +include { INSILICO_LIBRARY_GENERATION as TUNED_LIBRARY_GENERATION } from '../modules/local/diann/insilico_library_generation/main' include { FINE_TUNE_MODELS } from '../modules/local/diann/fine_tune_models/main' include { INDIVIDUAL_ANALYSIS } from '../modules/local/diann/individual_analysis/main' include { FINAL_QUANTIFICATION } from '../modules/local/diann/final_quantification/main' @@ -68,11 +70,6 @@ workflow DIA { error("Model fine-tuning requires DIA-NN >= 2.0. Current version: ${params.diann_version}. Use -profile diann_v2_1_0 or later") } - // Fine-tuning requires the standard preliminary + assembly path (not skip_preliminary_analysis) - if (params.enable_fine_tuning && params.skip_preliminary_analysis) { - error("--enable_fine_tuning cannot be used with --skip_preliminary_analysis. Fine-tuning requires an empirical library from the assembly step.") - } - // Warn about contradictory normalization flags if (!params.normalize && (params.channel_run_norm || params.channel_spec_norm)) { log.warn "Both --normalize false (adds --no-norm) and channel normalization flags are set. " + @@ -107,13 +104,88 @@ workflow DIA { ch_diann_cfg_val = ch_diann_cfg // - // MODULE: SILICOLIBRARYGENERATION + // PHASE 0 (optional): FINE-TUNE DL MODELS // - if (params.speclib != null && params.speclib.toString() != "") { - speclib = channel.from(file(params.speclib, checkIfExists: true)) - } else { + // Per DIA-NN author's recommendation (Vadim Demichev): + // 1. Run InfinDIA on a subset of files with RT/IM filtering set to Relaxed + // 2. Fine-tune models using the resulting empirical library + // 3. Then run the full pipeline from in-silico library generation with tuned models + // + // The tuned models feed into INSILICO_LIBRARY_GENERATION at the very start. + // + ch_tuned_tokens = Channel.empty() + ch_tuned_rt = Channel.empty() + ch_tuned_im = Channel.empty() + + if (params.enable_fine_tuning) { + // Step 0a: Generate a tuning library via InfinDIA on a subset of files + // Use a random subset (or all files if small dataset) for the tuning search + tuning_files = ch_file_preparation_results + .toSortedList{ a, b -> file(a[1]).getName() <=> file(b[1]).getName() } + .flatMap() + .take(params.tune_n_files) + + // Run in-silico library generation first (with default models) for the tuning search INSILICO_LIBRARY_GENERATION(ch_searchdb, ch_diann_cfg_val, ch_is_dda, [], [], []) - speclib = INSILICO_LIBRARY_GENERATION.out.predict_speclib + tune_speclib = INSILICO_LIBRARY_GENERATION.out.predict_speclib + + // Run preliminary analysis on the tuning subset to produce .quant files + TUNE_PRELIMINARY_ANALYSIS(tuning_files.combine(tune_speclib), ch_diann_cfg_val) + + // Assemble the tuning empirical library from the subset + tune_lib_files = tuning_files + .map { result -> result[1] } + .collect( sort: { a, b -> file(a).getName() <=> file(b).getName() } ) + + TUNE_ASSEMBLE_LIBRARY( + tune_lib_files, + ch_experiment_meta, + TUNE_PRELIMINARY_ANALYSIS.out.diann_quant.collect(), + tune_speclib, + ch_diann_cfg_val + ) + ch_software_versions = ch_software_versions + .mix(TUNE_PRELIMINARY_ANALYSIS.out.versions) + .mix(TUNE_ASSEMBLE_LIBRARY.out.versions) + + // Step 0b: Fine-tune models on the empirical library + FINE_TUNE_MODELS( + TUNE_ASSEMBLE_LIBRARY.out.empirical_library, + ch_searchdb, + ch_diann_cfg_val + ) + ch_software_versions = ch_software_versions + .mix(FINE_TUNE_MODELS.out.versions) + + ch_tuned_tokens = FINE_TUNE_MODELS.out.tokens + ch_tuned_rt = FINE_TUNE_MODELS.out.rt_model + ch_tuned_im = FINE_TUNE_MODELS.out.im_model + + // Step 0c: Re-generate in-silico library with tuned models + TUNED_LIBRARY_GENERATION( + ch_searchdb, + ch_diann_cfg_val, + ch_is_dda, + ch_tuned_tokens, + ch_tuned_rt, + ch_tuned_im + ) + ch_software_versions = ch_software_versions + .mix(TUNED_LIBRARY_GENERATION.out.versions) + + speclib = TUNED_LIBRARY_GENERATION.out.predict_speclib + } + + // + // MODULE: INSILICO_LIBRARY_GENERATION (standard, when not fine-tuning) + // + if (!params.enable_fine_tuning) { + if (params.speclib != null && params.speclib.toString() != "") { + speclib = channel.from(file(params.speclib, checkIfExists: true)) + } else { + INSILICO_LIBRARY_GENERATION(ch_searchdb, ch_diann_cfg_val, ch_is_dda, [], [], []) + speclib = INSILICO_LIBRARY_GENERATION.out.predict_speclib + } } if (params.skip_preliminary_analysis) { @@ -195,53 +267,6 @@ workflow DIA { return [ new_meta, ms_file, fasta, library ] } empirical_lib = ASSEMBLE_EMPIRICAL_LIBRARY.out.empirical_library - - // - // MODULE: FINE_TUNE_MODELS (optional) — fine-tune DL prediction models on empirical library - // - // When enabled, the empirical library is used to train better RT/IM/fragment models. - // These tuned models are then used to regenerate the in-silico library, which replaces - // the empirical library for individual analysis — producing better-predicted spectra - // for non-standard modifications. - // - if (params.enable_fine_tuning) { - FINE_TUNE_MODELS( - ASSEMBLE_EMPIRICAL_LIBRARY.out.empirical_library, - ch_searchdb, - ch_diann_cfg_val - ) - ch_software_versions = ch_software_versions - .mix(FINE_TUNE_MODELS.out.versions) - - // Re-generate in-silico library using fine-tuned models - TUNED_LIBRARY_GENERATION( - ch_searchdb, - ch_diann_cfg_val, - ch_is_dda, - FINE_TUNE_MODELS.out.tokens, - FINE_TUNE_MODELS.out.rt_model, - FINE_TUNE_MODELS.out.im_model - ) - ch_software_versions = ch_software_versions - .mix(TUNED_LIBRARY_GENERATION.out.versions) - - // Replace the library used for individual analysis with the tuned one - tuned_speclib = TUNED_LIBRARY_GENERATION.out.predict_speclib - indiv_fin_analysis_in = ch_file_preparation_results - .combine(ch_searchdb) - .combine(tuned_speclib) - .combine(ch_parsed_vals) - .map { meta_map, ms_file, fasta, library, param_string -> - def values = param_string.trim().split(',') - def new_meta = meta_map + [ - mass_acc_ms2 : values[0], - mass_acc_ms1 : values[1], - scan_window : values[2] - ] - return [ new_meta, ms_file, fasta, library ] - } - empirical_lib = tuned_speclib - } } // From a4f54a5ace134206a8535e93ba62eecd6c1d11f8 Mon Sep 17 00:00:00 2001 From: yueqixuan Date: Fri, 17 Apr 2026 20:05:17 +0800 Subject: [PATCH 61/69] fix --- CHANGELOG.md | 2 +- modules/local/diann/fine_tune_models/main.nf | 6 +++--- workflows/dia.nf | 17 +++++++++++------ 3 files changed, 15 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index db463a8..b0143b2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,7 +3,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [2.0.0] bigbio/quantmsdiann — Rome - 2026-04-13 +## [2.0.0] bigbio/quantmsdiann — Rome - 2026-04-17 ### `Added` diff --git a/modules/local/diann/fine_tune_models/main.nf b/modules/local/diann/fine_tune_models/main.nf index c74c808..be798e7 100644 --- a/modules/local/diann/fine_tune_models/main.nf +++ b/modules/local/diann/fine_tune_models/main.nf @@ -14,9 +14,9 @@ process FINE_TUNE_MODELS { output: path "*.dict.txt", emit: tokens - path "*.rt.d0.pt", emit: rt_model - path "*.im.d0.pt", emit: im_model, optional: true - path "*.fr.d0.pt", emit: fr_model, optional: true + path "*.{tuned_rt,rt.d0}.pt", emit: rt_model, optional: true + path "*.{tuned_im,im.d0}.pt", emit: im_model, optional: true + path "*.{tuned_fr,fr.d0}.pt", emit: fr_model, optional: true path "fine_tune.log", emit: log path "versions.yml", emit: versions diff --git a/workflows/dia.nf b/workflows/dia.nf index fc4154a..7c74924 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -241,17 +241,22 @@ workflow DIA { ch_software_versions = ch_software_versions .mix(ASSEMBLE_EMPIRICAL_LIBRARY.out.versions) // Parse calibrated params from the assembly log on the head node + // Format changed in 2.5.0 ch_parsed_vals = ASSEMBLE_EMPIRICAL_LIBRARY.out.log .map { log_file -> + def ms1 = "${params.mass_acc_ms1}" + def ms2 = "${params.mass_acc_ms2}" + def sw = "${params.scan_window}" def match = log_file.text.readLines().find { it.contains("Averaged recommended settings") } if (match) { - def parts = match.trim().split(/\s+/) - def ms2 = parts.size() > 10 ? parts[10].replaceAll(/[^0-9.]/, '') : "${params.mass_acc_ms2}" - def ms1 = parts.size() > 14 ? parts[14].replaceAll(/[^0-9.]/, '') : "${params.mass_acc_ms1}" - def sw = parts.size() > 18 ? parts[18].replaceAll(/[^0-9.]/, '') : "${params.scan_window}" - return "${ms2},${ms1},${sw}" + def ms1_match = match =~ /MS1 accuracy\s*=\s*([0-9.]+)/ + if (ms1_match.find()) ms1 = ms1_match.group(1) + def ms2_match = match =~ /(?:MS2|Mass) accuracy\s*=\s*([0-9.]+)/ + if (ms2_match.find()) ms2 = ms2_match.group(1) + def sw_match = match =~ /Scan window\s*=\s*([0-9.]+)/ + if (sw_match.find()) sw = sw_match.group(1) } - return "${params.mass_acc_ms2},${params.mass_acc_ms1},${params.scan_window}" + return "${ms2},${ms1},${sw}" } indiv_fin_analysis_in = ch_file_preparation_results .combine(ch_searchdb) From e8e6d154d26459fbc7f2db6c418c44f014e2c5e9 Mon Sep 17 00:00:00 2001 From: yueqixuan Date: Fri, 17 Apr 2026 20:13:32 +0800 Subject: [PATCH 62/69] update --- nextflow.config | 2 +- workflows/dia.nf | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/nextflow.config b/nextflow.config index df36984..0d48e94 100644 --- a/nextflow.config +++ b/nextflow.config @@ -61,7 +61,7 @@ params { export_quant = false // add '--export-quant' for fragment-level parquet export (DIA-NN >= 2.0) site_ms1_quant = false // add '--site-ms1-quant' for MS1 apex PTM quantification (DIA-NN >= 2.0) - // DIA-NN: Model fine-tuning (v2.0+) + // DIA-NN: Model fine-tuning (v2.3.2+) enable_fine_tuning = false // Enable model fine-tuning before the main analysis tune_n_files = 3 // Number of files to use for the tuning search (largest/best quality recommended) tune_fr = false // Also fine-tune the fragmentation model (quality-sensitive) diff --git a/workflows/dia.nf b/workflows/dia.nf index 7c74924..142f319 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -66,8 +66,8 @@ workflow DIA { } // Version guard for model fine-tuning - if (params.enable_fine_tuning && VersionUtils.versionLessThan(params.diann_version, '2.0')) { - error("Model fine-tuning requires DIA-NN >= 2.0. Current version: ${params.diann_version}. Use -profile diann_v2_1_0 or later") + if (params.enable_fine_tuning && VersionUtils.versionLessThan(params.diann_version, '2.3.2')) { + error("Model fine-tuning requires DIA-NN >= 2.3.2. Current version: ${params.diann_version}. Use -profile diann_v2_3_2 or later") } // Warn about contradictory normalization flags From 63e86e73252c87faa3385c1b63e3c32ff716ae2a Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Sat, 18 Apr 2026 11:18:05 +0100 Subject: [PATCH 63/69] Update docs/parameters.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/parameters.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/parameters.md b/docs/parameters.md index 2ed864c..0844985 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -27,7 +27,6 @@ This document lists every pipeline parameter organised by category. Default valu | `--reindex_mzml` | boolean | `true` | Force re-indexing of input mzML files at the start of the pipeline for safety. | | `--mzml_statistics` | boolean | `false` | Compute MS1/MS2 statistics from mzML files. Generates `*_ms_info.parquet` files for QC. Bruker `.d` files are always skipped. | | `--mzml_features` | boolean | `false` | Compute MS1-level features during the mzML statistics step. Only available for mzML files. | -| `--convert_dotd` | boolean | `false` | Convert Bruker .d files to mzML format before processing. | ## 4. Search Parameters From cc4252eae9d9c138cc9ae5a15a1b4e68f3f6e28b Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Sat, 18 Apr 2026 11:24:13 +0100 Subject: [PATCH 64/69] Update docs/usage.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/usage.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/usage.md b/docs/usage.md index b85b323..1ef1820 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -367,9 +367,9 @@ DIA-NN's neural networks encode each amino acid and modification as a "token" The fine-tuned models are then used in place of the defaults when generating predicted spectral libraries. > [!NOTE] -> **`--tune-lib` cannot be combined with `--gen-spec-lib` in a single DIA-NN invocation** ([confirmed in DIA-NN #1499](https://github.com/vdemichev/DiaNN/issues/1499)). Fine-tuning and library generation are separate DIA-NN commands. This means the workflow currently requires two pipeline runs. +> **`--tune-lib` cannot be combined with `--gen-spec-lib` in a single DIA-NN invocation** ([confirmed in DIA-NN #1499](https://github.com/vdemichev/DiaNN/issues/1499)). Fine-tuning and library generation are still separate DIA-NN commands, but quantmsdiann can now orchestrate them within a single pipeline run when `--enable_fine_tuning` is used. Integrated fine-tuning requires DIA-NN v2.5.0 or later. The two-run/manual approach below is only needed when integrated fine-tuning is not enabled, or when using an older DIA-NN version that does not support this workflow. -### Current workflow (manual fine-tuning) +### Manual fallback workflow (two-run fine-tuning) **Run 1 — Generate the tuning library:** From 0a2e4a3178e4cb6ad725d105b753278d09fa74f2 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Sat, 18 Apr 2026 11:24:25 +0100 Subject: [PATCH 65/69] Update docs/usage.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/usage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/usage.md b/docs/usage.md index 1ef1820..1913cd1 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -100,8 +100,8 @@ The pipeline uses the same workflow for DDA as DIA — the `--dda` flag is passe - `--reindex_mzml` (default: true) — Re-index mzML files before processing. Disable with `--reindex_mzml false` if files are already indexed. - `--mzml_statistics` (default: false) — Generate mzML statistics (parquet format) for QC. - `--mzml_features` (default: false) — Enable feature detection in mzML statistics. -- `--convert_dotd` (default: false) — Convert Bruker .d files to mzML via tdf2mzml instead of passing natively to DIA-NN. +Bruker `.d` files are supported natively by the current workflow and are passed directly to DIA-NN; there is no `--convert_dotd` preprocessing option. ### Passing Extra Arguments to DIA-NN Use `--extra_args` to pass additional flags to all DIA-NN steps. The pipeline validates and strips flags it manages internally to prevent conflicts. From 7ffac8e90ba23c1d3acbfbd619aed332429c40b4 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Sat, 18 Apr 2026 11:31:45 +0100 Subject: [PATCH 66/69] style: fix prettier formatting in usage.md Add missing blank line before heading. Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/usage.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/usage.md b/docs/usage.md index 1913cd1..e2891fe 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -102,6 +102,7 @@ The pipeline uses the same workflow for DDA as DIA — the `--dda` flag is passe - `--mzml_features` (default: false) — Enable feature detection in mzML statistics. Bruker `.d` files are supported natively by the current workflow and are passed directly to DIA-NN; there is no `--convert_dotd` preprocessing option. + ### Passing Extra Arguments to DIA-NN Use `--extra_args` to pass additional flags to all DIA-NN steps. The pipeline validates and strips flags it manages internally to prevent conflicts. From 04d275d359a9dd9d45914c9c41eda3c25c4a3a07 Mon Sep 17 00:00:00 2001 From: Yasset Perez-Riverol Date: Sat, 18 Apr 2026 11:34:58 +0100 Subject: [PATCH 67/69] docs: update version references for 2.0.0 release - Update reproducibility example from -r 1.0.0 to -r 2.0.0 - Update "Latest DIA-NN version" comment from 2.1.0 to 2.5.0 Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/usage.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/usage.md b/docs/usage.md index e2891fe..222ed63 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -157,7 +157,7 @@ outdir: "./results" Specify the pipeline version when running on your data: ```bash -nextflow run bigbio/quantmsdiann -r 1.0.0 -profile docker --input sdrf.tsv --database db.fasta --outdir results +nextflow run bigbio/quantmsdiann -r 2.0.0 -profile docker --input sdrf.tsv --database db.fasta --outdir results ``` ## Core Nextflow arguments @@ -190,7 +190,7 @@ nextflow run . -profile test_dia,docker --outdir results # DIA with Bruker .d files nextflow run . -profile test_dia_dotd,docker --outdir results -# Latest DIA-NN version (2.1.0) +# Latest DIA-NN version (2.5.0) nextflow run . -profile test_latest_dia,docker --outdir results ``` From 66ecc10c311f91a74f889e44e858a8ecf16c75f6 Mon Sep 17 00:00:00 2001 From: yueqixuan Date: Sat, 18 Apr 2026 21:39:55 +0800 Subject: [PATCH 68/69] fix --- CHANGELOG.md | 2 +- docs/usage.md | 5 ++--- modules/local/diann/insilico_library_generation/meta.yml | 2 +- nextflow_schema.json | 2 +- subworkflows/local/create_input_channel/main.nf | 5 +++-- workflows/dia.nf | 2 +- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b0143b2..ca35846 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,7 +3,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [2.0.0] bigbio/quantmsdiann — Rome - 2026-04-17 +## [2.0.0] bigbio/quantmsdiann — Rome - 2026-04-18 ### `Added` diff --git a/docs/usage.md b/docs/usage.md index 222ed63..52e6378 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -419,9 +419,8 @@ The `--tokens`, `--rt-model`, and `--im-model` flags are passed to all DIA-NN st > [!IMPORTANT] > Use **absolute paths** for model files. The `--parent` flag is blocked by the pipeline (it controls the container's DIA-NN installation path). -### Future: integrated fine-tuning step - -We are exploring adding an optional `FINE_TUNE_MODELS` step directly in the pipeline, which would eliminate the need for two separate runs. The integrated workflow would be: +### Integrated fine-tuning step +The pipeline now includes an optional integrated fine-tuning phase, which eliminates the need for two separate runs. You can enable this feature by using the `--enable_fine_tuning` flag. The integrated workflow is: ``` INSILICO_LIBRARY → PRELIMINARY_ANALYSIS → ASSEMBLE_EMPIRICAL_LIBRARY diff --git a/modules/local/diann/insilico_library_generation/meta.yml b/modules/local/diann/insilico_library_generation/meta.yml index 27fee04..c1af0db 100644 --- a/modules/local/diann/insilico_library_generation/meta.yml +++ b/modules/local/diann/insilico_library_generation/meta.yml @@ -21,7 +21,7 @@ input: pattern: "*.cfg" - is_dda: type: boolean - description: Whether DDA mode is enabled (auto-detected from SDRF or set via --diann_dda) + description: Whether DDA mode is enabled (auto-detected from SDRF or set via --dda) output: - predict_speclib: type: file diff --git a/nextflow_schema.json b/nextflow_schema.json index 5f2f615..31823d7 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -509,7 +509,7 @@ }, "enable_fine_tuning": { "type": "boolean", - "description": "Enable DL model fine-tuning before the main analysis. Runs a tuning search on a file subset, fine-tunes RT/IM models, then runs the full pipeline with tuned models. Requires DIA-NN >= 2.0.", + "description": "Enable DL model fine-tuning before the main analysis. Runs a tuning search on a file subset, fine-tunes RT/IM models, then runs the full pipeline with tuned models. Requires DIA-NN >= 2.3.2.", "fa_icon": "fas fa-brain", "default": false, "help_text": "When enabled, Phase 0 runs before the main pipeline: (1) in-silico library generation with default models, (2) preliminary analysis + assembly on a subset of files (controlled by --tune_n_files), (3) fine-tuning RT/IM models on the resulting empirical library, (4) re-generation of the in-silico library with tuned models. The main pipeline then runs from preliminary analysis using the tuned library." diff --git a/subworkflows/local/create_input_channel/main.nf b/subworkflows/local/create_input_channel/main.nf index d2ca4d3..1a8f144 100644 --- a/subworkflows/local/create_input_channel/main.nf +++ b/subworkflows/local/create_input_channel/main.nf @@ -18,8 +18,6 @@ workflow CREATE_INPUT_CHANNEL { ch_expdesign = SDRF_PARSING.out.ch_expdesign ch_diann_cfg = SDRF_PARSING.out.ch_diann_cfg - def enzymes = new HashSet() - def files = new HashSet() // Extract experiment_id from the SDRF filename ch_experiment_id = ch_sdrf.map { sdrf_file -> file(sdrf_file).baseName } @@ -101,6 +99,9 @@ def create_meta_channel_grouped(String filestr, List rows, Map wrapper) { meta.enzyme = enzymes ? enzymes[0] : null def fixedMods = rows.collect { it.FixedModifications?.toString()?.trim() }.findAll { it }.unique() + if (fixedMods.size() > 1) { + log.error("SDRF conflict: Multiple FixedModifications (${fixedMods.join(',')}) found for file ${meta.id}. Please fix the SDRF.") + } meta.fixedmodifications = fixedMods ? fixedMods[0] : null // Validate required SDRF columns diff --git a/workflows/dia.nf b/workflows/dia.nf index 142f319..839e7e7 100644 --- a/workflows/dia.nf +++ b/workflows/dia.nf @@ -119,7 +119,7 @@ workflow DIA { if (params.enable_fine_tuning) { // Step 0a: Generate a tuning library via InfinDIA on a subset of files - // Use a random subset (or all files if small dataset) for the tuning search + // Use a deterministic subset (or all files if small dataset) for the tuning search tuning_files = ch_file_preparation_results .toSortedList{ a, b -> file(a[1]).getName() <=> file(b[1]).getName() } .flatMap() From 7c1099e6e9386c3f288ac7ef36336ae2d8dc59d5 Mon Sep 17 00:00:00 2001 From: yueqixuan Date: Sat, 18 Apr 2026 21:42:40 +0800 Subject: [PATCH 69/69] update --- docs/usage.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/usage.md b/docs/usage.md index 52e6378..2abcd2e 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -420,6 +420,7 @@ The `--tokens`, `--rt-model`, and `--im-model` flags are passed to all DIA-NN st > Use **absolute paths** for model files. The `--parent` flag is blocked by the pipeline (it controls the container's DIA-NN installation path). ### Integrated fine-tuning step + The pipeline now includes an optional integrated fine-tuning phase, which eliminates the need for two separate runs. You can enable this feature by using the `--enable_fine_tuning` flag. The integrated workflow is: ```