Skip to content

Deepvariant subworkflow (issue #614) #651

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 5 commits into
base: dev
Choose a base branch
from
Draft

Deepvariant subworkflow (issue #614) #651

wants to merge 5 commits into from

Conversation

fa2k
Copy link
Contributor

@fa2k fa2k commented Dec 6, 2024 •
edited
Loading

This PR splits DeepVariant into three processes, to optimise the resource usage. (#614)

It doesn't support GPU out of the box, but I have added usage documentation for how to enable it.

PR checklist

  • This comment contains a description of changes (with reason).
  • [na] If you've fixed a bug or added code that should be tested, add tests!
  • [na] If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • [na] If necessary, also make a PR on the nf-core/raredisease branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nextflow run . -profile test,singularity --outdir <OUTDIR>).
  • Ensure the test suite passes (nextflow run . -profile test_one_sample,singularity --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • [na] Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • [na] README.md is updated (including new tool citations and authors/contributors).

@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 6, 2024 •
edited
Loading

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit e2963a9

+| ✅ 201 tests passed       |+
#| ❔   7 tests were ignored |#
!| ❗   2 tests had warnings |!

❗ Test warnings:

❔ Tests ignored:

  • files_exist - File is ignored: conf/modules.config
  • files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: assets/nf-core-raredisease_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-raredisease_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-raredisease_logo_dark.png
  • modules_config - modules_config

✅ Tests passed:

Run details

  • nf-core/tools version 3.0.2
  • Run at 2024-12-06 12:32:25

@ramprasadn
Copy link
Collaborator

Hey @fa2k! How's this one coming along? I've got some time and can jump in if you're busy. 😄

@fa2k
Copy link
Contributor Author

fa2k commented Mar 18, 2025

Hi @ramprasadn it's nice that you're looking into this, but the subworkflow solution is proving to be a bit troublesome. DeepVariant made some large changes to the pipeline structure with version 1.8.0, but me and @fellen31 have updated the subworkflow so it works again. The work to integrate new versions was a bit too much, both for the subworkflow itself, and for the pipelines using it (none at the moment AFAIK), as they must update the ext.args config.

Furthermore DeepVariant has added a "fast pipeline" with the new version, which is an alternative way to optimise the resource usage (instead of the subworkflow approach). The fast pipeline is not very user friendly at the moment, but could be improved. I'm supposed to do some benchmarking of the subworkflow vs the module, but I won't have time to do it any time soon, and I didn't plan to also include the "fast pipeline".

It may be smart to wait for the next DeepVariant release, to get a feel for the amount of changes needed for the subworkflow, and whether the fast pipeline could be an alternative.

@ramprasadn
Copy link
Collaborator

Okay, I will just update the module to v1.8.0 so we can take advantage of the runtime reduction achieved with the small model for now. Like you said, lets wait for the for the next release to see what DeepVariant comes up with and then decide on how to move forward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@fa2k @ramprasadn