add align, chain, and net workflow #818

martenson · 2025-04-28T14:06:21Z

FOR CONTRIBUTOR:

I have read the Adding workflows guidelines
License permits unrestricted use (educational + commercial)
Please also take note of the reviewer guidelines below to facilitate a smooth review process.

FOR REVIEWERS:

.dockstore.yml: file is present and aligned with creator metadata in workflow. ORCID identifiers are strongly encouraged in creator metadata. The .dockstore.yml file is required to run tests
Workflow is sufficiently generic to be used with lab data and does not hardcode sample names, reference data and can be run without reading an accompanying tutorial.
In workflow: annotation field contains short description of what the workflow does. Should start with This workflow does/runs/performs … xyz … to generate/analyze/etc …
In workflow: workflow inputs and outputs have human readable names (spaces are fine, no underscore, dash only where spelling dictates it), no abbreviation unless it is generally understood. Altering input or output labels requires adjusting these labels in the the workflow-tests.yml file as well
In workflow: name field should be human readable (spaces are fine, no underscore, dash only where spelling dictates it), no abbreviation unless generally understood
Workflow folder: prefer dash (-) over underscore (_), prefer all lowercase. Folder becomes repository in iwc-workflows organization and is included in TRS id
Readme explains what workflow does, what are valid inputs and what outputs users can expect. If a tutorial or other resources exist they can be linked. If a similar workflow exists in IWC readme should explain differences with existing workflow and when one might prefer one workflow over another
Changelog contains appropriate entries
Large files (> 100 KB) are uploaded to zenodo and location urls are used in test file

workflows/genome-alignment/align-chain-net/.dockstore.yml

mvdbeek · 2025-04-28T14:41:32Z

Is there a non-GPU flag we can use in testing ? Otherwise I'm afraid we'll have to run the GPU step elsewhere, but I worry that's going to be a pain.

martenson · 2025-04-28T14:46:04Z

@mvdbeek I am not sure if there is some non-gpu hack, I'll have a closer look. The test runs fine against vgp.usegalaxy.org but that is likely not very helpful here.

edit: no hack found

workflows/genome-alignment/align-chain-net/align.ga

mvdbeek · 2025-04-28T15:03:19Z

I think we'll have to invent some way to say that a workflow needs to run against a live instance when it really can't run in CI. Or perhaps the better way is to setup a "only for IWC" gpu pulsar destination. Then we only launch the step that needs special stuff against the remote resource.

github-actions · 2025-05-09T14:06:19Z

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	1
Passed	0
Error	1
Failure	0
Skipped	0

Errored Tests

❌ align.ga_0

Execution Problem:

Failed to run workflow, at least one job is in [error] state.

Workflow invocation details

Invocation Messages

Steps

Step 1: TARGET_fasta_index:
- step_state: scheduled
Step 2: QUERY_fasta_index:
- step_state: scheduled

Step 11: chainprenet_step:

step_state: scheduled

Jobs

Job 1:

Job state is new

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"36be61022cde11f0a4526045bd08c38f"`
chromInfo	`"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`
inclHap	`false`
pad	`None`
query_reference_index_source	`{"__current_case__": 1, "in_que_ref_index": {"values": [{"id": 2, "src": "hda"}]}, "query_reference_index_source_selector": "history"}`
target_reference_index_source	`{"__current_case__": 1, "in_tar_ref_index": {"values": [{"id": 1, "src": "hda"}]}, "target_reference_index_source_selector": "history"}`

Step 12: chainsort_step:

step_state: scheduled

Jobs

Job 1:

Job state is new

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"36be61022cde11f0a4526045bd08c38f"`
chromInfo	`"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`
sort_by	`""`

Step 13: chainne_step:

step_state: scheduled

Jobs

Job 1:

Job state is new

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"36be61022cde11f0a4526045bd08c38f"`
chromInfo	`"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`
inclHap	`false`
minFill	`None`
minScore	`None`
minSpace	`None`
query_reference_index_source	`{"__current_case__": 1, "in_que_ref_index": {"values": [{"id": 2, "src": "hda"}]}, "query_reference_index_source_selector": "history"}`
target_reference_index_source	`{"__current_case__": 1, "in_tar_ref_index": {"values": [{"id": 1, "src": "hda"}]}, "target_reference_index_source_selector": "history"}`
verbose	`None`

Step 14: netsyntenic_step:

step_state: scheduled

Jobs

Job 1:

Job state is new

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"36be61022cde11f0a4526045bd08c38f"`
chromInfo	`"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`

Step 15: netfilter_step:

step_state: scheduled

Jobs

Job 1:

Job state is new

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"36be61022cde11f0a4526045bd08c38f"`
chromInfo	`"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`
minGap	`None`
synteny	`{"__current_case__": 1, "minSynAli": null, "minSynScore": null, "minSynSize": null, "syn_filter": "filtersyn", "syntype": "-syn"}`

Step 16: nettoaxt_step:

step_state: scheduled

Jobs

Job 1:

Job state is new

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"36be61022cde11f0a4526045bd08c38f"`
chromInfo	`"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`
maxGap	`None`
noSplit	`false`
qChain	`false`

Step 17: axttomaf_step:

step_state: scheduled

Jobs

Job 1:

Job state is new

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"36be61022cde11f0a4526045bd08c38f"`
chromInfo	`"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`
q_prefix	`None`
query_reference_index_source	`{"__current_case__": 1, "in_que_ref_index": {"values": [{"id": 2, "src": "hda"}]}, "query_reference_index_source_selector": "history"}`
score	`false`
scoreZero	`false`
t_prefix	`None`
target_reference_index_source	`{"__current_case__": 1, "in_tar_ref_index": {"values": [{"id": 1, "src": "hda"}]}, "target_reference_index_source_selector": "history"}`

Step 3: TARGET_Sequence:
- step_state: scheduled
Step 4: QUERY_Sequence:
- step_state: scheduled

Step 5: target_2bit_step:

step_state: scheduled

Jobs

Job 1:

Job state is ok

Command Line:

faToTwoBit '/tmp/tmpl552jumv/files/1/d/d/dataset_1ddd3eaf-895e-4cc5-b885-4d80f33168b4.dat' '/tmp/tmpl552jumv/job_working_directory/000/6/outputs/dataset_60a339ec-e8cd-47b5-9fec-b727e246f364.dat'

Exit Code:

```
0
```

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"fasta"`
__workflow_invocation_uuid__	`"36be61022cde11f0a4526045bd08c38f"`
chromInfo	`"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`

Step 6: kegalign_step:

step_state: scheduled

Jobs

Job 1:

Job state is error

Command Line:

mkdir -p "$(pwd)/work" && faToTwoBit <(gzip -cdfq '/tmp/tmpl552jumv/files/a/5/c/dataset_a5c31a61-a7c4-4514-af48-fe1da054d7d0.dat') "$(pwd)/work/ref.2bit" && faToTwoBit <(gzip -cdfq '/tmp/tmpl552jumv/files/7/9/d/dataset_79d24603-74b4-429f-aded-95f07a60488d.dat') "$(pwd)/work/query.2bit" &&   python '/tmp/shed_dir/toolshed.g2.bx.psu.edu/repos/richard-burhans/kegalign/544ea81a6914/kegalign/runner.py' --output-type tarball --output-file '/tmp/tmpl552jumv/job_working_directory/000/7/outputs/dataset_02c82e58-8f24-48e7-8e58-14ba1ed41b70.dat' --diagonal-partition --num-cpu ${GALAXY_SLOTS:-2} --tool_directory '/tmp/shed_dir/toolshed.g2.bx.psu.edu/repos/richard-burhans/kegalign/544ea81a6914/kegalign' '/tmp/tmpl552jumv/files/a/5/c/dataset_a5c31a61-a7c4-4514-af48-fe1da054d7d0.dat' '/tmp/tmpl552jumv/files/7/9/d/dataset_79d24603-74b4-429f-aded-95f07a60488d.dat'   --strand 'both'     --seed '12of19' --step '1'   --xdrop '910' --hspthresh '3000'   --ydrop '9430'   --format 'axt'   --wga_chunk_size '250000' --lastz_interval_size '10000000' --seq_block_size '400000000' --num_gpu '-1'   && python '/tmp/shed_dir/toolshed.g2.bx.psu.edu/repos/richard-burhans/kegalign/544ea81a6914/kegalign/package_output.py' --tool_directory '/tmp/shed_dir/toolshed.g2.bx.psu.edu/repos/richard-burhans/kegalign/544ea81a6914/kegalign' --format_selector 'axt'

Exit Code:

```
1
```

Standard Error:

cfg.num_threads = 1
Using 1 threads
Error: No GPU device found!
Error: kegalign exited with returncode 1

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"36be61022cde11f0a4526045bd08c38f"`
chromInfo	`"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`
gapped_extension_options	`{"gappedthresh": null, "notrivial": false, "ydrop": "9430"}`
output_options	`{"format": {"__current_case__": 4, "axt_type": "axt", "format_selector": "axt"}}`
scoring_options	`{"ambiguous_params": {"__current_case__": 0, "set_ambiguous_params_selector": "false"}, "ambiguous_selector": "x", "scoring": null}`
seeding_options	`{"notransition": false, "seed": {"__current_case__": 0, "seed_selector": "12of19"}, "step": "1"}`
sequence_options	`{"strand_selector": "both"}`
system_options	`{"debug": false, "lastz_interval_size": "10000000", "num_gpu": "-1", "seq_block_size": "400000000", "wga_chunk_size": "250000"}`
ungapped_extension_options	`{"hspthresh": "3000", "noentropy": false, "xdrop": "910"}`

Step 7: query_2bit_step:

step_state: scheduled

Jobs

Job 1:

Job state is ok

Command Line:

faToTwoBit '/tmp/tmpl552jumv/files/4/1/5/dataset_415d5664-ac16-4346-a410-651ca2d7d4b0.dat' '/tmp/tmpl552jumv/job_working_directory/000/9/outputs/dataset_78d6bb60-42d1-4027-b000-6668fd70669d.dat'

Exit Code:

```
0
```

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"fasta"`
__workflow_invocation_uuid__	`"36be61022cde11f0a4526045bd08c38f"`
chromInfo	`"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`

Step 8: batched_lastz_step:

step_state: scheduled

Jobs

Job 1:

Job state is paused

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"36be61022cde11f0a4526045bd08c38f"`
chromInfo	`"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`

Step 9: axtchain_step:

step_state: scheduled

Jobs

Job 1:

Job state is new

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"36be61022cde11f0a4526045bd08c38f"`
chromInfo	`"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`
linear_gap_options	`{"__current_case__": 0, "linear_gap": "medium"}`
minScore	`None`
scoreScheme	`None`

Step 10: chainantirepeat_step:

step_state: scheduled

Jobs

Job 1:

Job state is new

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"36be61022cde11f0a4526045bd08c38f"`
chromInfo	`"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`
minScore	`None`
noCheckScore	`None`

Other invocation details
- error_message
  - Failed to run workflow, at least one job is in [error] state.
- history_id
  - 3b47e5167388cd76
- history_state
  - error
- invocation_id
  - 3b47e5167388cd76
- invocation_state
  - scheduled
- workflow_id
  - 3b47e5167388cd76

martenson added 2 commits April 28, 2025 16:01

wip alignment

b1252bc

metafiles

f7f0470

mvdbeek reviewed Apr 28, 2025

View reviewed changes

workflows/genome-alignment/align-chain-net/.dockstore.yml Outdated Show resolved Hide resolved

This comment was marked as outdated.

Sign in to view

mvdbeek mentioned this pull request Apr 28, 2025

Don't escape non-ascii unicode characters in json.dumps galaxyproject/planemo#1523

Merged

martenson commented Apr 28, 2025

View reviewed changes

workflows/genome-alignment/align-chain-net/align.ga Show resolved Hide resolved

mvdbeek closed this in galaxyproject/planemo#1523 Apr 29, 2025

martenson reopened this Apr 29, 2025

start readme

b3bb3dc

This comment was marked as outdated.

Sign in to view

martenson marked this pull request as ready for review May 9, 2025 13:49

fix ma name

a0edcb1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add align, chain, and net workflow #818

add align, chain, and net workflow #818

Uh oh!

martenson commented Apr 28, 2025 •

edited

Loading

Uh oh!

Uh oh!

This comment was marked as outdated.

mvdbeek commented Apr 28, 2025

Uh oh!

martenson commented Apr 28, 2025 •

edited

Loading

Uh oh!

Uh oh!

mvdbeek commented Apr 28, 2025

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

github-actions bot commented May 9, 2025

Workflow invocation details

Uh oh!

Uh oh!

add align, chain, and net workflow #818

Are you sure you want to change the base?

add align, chain, and net workflow #818

Uh oh!

Conversation

martenson commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

mvdbeek commented Apr 28, 2025

Uh oh!

martenson commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

mvdbeek commented Apr 28, 2025

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

github-actions bot commented May 9, 2025

Test Results (powered by Planemo)

Test Summary

Workflow invocation details

Uh oh!

Uh oh!

martenson commented Apr 28, 2025 •

edited

Loading

martenson commented Apr 28, 2025 •

edited

Loading