StepRun state management in Pipeline runtime

[WiktorMadejski]

Consider python azureml-sdk Pipeline with three PythonScriptStep [step_1, step_a, step_b] and edges:

step_a.run_after(step_1)
step_b.run_after(step_1)

where in step_1 runtime "a condition" is checked based on which we want to cancel running step_b but want to continue running pipeline (ie. step_b). Please find a graph appended.

I tried to implement it in two ways:

from azureml.core import Run
from azureml.pipeline.core.run import PipelineRun

1. Deleting node in graph (Is this API executed on instance of Graph object for only this Pipeline run?)

if a_condition:
run = Run.get_context()
pipeline_run = PipelineRun(run.experiment, run.id)
graph=pipeline_run.get_graph()
for n in graph.nodes:
if n.name =='step_1':
graph.delete_node(n.node_id)

Nonetheless:
azureml/pipeline/core/graph.py in delete_node(self, node_id)
raise NotImplementedError

2. Having some higher level API in the picture of:
   from azureml.core import Run
   from azureml.pipeline.core.run import PipelineRun
   if a_condition:
   run = Run.get_context()
   pipeline_run = PipelineRun(run.experiment, run.id)
   step_a_run=pipeline_run.find_step_run(step_a_name)
   step_a_run.cancel() #alternatively .fail()
   step_a_run is an empty list as step_a have not started to run. Besides StepRun inherits cancel() after Run so I it cancels entire pipeline.

Is there some work planned in Azure ML Python SDK for Pipeline(or its Graph) steps management during its runtime? Especially this scenario to cancel running selected before it starts to run/prepare?

Its useful when you have highly automated pipelines.

[shbijlan]

@WiktorMadejski Thanks for sharing feedback. Currently, we don't have this capability while constructing pipelines, but I have opened a feature request to our team. If approved, the feature will be implemented as part of future planning and release cycles.